Papers

Field

Top tools here

All tools

billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
TanStack AI+68% downloads · 1M/wk+67.7%
LMDeploy+109% downloads · 10.8k/wk+109.0%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%

Top tools here

Open problems in Frontier LLMs

Evaluating decision models beyond accuracy1 paper raises it · 4 address it
Aggregate accuracy hides failures on controlled variations, calibration, and confidence reliability, which are critical for safe deployment in LLM systems.
- DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- Jeva: System One Models from First Principlesaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient, structured decision-making for agents3 papers raise it · 3 address it
Generative reasoning is slow and unreliable for short, repeated decisions, while fixed-depth decision models lack adaptability.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Long-horizon context retention and reuse2 papers raise it · 3 address it
Agents lose critical information when context windows are exceeded, and textual summaries alone may not preserve decision-relevant details.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cortex: Consolidating Context into Parametric Memoryaddresses
Real-world web search and evidence integration1 paper raises it · 2 address it
Agents struggle to autonomously explore, access, and integrate heterogeneous, interactive web sources to answer complex factual queries.
- Deep Search in Real-World Web Environmentsraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Cost-effective fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs can verify review comments but require reading entire papers for each check, making the process expensive and slow.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

Chaojinhua IntelligenceSeed · Oct 28undisclosed
HexSeries A · Oct 27$100M
Chaoqiumian TechnologySeed · Oct 26undisclosed
HypersphereSeed · Oct 26undisclosed
Star TherapeuticsSeries B+ · Oct 23$195M

Recently funded

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers48 this week
language models llms · large language models · language models301 papers46 this week
memory construction · persistent memory · long term memory285 papers37 this week
large language models · machine translation · multilingual236 papers33 this week
generative recommendation · sequential recommendation · personalized171 papers28 this week
deep research · scientific literature · citations155 papers25 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.AIOct 3
Asking Earns Nothing: Scoring the Decision to Act in BFCL Multi-Turn
Yangze Liu, Zhongyi Han
An agent that lacks the information it needs should ask rather than act, and the task definitions of agent leaderboards say so. BFCL multi-turn builds two of its four categories around a turn on which the model is supposed to ask, and its scorer never looks at that turn: the gold trajectory there is empty, the checker skips it, and the scripted user cannot answer, so asking earns nothing, guessing costs nothing on that turn, and asking twice loses the item. The benchmark also contains the control experiment for that decision. A should-ask item is a base item with one piece of information removed from one turn, so the same request appears twice at the same turn index, once complete and once not: on the first the model should make the call that changes the world, on the second it should ask. We score one decision per pair, whether the model attempted a world-changing call on that turn, read off the stored trajectories with no LLM judge; acting always and asking always both score 50. On the 223 pairs that pose this decision, gpt-5.4 attempts the call on 83.4% of the complete turns and holds back on 78.0% of the incomplete ones, the best decision accuracy of seven models at 80.7%; on…
PDF arXiv
arXiv cs.CLOct 3
Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Define
Saurabh Kumar Singh, Yogeshwar Singh Dadwhal, Malhar Vedak
Post-training quantization to the GGUF format's mixed-precision K-quants is commonly how open-weight language models reach consumer hardware, yet its effect on fine-grained lexical competence is uncharacterized. We audit 27 quantized artifacts across 13 families and four architecture backbones, 0.35B-14B parameters, evaluated down their published ladder to Q2_K (about 2.6 bits per weight), on 429 frequency-validated rare English words under two probes: surface inclusion of a prompt-supplied word and its one-sentence definition, scored by a tiered multi-synonym matcher, its error measured by a blind LLM-judge census of every definition, with human verification. Three regimes emerge at Q2: total collapse into unusable builds, severe semantic dissociation in sub-2B models, and mostly robust preservation above about 3B. In every sub-2B artifact, definitions fall 20-67% below the artifact's baseline, typically several times the inclusion loss. Two controls separate rarity from task difficulty: within the rare set, loss rises with rarity in six of seven sub-2B artifacts, and on a 100-word common-word set rare words lose more than common words in all eight, significantly in six.…
PDF arXiv
arXiv cs.CLOct 3
Ideological Stance Detection in a Low-Resource Language: Polarization in Bangladeshi Public vs Private University Discourse on Social Media
Safaruzzaman Shovo, Monowar Islam, Asif Hossain, Sameya Akhter, Md. Shamsul Islam
Public vs. private universities is a debatable issue, and it creates polarization on social media in Bangladesh. Debate on quality, jobs, and prestige is passionate among the students, parents, and graduates, the majority of whom speak Bengali, a low-resource language. To measure this polarization, this paper introduces a manually annotated dataset of 4,060 Bengali comments labeled as Pro-Public, Pro-Private, or Neutral. We evaluated the quality of our annotations by Fleiss's Kappa agreement that was 0.89, corresponding to a high agreement among annotators. The classical ML (SVM, Random Forest, XGBoost), BiLSTM network, hybrid BanglaBERT+XGBoost models and the state-of-the-art zero-shot LLMs (Claude Sonnet 4, DeepSeek-V3.1, Llama 4 Maverick, Kimi K2 Thinking, Qwen3-235B Thinking) models are evaluated. The accuracy of BanglaBERT+XGBoost is 91.81% and macro F1 score is 91.70%, which is higher than all the supervised baselines. The zero-shot Llama 4 Maverick Thinking achieves a macro F1 of 0.931 (overall accuracy of 93.31%) without any fine-tuning. All machine learning (ML), deep machine learning (DL) and transformer models were outperformed by the zero-shot Llama 4 Maverick model.…
PDF arXiv
arXiv cs.CLOct 3
Boundaries Agree, Labels Do Not: Intra-Annotator Dynamics as a Kind of Training Data
Marharyta Shvets
Data quality now matters as much as compute for training language models. Much training data comes from human annotation of text, and interpretive annotation has no ground truth that could settle what is "accurate". Two lines of work respond to this. One combines annotators into a "ground truth" and measures how well they agree with each other; the other treats their disagreement as a signal. Both compare different people at one point in time. We measure something else: how well one reader reproduces their own reading of the same text over time. One expert human reader and three LLM families segmented three Sumerian myths and labelled the causal function of each segment with one of seven states. Across runs months apart, the human cut the text in much the same places but named the segments differently, in every myth. The models show no such consistent pattern: their gap between the two layers is positive in some myths and negative in others, and its size varies. The human's label changes are not random: the runs go through much the same functions but start them one step apart, while model runs start them at the same places. We argue that this pattern is a usable measure of data…
PDF arXiv
arXiv cs.AIOct 3
ShadowMiner v1 - An Experience Report on Implementing and Measuring a Problem-and-Hypothesis Discovery Engine
Jinhyuk Choi
ShadowMiner v1 is a system that automatically discovers research problems and generates hypotheses from AI papers. It is a nine-stage pipeline. It structures documents into a knowledge graph and finds graph gaps in it - structural blind spots in research. These graph gaps are included in the LLM generation prompt. Each generated hypothesis is then verified by checking whether it is already covered by existing research, scoring its quality, and checking that the facts it relies on are accurately drawn from its sources. This report does not propose a new generation or evaluation technique. It describes our experience of implementing and applying ideas from prior work, and measuring whether each one actually contributed.
PDF arXiv
arXiv cs.AIOct 3
Suppressing Pressure, Amplifying Evidence: Self-Guided Attention Steering to Mitigate Sycophancy and Stubbornness
Yinghao He, Mengyu Xu, Haixiang Sun, Donghan Li, Yibo Wang +6
Reliable language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade-off, we introduce CoPE-Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training-free framework that uses the model's own judgments to identify relevant tokens, suppressing user pressure and amplifying contextual information through token-level attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases…
PDF arXiv

Papers

Recently funded

Market

Chaojinhua IntelligenceSeed · Oct 28undisclosed
HexSeries A · Oct 27$100M
Chaoqiumian TechnologySeed · Oct 26undisclosed
HypersphereSeed · Oct 26undisclosed
Star TherapeuticsSeries B+ · Oct 23$195M

Top tools here

All tools

billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
TanStack AI+68% downloads · 1M/wk+67.7%
LMDeploy+109% downloads · 10.8k/wk+109.0%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers48 this week
language models llms · large language models · language models301 papers46 this week
memory construction · persistent memory · long term memory285 papers37 this week
large language models · machine translation · multilingual236 papers33 this week
generative recommendation · sequential recommendation · personalized171 papers28 this week
deep research · scientific literature · citations155 papers25 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.AIOct 3
Asking Earns Nothing: Scoring the Decision to Act in BFCL Multi-Turn
Yangze Liu, Zhongyi Han
An agent that lacks the information it needs should ask rather than act, and the task definitions of agent leaderboards say so. BFCL multi-turn builds two of its four categories around a turn on which the model is supposed to ask, and its scorer never looks at that turn: the gold trajectory there is empty, the checker skips it, and the scripted user cannot answer, so asking earns nothing, guessing costs nothing on that turn, and asking twice loses the item. The benchmark also contains the control experiment for that decision. A should-ask item is a base item with one piece of information removed from one turn, so the same request appears twice at the same turn index, once complete and once not: on the first the model should make the call that changes the world, on the second it should ask. We score one decision per pair, whether the model attempted a world-changing call on that turn, read off the stored trajectories with no LLM judge; acting always and asking always both score 50. On the 223 pairs that pose this decision, gpt-5.4 attempts the call on 83.4% of the complete turns and holds back on 78.0% of the incomplete ones, the best decision accuracy of seven models at 80.7%; on…
PDF arXiv
arXiv cs.CLOct 3
Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Define
Saurabh Kumar Singh, Yogeshwar Singh Dadwhal, Malhar Vedak
Post-training quantization to the GGUF format's mixed-precision K-quants is commonly how open-weight language models reach consumer hardware, yet its effect on fine-grained lexical competence is uncharacterized. We audit 27 quantized artifacts across 13 families and four architecture backbones, 0.35B-14B parameters, evaluated down their published ladder to Q2_K (about 2.6 bits per weight), on 429 frequency-validated rare English words under two probes: surface inclusion of a prompt-supplied word and its one-sentence definition, scored by a tiered multi-synonym matcher, its error measured by a blind LLM-judge census of every definition, with human verification. Three regimes emerge at Q2: total collapse into unusable builds, severe semantic dissociation in sub-2B models, and mostly robust preservation above about 3B. In every sub-2B artifact, definitions fall 20-67% below the artifact's baseline, typically several times the inclusion loss. Two controls separate rarity from task difficulty: within the rare set, loss rises with rarity in six of seven sub-2B artifacts, and on a 100-word common-word set rare words lose more than common words in all eight, significantly in six.…
PDF arXiv
arXiv cs.CLOct 3
Ideological Stance Detection in a Low-Resource Language: Polarization in Bangladeshi Public vs Private University Discourse on Social Media
Safaruzzaman Shovo, Monowar Islam, Asif Hossain, Sameya Akhter, Md. Shamsul Islam
Public vs. private universities is a debatable issue, and it creates polarization on social media in Bangladesh. Debate on quality, jobs, and prestige is passionate among the students, parents, and graduates, the majority of whom speak Bengali, a low-resource language. To measure this polarization, this paper introduces a manually annotated dataset of 4,060 Bengali comments labeled as Pro-Public, Pro-Private, or Neutral. We evaluated the quality of our annotations by Fleiss's Kappa agreement that was 0.89, corresponding to a high agreement among annotators. The classical ML (SVM, Random Forest, XGBoost), BiLSTM network, hybrid BanglaBERT+XGBoost models and the state-of-the-art zero-shot LLMs (Claude Sonnet 4, DeepSeek-V3.1, Llama 4 Maverick, Kimi K2 Thinking, Qwen3-235B Thinking) models are evaluated. The accuracy of BanglaBERT+XGBoost is 91.81% and macro F1 score is 91.70%, which is higher than all the supervised baselines. The zero-shot Llama 4 Maverick Thinking achieves a macro F1 of 0.931 (overall accuracy of 93.31%) without any fine-tuning. All machine learning (ML), deep machine learning (DL) and transformer models were outperformed by the zero-shot Llama 4 Maverick model.…
PDF arXiv
arXiv cs.CLOct 3
Boundaries Agree, Labels Do Not: Intra-Annotator Dynamics as a Kind of Training Data
Marharyta Shvets
Data quality now matters as much as compute for training language models. Much training data comes from human annotation of text, and interpretive annotation has no ground truth that could settle what is "accurate". Two lines of work respond to this. One combines annotators into a "ground truth" and measures how well they agree with each other; the other treats their disagreement as a signal. Both compare different people at one point in time. We measure something else: how well one reader reproduces their own reading of the same text over time. One expert human reader and three LLM families segmented three Sumerian myths and labelled the causal function of each segment with one of seven states. Across runs months apart, the human cut the text in much the same places but named the segments differently, in every myth. The models show no such consistent pattern: their gap between the two layers is positive in some myths and negative in others, and its size varies. The human's label changes are not random: the runs go through much the same functions but start them one step apart, while model runs start them at the same places. We argue that this pattern is a usable measure of data…
PDF arXiv
arXiv cs.AIOct 3
ShadowMiner v1 - An Experience Report on Implementing and Measuring a Problem-and-Hypothesis Discovery Engine
Jinhyuk Choi
ShadowMiner v1 is a system that automatically discovers research problems and generates hypotheses from AI papers. It is a nine-stage pipeline. It structures documents into a knowledge graph and finds graph gaps in it - structural blind spots in research. These graph gaps are included in the LLM generation prompt. Each generated hypothesis is then verified by checking whether it is already covered by existing research, scoring its quality, and checking that the facts it relies on are accurately drawn from its sources. This report does not propose a new generation or evaluation technique. It describes our experience of implementing and applying ideas from prior work, and measuring whether each one actually contributed.
PDF arXiv
arXiv cs.AIOct 3
Suppressing Pressure, Amplifying Evidence: Self-Guided Attention Steering to Mitigate Sycophancy and Stubbornness
Yinghao He, Mengyu Xu, Haixiang Sun, Donghan Li, Yibo Wang +6
Reliable language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade-off, we introduce CoPE-Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training-free framework that uses the model's own judgments to identify relevant tokens, suppressing user pressure and amplifying contextual information through token-level attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 5m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

369papers this week−24.1% vs last week

All papers · 2.6k With code · 72

Newest Most relevant

2k papers this month

Open problems in Frontier LLMs

Evaluating decision models beyond accuracy1 paper raises it · 4 address it
Aggregate accuracy hides failures on controlled variations, calibration, and confidence reliability, which are critical for safe deployment in LLM systems.
- DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- Jeva: System One Models from First Principlesaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient, structured decision-making for agents3 papers raise it · 3 address it
Generative reasoning is slow and unreliable for short, repeated decisions, while fixed-depth decision models lack adaptability.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Long-horizon context retention and reuse2 papers raise it · 3 address it
Agents lose critical information when context windows are exceeded, and textual summaries alone may not preserve decision-relevant details.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cortex: Consolidating Context into Parametric Memoryaddresses
Real-world web search and evidence integration1 paper raises it · 2 address it
Agents struggle to autonomously explore, access, and integrate heterogeneous, interactive web sources to answer complex factual queries.
- Deep Search in Real-World Web Environmentsraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Cost-effective fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs can verify review comments but require reading entire papers for each check, making the process expensive and slow.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.