Spotlight

Papers

Field

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno+17% downloads · 519.1k/wk+16.8%
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%
LangGraph7 releases this week

Top tools here

Open problems in Frontier LLMs

Structural decision-making in LLMs1 paper raises it · 3 address it
Current LLMs lack native support for decision problems with structural requirements like null-awareness, permutation invariance, and scalability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Adaptive computation for decision models1 paper raises it · 1 addresses it
Fixed-depth decision models waste computation on easy questions and cannot scale depth dynamically.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
Reliable no-thinking behavior in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Memory consolidation in context-augmented LLMs1 paper raises it · 1 addresses it
Gains from context are lost after its removal, and models struggle to retain learned knowledge without forgetting.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions, causing performance collapse.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Recently funded

Topics in this field

reasoning traces · reasoning models · latent reasoning372 papers74 this week
attention heads · large language models · language models218 papers53 this week
dense retrieval · retrieval augmented generation · retrieval246 papers50 this week
large language models · attribution · language models llms196 papers39 this week
large language models · large language · language models178 papers36 this week
typed decision models · typed decision model · typed decisions129 papers35 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

★ Spotlight this month

arXiv cs.CLOct 15

Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle

Yuanjian Xu, Jianing Hao, Tianze Sun, Yongyao Duan, Sam Chen +5

Data strategy plays a central role in training large language models (LLMs). It governs the selection, weighting, and scheduling of training data, thereby determining the capabilities a model can acquire from a fixed data budget. We treat high-quality data as having high training-dependent utility, which depends on the target objective, the model, and its training state rather than on the text alone. Research in this area now spans pretraining and post-training, but related ideas use inconsistent terminology and evaluation protocols. This work reviews data strategies across the training lifecycle and formulates them as policies that allocate a training budget over candidate examples. We organise methods by their decision granularity, model dependence, and temporal adaptivity, then compare their computational requirements and empirical support. Fine-grained or state-dependent policies offer more control but incur additional scoring, search, or rollout costs. Evidence for transfer across model scales and comparable end-to-end cost remains limited. A continuously updated list of papers, along with their public implementations, is available at…

PDF arXiv page Details

Spotlight

Recent papers

All in Research →

arXiv cs.CLOct 19
KEEP-Bench: Evaluating Evidence Preservation in Context Compression
Ziqing Man, Di Wu, Jia-Chen Gu, Junyi Zhang, Kai-Wei Chang +1
Context compression improves long-context efficiency but may introduce faithfulness issues through an information bottleneck. Existing evaluations rely on QA metrics, but a reader can still answer correctly from an unfaithful compression, so these metrics underestimate such loss. To bridge this gap, we introduce KEEP-Bench, a diagnostic benchmark that directly assesses compression faithfulness with 4,102 cases across three challenging settings that require keeping evidence for plausible reference resolutions, explicit qualifiers, and minority perspectives. Experiments show that even with tailored prompting, the strongest models remain unreliable: despite context windows exceeding 100K tokens and inputs averaging only 3.1K tokens, the best open-weight model (31B) preserves all required evidence in only 66.8% of cases, and the best proprietary model in only 61.6%. Performance also varies sharply across evidence structures, so conclusions from limited structures may be biased. We further introduce a metric, Evidence Preservation Return (EPR), to relate evidence recall to inference FLOPs, and find that it peaks at moderate scales (4B-8B), where recall remains low. Additional output…
PDF arXiv
arXiv stat.MLOct 19
Teacher Supervision, Fixed Representations and Calibration: Exact Risk Calculations for Candidate-Scoring Decision Models
Agus Sudjianto, Denis Burakov
Candidate-scoring decision models return a probability distribution over a presented set of answers. Public documentation of one such system, Jev, and of an open counterpart, JevAny, describes three training ingredients that determine what these probabilities estimate: soft targets from larger models, a frozen pretrained backbone and temperature calibration. We analyze each ingredient in Gaussian surrogates where the finite-sample risk can be computed exactly. A student trained on teacher targets inherits the teacher's bias, which more student data does not remove and agreement with the teacher does not reveal. A fixed representation lowers the risk only below an explicit sample-size threshold. In a probit model the exact recalibration map is a temperature scaling, which removes all excess risk only when the teacher's coefficient has the correct direction and otherwise leaves a residual that calibration diagnostics do not detect. Under Luce's choice axiom, candidate scores are coherent across refinements of an event if and only if they aggregate by log-sum-exp. On 231,521 credit card complaints from the Consumer Financial Protection Bureau, where the decision is whether a…
PDF arXiv
arXiv cs.AIOct 19
SOMA: Continuous Monitoring and Intervention for System Two Reasoning with System One Models
Ruizhi Qian, Lei Shi, Tianyu Hu, Bo Yan, Song Wang +1
Reasoning models often run into problems without noticing them, even when symptoms are visible in their own text. In humans, metacognition separates monitoring from control: fast, intuitive signals keep watch while slow, deliberate reasoning unfolds and help decide whether to continue, change course, or stop. System One (S1) decision models, such as Jev, are a natural candidate for this role: they return a calibrated probability for a natural-language question about a text in one fast call, yet they are used only as gates before reasoning. We introduce SOMA (System One Metacognitive Awareness), in which an S1 model continuously monitors a System Two (S2) reasoning model and intervenes when needed. SOMA has three layers: a scenario layer states each objective in natural language with predefined actions, an awareness layer lets S1 judge whether the described behavior appears, and a control layer accumulates these judgments, first reminds S2, and takes over only if the behavior persists. Sharing one S1 model and one controller across tasks, without training, SOMA addresses two different failures on two reasoning models: on overthinking, it cuts reasoning tokens by up to 36\% while…
PDF arXiv
arXiv cs.CLOct 19
RESIN: Multi-Chain Reasoning with Entity-Structured Indexing
Xuanyu Meng, Jash Rajesh Parekh, Jiawei Han
Multi-chain reasoning is a challenging task not only because the multi-chain evidence is dispersed, but also because different parts of reasoning may require distinct, interacting chains of support. Flat retrieval can flood a model with loosely related text, whereas early selection of a single reasoning path can discard evidence that becomes useful only after other facts are considered. We introduce RESIN (Reasoning with Entity Structured Indexing), which uses structure to locate evidence while keeping the original source as the basis for reasoning. RESIN decomposes a question into focused one-hop needs, retrieves a small set of source-grounded units for each hop, and places only this relevant evidence in a compact reasoning buffer. The buffer preserves alternative supports until final composition, making intermediate decisions explicit without forcing a brittle path. On the full HolisQA benchmark, RESIN achieves 82.92 context recall and 76.70 answer relevancy over all questions, with higher reported CR and AR across five domains and higher F1 in four domains than the published baselines. Ablations show that both typed evidence diversity and hop-wise decomposition are essential,…
PDF arXiv
arXiv cs.CLOct 19
EpiLens: Lightweight RAG for Epidemiological Knowledge Acquisition
Vincenzo Perri, Ciro Cattuto, Daniela Paolotti
To prepare for future outbreaks, researchers need to know what studies have been done and what data they use. Mapping a large literature by hand is slow, while large language models alone can misstate or conflate sources. We present EpiLens, a lightweight, open-source retrieval-augmented generation package that applies a fixed retrieval template and output schema to every paper in a corpus and links outputs to source passages. We build a corpus of 2,297 papers from the sources of a multi-pathogen parameter study and evaluate EpiLens's paper-classification workflow against 214 manually labelled papers. Without task-specific training data, the classifier matches or exceeds the tested supervised and zero-shot baselines across four tasks. The study's early extraction sheets cite published sources for 327 estimates. We link each estimate to a cited document or to corpus studies included by a review. Then, we use the classifier's labels to compare the papers associated with different parameters and pathogens. EpiLens labels more papers linked to reproduction numbers than fatality risks as model-based (54% versus 22%, compared with 62% versus 24% in the manual labels). The gap is…
PDF arXiv
arXiv cs.CLOct 19
HoloCosm: Beyond Single-Metric Benchmark Compression Across Multiple Metrics
Yupei Lin, Zhongzhan Huang, Junxin Li, Guoming Ling, Shanshan Zhong +3
Benchmark compression reduces evaluation cost by selecting a small set of samples. Existing methods typically guide selection using a single metric, although benchmarks often assess models along multiple dimensions. We find that model rankings are often weakly correlated across metrics, indicating that different metrics capture distinct aspects of model performance. Samples selected for one metric do not reliably preserve scores and rankings on others when used for compressed evaluation. At 80\% compression, each of the five single-metric baselines yields higher reconstruction error than equal-size random sampling in over 70\% of cross-metric comparisons. To address this challenge, we present HoloCosm, which jointly minimizes normalized score-reconstruction error across source models and metrics while constraining ranking distortion. We evaluate HoloCosm across multiple compression rates on 15 language, multimodal, VLA, and agent benchmark configurations with 85 component metrics. With 80\% compression, HoloCosm outperforms Random, the strongest baseline overall: average NMAE drops by 35.3\% to 0.236, worst-component NMAE drops by 39.5\% to 0.330, and average Spearman correlation…
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno+17% downloads · 519.1k/wk+16.8%
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%
LangGraph7 releases this week

Topics in this field

reasoning traces · reasoning models · latent reasoning372 papers74 this week
attention heads · large language models · language models218 papers53 this week
dense retrieval · retrieval augmented generation · retrieval246 papers50 this week
large language models · attribution · language models llms196 papers39 this week
large language models · large language · language models178 papers36 this week
typed decision models · typed decision model · typed decisions129 papers35 this week

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

★ Spotlight this month

arXiv cs.CLOct 15

Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle

Yuanjian Xu, Jianing Hao, Tianze Sun, Yongyao Duan, Sam Chen +5

Data strategy plays a central role in training large language models (LLMs). It governs the selection, weighting, and scheduling of training data, thereby determining the capabilities a model can acquire from a fixed data budget. We treat high-quality data as having high training-dependent utility, which depends on the target objective, the model, and its training state rather than on the text alone. Research in this area now spans pretraining and post-training, but related ideas use inconsistent terminology and evaluation protocols. This work reviews data strategies across the training lifecycle and formulates them as policies that allocate a training budget over candidate examples. We organise methods by their decision granularity, model dependence, and temporal adaptivity, then compare their computational requirements and empirical support. Fine-grained or state-dependent policies offer more control but incur additional scoring, search, or rollout costs. Evidence for transfer across model scales and comparable end-to-end cost remains limited. A continuously updated list of papers, along with their public implementations, is available at…

PDF arXiv page Details

Active contributors · 90 days

Activity · 12 weeks

Coming with more sources: repos

Recent papers

All in Research →

arXiv cs.CLOct 19
KEEP-Bench: Evaluating Evidence Preservation in Context Compression
Ziqing Man, Di Wu, Jia-Chen Gu, Junyi Zhang, Kai-Wei Chang +1
Context compression improves long-context efficiency but may introduce faithfulness issues through an information bottleneck. Existing evaluations rely on QA metrics, but a reader can still answer correctly from an unfaithful compression, so these metrics underestimate such loss. To bridge this gap, we introduce KEEP-Bench, a diagnostic benchmark that directly assesses compression faithfulness with 4,102 cases across three challenging settings that require keeping evidence for plausible reference resolutions, explicit qualifiers, and minority perspectives. Experiments show that even with tailored prompting, the strongest models remain unreliable: despite context windows exceeding 100K tokens and inputs averaging only 3.1K tokens, the best open-weight model (31B) preserves all required evidence in only 66.8% of cases, and the best proprietary model in only 61.6%. Performance also varies sharply across evidence structures, so conclusions from limited structures may be biased. We further introduce a metric, Evidence Preservation Return (EPR), to relate evidence recall to inference FLOPs, and find that it peaks at moderate scales (4B-8B), where recall remains low. Additional output…
PDF arXiv
arXiv stat.MLOct 19
Teacher Supervision, Fixed Representations and Calibration: Exact Risk Calculations for Candidate-Scoring Decision Models
Agus Sudjianto, Denis Burakov
Candidate-scoring decision models return a probability distribution over a presented set of answers. Public documentation of one such system, Jev, and of an open counterpart, JevAny, describes three training ingredients that determine what these probabilities estimate: soft targets from larger models, a frozen pretrained backbone and temperature calibration. We analyze each ingredient in Gaussian surrogates where the finite-sample risk can be computed exactly. A student trained on teacher targets inherits the teacher's bias, which more student data does not remove and agreement with the teacher does not reveal. A fixed representation lowers the risk only below an explicit sample-size threshold. In a probit model the exact recalibration map is a temperature scaling, which removes all excess risk only when the teacher's coefficient has the correct direction and otherwise leaves a residual that calibration diagnostics do not detect. Under Luce's choice axiom, candidate scores are coherent across refinements of an event if and only if they aggregate by log-sum-exp. On 231,521 credit card complaints from the Consumer Financial Protection Bureau, where the decision is whether a…
PDF arXiv
arXiv cs.AIOct 19
SOMA: Continuous Monitoring and Intervention for System Two Reasoning with System One Models
Ruizhi Qian, Lei Shi, Tianyu Hu, Bo Yan, Song Wang +1
Reasoning models often run into problems without noticing them, even when symptoms are visible in their own text. In humans, metacognition separates monitoring from control: fast, intuitive signals keep watch while slow, deliberate reasoning unfolds and help decide whether to continue, change course, or stop. System One (S1) decision models, such as Jev, are a natural candidate for this role: they return a calibrated probability for a natural-language question about a text in one fast call, yet they are used only as gates before reasoning. We introduce SOMA (System One Metacognitive Awareness), in which an S1 model continuously monitors a System Two (S2) reasoning model and intervenes when needed. SOMA has three layers: a scenario layer states each objective in natural language with predefined actions, an awareness layer lets S1 judge whether the described behavior appears, and a control layer accumulates these judgments, first reminds S2, and takes over only if the behavior persists. Sharing one S1 model and one controller across tasks, without training, SOMA addresses two different failures on two reasoning models: on overthinking, it cuts reasoning tokens by up to 36\% while…
PDF arXiv
arXiv cs.CLOct 19
RESIN: Multi-Chain Reasoning with Entity-Structured Indexing
Xuanyu Meng, Jash Rajesh Parekh, Jiawei Han
Multi-chain reasoning is a challenging task not only because the multi-chain evidence is dispersed, but also because different parts of reasoning may require distinct, interacting chains of support. Flat retrieval can flood a model with loosely related text, whereas early selection of a single reasoning path can discard evidence that becomes useful only after other facts are considered. We introduce RESIN (Reasoning with Entity Structured Indexing), which uses structure to locate evidence while keeping the original source as the basis for reasoning. RESIN decomposes a question into focused one-hop needs, retrieves a small set of source-grounded units for each hop, and places only this relevant evidence in a compact reasoning buffer. The buffer preserves alternative supports until final composition, making intermediate decisions explicit without forcing a brittle path. On the full HolisQA benchmark, RESIN achieves 82.92 context recall and 76.70 answer relevancy over all questions, with higher reported CR and AR across five domains and higher F1 in four domains than the published baselines. Ablations show that both typed evidence diversity and hop-wise decomposition are essential,…
PDF arXiv
arXiv cs.CLOct 19
EpiLens: Lightweight RAG for Epidemiological Knowledge Acquisition
Vincenzo Perri, Ciro Cattuto, Daniela Paolotti
To prepare for future outbreaks, researchers need to know what studies have been done and what data they use. Mapping a large literature by hand is slow, while large language models alone can misstate or conflate sources. We present EpiLens, a lightweight, open-source retrieval-augmented generation package that applies a fixed retrieval template and output schema to every paper in a corpus and links outputs to source passages. We build a corpus of 2,297 papers from the sources of a multi-pathogen parameter study and evaluate EpiLens's paper-classification workflow against 214 manually labelled papers. Without task-specific training data, the classifier matches or exceeds the tested supervised and zero-shot baselines across four tasks. The study's early extraction sheets cite published sources for 327 estimates. We link each estimate to a cited document or to corpus studies included by a review. Then, we use the classifier's labels to compare the papers associated with different parameters and pathogens. EpiLens labels more papers linked to reproduction numbers than fatality risks as model-based (54% versus 22%, compared with 62% versus 24% in the manual labels). The gap is…
PDF arXiv
arXiv cs.CLOct 19
HoloCosm: Beyond Single-Metric Benchmark Compression Across Multiple Metrics
Yupei Lin, Zhongzhan Huang, Junxin Li, Guoming Ling, Shanshan Zhong +3
Benchmark compression reduces evaluation cost by selecting a small set of samples. Existing methods typically guide selection using a single metric, although benchmarks often assess models along multiple dimensions. We find that model rankings are often weakly correlated across metrics, indicating that different metrics capture distinct aspects of model performance. Samples selected for one metric do not reliably preserve scores and rankings on others when used for compressed evaluation. At 80\% compression, each of the five single-metric baselines yields higher reconstruction error than equal-size random sampling in over 70\% of cross-metric comparisons. To address this challenge, we present HoloCosm, which jointly minimizes normalized score-reconstruction error across source models and metrics while constraining ranking distortion. We evaluate HoloCosm across multiple compression rates on 15 language, multimodal, VLA, and agent benchmark configurations with 85 component metrics. With 80\% compression, HoloCosm outperforms Random, the strongest baseline overall: average NMAE drops by 35.3\% to 0.236, worst-component NMAE drops by 39.5\% to 0.330, and average Spearman correlation…
PDF arXiv

Fields · AI · updated 1h ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

459papers this week+1.8% vs last week

All papers · 2.2k With code · 50

Newest Most relevant

2.1k papers this month

Open problems in Frontier LLMs

Structural decision-making in LLMs1 paper raises it · 3 address it
Current LLMs lack native support for decision problems with structural requirements like null-awareness, permutation invariance, and scalability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Adaptive computation for decision models1 paper raises it · 1 addresses it
Fixed-depth decision models waste computation on easy questions and cannot scale depth dynamically.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
Reliable no-thinking behavior in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Memory consolidation in context-augmented LLMs1 paper raises it · 1 addresses it
Gains from context are lost after its removal, and models struggle to retain learned knowledge without forgetting.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions, causing performance collapse.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses

Named by an AI from these papers' abstracts.