Papers

Field

Top tools here

All tools

billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
TanStack AI+68% downloads · 1M/wk+67.7%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
NVIDIA Dynamo+79% downloads · 17.5k/wk+79.1%

Top tools here

Open problems in Frontier LLMs

Evaluating and benchmarking decision models for reliability and calibration3 papers raise it · 4 address it
Decision models are increasingly used in LLM systems but lack standardized ways to measure correctness, confidence, and robustness across controlled variations.
- DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient and accurate fact-checking of AI and human reviews1 paper raises it · 1 addresses it
Peer review systems struggle with factual errors in comments, and current methods are either too slow or inaccurate for large-scale use.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Long-horizon agent context management and memory retention2 papers raise it · 3 address it
Agents lose critical information when context windows are exceeded, and current compaction methods fail to preserve decision-relevant details.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cortex: Consolidating Context into Parametric Memoryaddresses
Real-world web-based deep search and evidence integration1 paper raises it · 1 addresses it
Agents cannot reliably perform multi-step research in live, open web environments with diverse, interactive, and heterogeneous information sources.
- Deep Search in Real-World Web Environmentsraises · addresses
User behavior modeling for long-term personalization1 paper raises it · 1 addresses it
LLMs lack the ability to infer and adapt to evolving user preferences and intent from real-world behavior data over time.
- Vela: User Behavior as Supervision Sparks LLM Intelligenceraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

OpenAIStage not given · Oct 22$30B
DeepSeekStage not given · Oct 22$15B
Flapping AirplanesSeries A · Oct 21$1B
Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M

Recently funded

Topics in this field

language models llms · large language models · language models270 papers45 this week
retrieval augmented generation · knowledge graphs · retrieval augmented246 papers36 this week
memory construction · persistent memory · long term memory260 papers35 this week
large language models · machine translation · multilingual215 papers26 this week
generative recommendation · sequential recommendation · personalized149 papers20 this week
activation steering · attention heads · attention166 papers19 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.SEOct 7
Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures
Nicolas Lacroix, Frederic Precioso, Mireille Blay-Fornarino, Sebastien Mosser
Context: Once defined a taxonomy of stages structuring Machine Learning (ML) pipelines (e.g. Data Preprocessing, Modeling...), extracting these stages from source code is key for better understanding ML practices. However, the diversity caused by the constant evolution of ML (e.g., algorithms, datasets) makes this task challenging. Existing approaches either rely on non-scalable manual labeling or on classifiers that do not properly support domain's diversity. These limitations call for more reliable solutions. Objective: We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and enhance our understanding of practices in ML. Method: We conduct a confirmatory study based on two relevant reference works representing current limitations in the state-of-the-art. We first compare several SLMs using Cochran's Q test, then evaluate the best-performing model against reference studies via two McNemar's tests. An additional Cochran's Q test examines how taxonomy definition variations affect the SLM performance. Finally, goodness-of-fit tests compare ML practice insights from SLM classification with…
PDF arXiv
arXiv cs.IROct 7
Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two Corpora
Andrey Kuehlkamp, Priscila Correa Saboia Moreira, Samuel Rund
Retrieval-augmented generation systems increasingly rely on document-structure treatments: structure-aligned chunking, LLM-generated chunk contexts, heading-path metadata, and hierarchical two-stage retrieval. Separate studies support each on different corpora, embedders, and metrics, and none control for a shared confound: any text prepended to a chunk perturbs its embedding. We present a mechanism-isolating ablation testing all four treatments under one protocol, matching chunk sizes across conditions and adding a semantically null placebo---heading paths that are structurally valid but shuffled across documents. We score retrieval with a coverage-aware nDCG and test four pre-registered contrasts via document-clustered bootstrap with Holm correction, on two distant corpora: 200 Wikipedia Featured Articles (951 queries) and 1,585 QASPER papers (4,303 questions). Organization helps, and the cause is content, not tokens: structure-aligned chunks with real heading paths beat contextualized fixed windows (+0.022 / +0.012 cov-nDCG@10) and the placebo (+0.010 / +0.016). Naive two-stage hierarchical retrieval hurts (-0.033 / -0.015), traceable to first-stage section recall. Gold…
PDF arXiv
arXiv cs.CLOct 7
LLM4Impact: Integrating Heterogeneous Information for Scientific Impact Prediction
Yong Cao, Markus Flicke, Haoyu He, Katrin Renz, Andreas Geiger
Predicting the future impact of a newly published paper is challenging because it must be inferred from heterogeneous evidence available at publication time. Existing approaches often rely on a single source of information or combine multiple sources without accounting for their different predictive roles. In this paper, we present LLM4Impact, an evidence-aware method for scientific impact prediction that learns to represent, integrate, and calibrate heterogeneous information. LLM4Impact combines semantic, graph, LLM, and temporal representations, and injects graph information into a frozen LLM through continuous prefix tokens. A context aware gating mechanism adaptively weights different evidence, while a separate calibration module accounts for domain and temporal variation in citation scales. We further construct a large-scale benchmark dataset with 2 million papers, leakage-safe point-in-time heterogeneous ego graphs, temporal splits, and both year-level and month-level citation targets. Experiments show that LLM4Impact consistently outperforms strong semantic, graph, and LLM based baselines, with a 10.13% reduction in year RMSE on the in distribution test set and a 6.87%…
PDF arXiv
arXiv cs.IROct 7
Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition
Xuesi Wang, Yangbin Shi, Xiaolin Zheng
Generative recommenders return a limited candidate set and may omit observed targets before reranking. A training strategy appends these missed targets to reranker training lists, although inference still ranks only original candidates. This operation simultaneously changes retrieved-target weight, adds supervision over appended targets, and makes the two groups compete for probability. An append/no-append comparison therefore cannot explain changes in returned-item rankings. We construct three matched losses that hold retrieved-target weight fixed while introducing appended-target supervision and group competition separately. The intermediate loss trains within both groups but normalizes them separately, preventing training-only targets from competing with inference candidates. Experiments with a released OneRec model and locally trained Amazon generators show that this competition can harm returned-item ranking. In four prespecified Amazon Video Games comparisons, removing it improved full-target normalized discounted cumulative gain (FT-NDCG) by 7.8--22.2\%; 95\% intervals over users and three of four intervals over training runs excluded zero. A conservative development-set…
PDF arXiv
arXiv cs.CLOct 7
I would rather quit NLP than read another paper like this: The rise of antithesis in NLP papers
Olga Zamaraeva, Adrián Gude, Roi Santos-Ríos, Carlos Gómez-Rodríguez
For better or worse, LLMs are by now used routinely for scientific writing.\footnote{This paper is no exception; we did use AI to assist with writing some of the sections (see Acknowledgments).} Many have noticed that recent models fill papers with unnecessary antithesis, stating over and over what the work does not do, in ways that do not contribute to its precision or quality of expression and annoy reviewers \emph{rather than impressing them}. We study the construction \emph{rather than} in ACL papers from 2019, ACL-style arXiv papers from 2026, and papers written by GPT models from the same titles and abstracts. Its rate in 2026 is seven times the 2019 rate, and higher still in the GPT papers. Two annotators, blind to the source, find almost no 2019 use \emph{annoying} and about one in ten 2026 uses; they seldom agree on which, yet about half of 2026 papers contain a use that annoys each of them. \emph{Annoying} uses present the rejected alternative less favorably than legitimate uses. Raters of preference data and open reward models favor the construction, and an instruction to be honest promotes it. We conjecture that it is a side effect of post-training on pairwise…
PDF arXiv
arXiv cs.CLOct 7
ExperienceIndex: Artifact-Grounded Memory
Peter Baile Chen, Geoffrey X. Yu, Xinming Liu, Samuel Madden, Dan Roth +3
Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literature). As humans interact with these corpora, they naturally accumulate experiential knowledge about artifacts, enabling them to quickly identify the complete set of relevant artifacts for each new task. However, existing AI agents lack appropriate memory solutions to build or reuse such artifact-grounded experience, leading to lower answer quality and higher online cost. Existing memory solutions extract and reuse information from prior task-solving traces, but they primarily focus on user preferences, factual attributes, or abstract reasoning patterns rather than persistent artifact-specific knowledge. We introduce ExperienceIndex, a novel experience layer for AI agents that captures and reuses knowledge about artifacts based on prior reasoning traces. ExperienceIndex stores two complementary forms of experience: (i) single-artifact experiences that summarize an artifact's contribution to prior tasks and (ii) artifact-pair experiences that encode structural relationships discovered during past reasoning. Integrated as lightweight…
PDF arXiv

Papers

Top tools here

All tools

billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
TanStack AI+68% downloads · 1M/wk+67.7%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
NVIDIA Dynamo+79% downloads · 17.5k/wk+79.1%

Recently funded

Market

OpenAIStage not given · Oct 22$30B
DeepSeekStage not given · Oct 22$15B
Flapping AirplanesSeries A · Oct 21$1B
Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M

Active contributors · 90 days

Topics in this field

language models llms · large language models · language models270 papers45 this week
retrieval augmented generation · knowledge graphs · retrieval augmented246 papers36 this week
memory construction · persistent memory · long term memory260 papers35 this week
large language models · machine translation · multilingual215 papers26 this week
generative recommendation · sequential recommendation · personalized149 papers20 this week
activation steering · attention heads · attention166 papers19 this week

Recent papers

All papers in Research

arXiv cs.SEOct 7
Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures
Nicolas Lacroix, Frederic Precioso, Mireille Blay-Fornarino, Sebastien Mosser
Context: Once defined a taxonomy of stages structuring Machine Learning (ML) pipelines (e.g. Data Preprocessing, Modeling...), extracting these stages from source code is key for better understanding ML practices. However, the diversity caused by the constant evolution of ML (e.g., algorithms, datasets) makes this task challenging. Existing approaches either rely on non-scalable manual labeling or on classifiers that do not properly support domain's diversity. These limitations call for more reliable solutions. Objective: We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and enhance our understanding of practices in ML. Method: We conduct a confirmatory study based on two relevant reference works representing current limitations in the state-of-the-art. We first compare several SLMs using Cochran's Q test, then evaluate the best-performing model against reference studies via two McNemar's tests. An additional Cochran's Q test examines how taxonomy definition variations affect the SLM performance. Finally, goodness-of-fit tests compare ML practice insights from SLM classification with…
PDF arXiv
arXiv cs.IROct 7
Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two Corpora
Andrey Kuehlkamp, Priscila Correa Saboia Moreira, Samuel Rund
Retrieval-augmented generation systems increasingly rely on document-structure treatments: structure-aligned chunking, LLM-generated chunk contexts, heading-path metadata, and hierarchical two-stage retrieval. Separate studies support each on different corpora, embedders, and metrics, and none control for a shared confound: any text prepended to a chunk perturbs its embedding. We present a mechanism-isolating ablation testing all four treatments under one protocol, matching chunk sizes across conditions and adding a semantically null placebo---heading paths that are structurally valid but shuffled across documents. We score retrieval with a coverage-aware nDCG and test four pre-registered contrasts via document-clustered bootstrap with Holm correction, on two distant corpora: 200 Wikipedia Featured Articles (951 queries) and 1,585 QASPER papers (4,303 questions). Organization helps, and the cause is content, not tokens: structure-aligned chunks with real heading paths beat contextualized fixed windows (+0.022 / +0.012 cov-nDCG@10) and the placebo (+0.010 / +0.016). Naive two-stage hierarchical retrieval hurts (-0.033 / -0.015), traceable to first-stage section recall. Gold…
PDF arXiv
arXiv cs.CLOct 7
LLM4Impact: Integrating Heterogeneous Information for Scientific Impact Prediction
Yong Cao, Markus Flicke, Haoyu He, Katrin Renz, Andreas Geiger
Predicting the future impact of a newly published paper is challenging because it must be inferred from heterogeneous evidence available at publication time. Existing approaches often rely on a single source of information or combine multiple sources without accounting for their different predictive roles. In this paper, we present LLM4Impact, an evidence-aware method for scientific impact prediction that learns to represent, integrate, and calibrate heterogeneous information. LLM4Impact combines semantic, graph, LLM, and temporal representations, and injects graph information into a frozen LLM through continuous prefix tokens. A context aware gating mechanism adaptively weights different evidence, while a separate calibration module accounts for domain and temporal variation in citation scales. We further construct a large-scale benchmark dataset with 2 million papers, leakage-safe point-in-time heterogeneous ego graphs, temporal splits, and both year-level and month-level citation targets. Experiments show that LLM4Impact consistently outperforms strong semantic, graph, and LLM based baselines, with a 10.13% reduction in year RMSE on the in distribution test set and a 6.87%…
PDF arXiv
arXiv cs.IROct 7
Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition
Xuesi Wang, Yangbin Shi, Xiaolin Zheng
Generative recommenders return a limited candidate set and may omit observed targets before reranking. A training strategy appends these missed targets to reranker training lists, although inference still ranks only original candidates. This operation simultaneously changes retrieved-target weight, adds supervision over appended targets, and makes the two groups compete for probability. An append/no-append comparison therefore cannot explain changes in returned-item rankings. We construct three matched losses that hold retrieved-target weight fixed while introducing appended-target supervision and group competition separately. The intermediate loss trains within both groups but normalizes them separately, preventing training-only targets from competing with inference candidates. Experiments with a released OneRec model and locally trained Amazon generators show that this competition can harm returned-item ranking. In four prespecified Amazon Video Games comparisons, removing it improved full-target normalized discounted cumulative gain (FT-NDCG) by 7.8--22.2\%; 95\% intervals over users and three of four intervals over training runs excluded zero. A conservative development-set…
PDF arXiv
arXiv cs.CLOct 7
I would rather quit NLP than read another paper like this: The rise of antithesis in NLP papers
Olga Zamaraeva, Adrián Gude, Roi Santos-Ríos, Carlos Gómez-Rodríguez
For better or worse, LLMs are by now used routinely for scientific writing.\footnote{This paper is no exception; we did use AI to assist with writing some of the sections (see Acknowledgments).} Many have noticed that recent models fill papers with unnecessary antithesis, stating over and over what the work does not do, in ways that do not contribute to its precision or quality of expression and annoy reviewers \emph{rather than impressing them}. We study the construction \emph{rather than} in ACL papers from 2019, ACL-style arXiv papers from 2026, and papers written by GPT models from the same titles and abstracts. Its rate in 2026 is seven times the 2019 rate, and higher still in the GPT papers. Two annotators, blind to the source, find almost no 2019 use \emph{annoying} and about one in ten 2026 uses; they seldom agree on which, yet about half of 2026 papers contain a use that annoys each of them. \emph{Annoying} uses present the rejected alternative less favorably than legitimate uses. Raters of preference data and open reward models favor the construction, and an instruction to be honest promotes it. We conjecture that it is a side effect of post-training on pairwise…
PDF arXiv
arXiv cs.CLOct 7
ExperienceIndex: Artifact-Grounded Memory
Peter Baile Chen, Geoffrey X. Yu, Xinming Liu, Samuel Madden, Dan Roth +3
Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literature). As humans interact with these corpora, they naturally accumulate experiential knowledge about artifacts, enabling them to quickly identify the complete set of relevant artifacts for each new task. However, existing AI agents lack appropriate memory solutions to build or reuse such artifact-grounded experience, leading to lower answer quality and higher online cost. Existing memory solutions extract and reuse information from prior task-solving traces, but they primarily focus on user preferences, factual attributes, or abstract reasoning patterns rather than persistent artifact-specific knowledge. We introduce ExperienceIndex, a novel experience layer for AI agents that captures and reuses knowledge about artifacts based on prior reasoning traces. ExperienceIndex stores two complementary forms of experience: (i) single-artifact experiences that summarize an artifact's contribution to prior tasks and (ii) artifact-pair experiences that encode structural relationships discovered during past reasoning. Integrated as lightweight…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 28m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

300papers this week−39.8% vs last week

All papers · 2.3k With code · 61

Newest Most relevant

1.9k papers this month

Open problems in Frontier LLMs

Evaluating and benchmarking decision models for reliability and calibration3 papers raise it · 4 address it
Decision models are increasingly used in LLM systems but lack standardized ways to measure correctness, confidence, and robustness across controlled variations.
- DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient and accurate fact-checking of AI and human reviews1 paper raises it · 1 addresses it
Peer review systems struggle with factual errors in comments, and current methods are either too slow or inaccurate for large-scale use.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Long-horizon agent context management and memory retention2 papers raise it · 3 address it
Agents lose critical information when context windows are exceeded, and current compaction methods fail to preserve decision-relevant details.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cortex: Consolidating Context into Parametric Memoryaddresses
Real-world web-based deep search and evidence integration1 paper raises it · 1 addresses it
Agents cannot reliably perform multi-step research in live, open web environments with diverse, interactive, and heterogeneous information sources.
- Deep Search in Real-World Web Environmentsraises · addresses
User behavior modeling for long-term personalization1 paper raises it · 1 addresses it
LLMs lack the ability to infer and adapt to evolving user preferences and intent from real-world behavior data over time.
- Vela: User Behavior as Supervision Sparks LLM Intelligenceraises · addresses

Named by an AI from these papers' abstracts.