Papers

Field

Top tools here

All tools

jcode+233% downloads · 25.5k/wk+233.1%
billion-context+259% downloads · 1.1M/wk+258.8%
LMDeploy+136% downloads · 12.3k/wk+135.8%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
Vercel AI SDK300 releases this week

Top tools here

Open problems in Frontier LLMs

Evaluating long-horizon agents in real-world web environments1 paper raises it · 1 addresses it
Current benchmarks do not test agents on live, open, and heterogeneous web tasks with complex interactions.
- Deep Search in Real-World Web Environmentsraises · addresses
Net value of agent memory over direct retrieval1 paper raises it · 1 addresses it
Many memory systems underperform naive retrieval, and their trade-offs are poorly understood.
- Memory Tax: Dissecting the Net Value of Agent Memoryraises · addresses
Inferring user habits from weak longitudinal evidence1 paper raises it · 1 addresses it
Agents struggle to learn and apply user habits from sparse or noisy historical interactions.
- HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memoryraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models often lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs are accurate but slow and expensive for checking implicit assertions in reviews.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

KineDBPre-seed · Oct 29$946.3k
Jingshuo TechnologySeed · Oct 29undisclosed
Kaola YouranSeries B+ · Oct 29undisclosed
Chaojinhua IntelligenceSeed · Oct 28undisclosed
Chaoqiumian TechnologySeed · Oct 26undisclosed

Recently funded

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers36 this week
language models llms · large language models · language models301 papers32 this week
memory construction · persistent memory · long term memory285 papers28 this week
large language models · machine translation · multilingual236 papers24 this week
generative recommendation · sequential recommendation · personalized171 papers24 this week
deep research · scientific literature · citations155 papers20 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGSep 30
Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves
Sohail, Sarkar, Shakuntala Baichoo
Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time. Its effect is reported as a scaling curve: accuracy against the number $k$ of sampled answers. The curve is cheap to draw and expensive to trust. A budget read off it is chosen after looking at every point, so only a band that covers all budgets at once protects the choice, and on a 100-question benchmark a fixed exact-binomial design needs 192,000 generated answers to certify 64 budgets to within $\pm1/32$ at 95%. Most of that cost pays for the wrong uncertainty. A benchmark is a fixed list of questions; at budget 64, about three quarters of the variance of a selected answer's correctness lies between questions, and an audit that revisits every question need not pay for it. We derive the minimax cost of certifying the whole curve, up to logarithmic factors. It has three parts: calibrating the tail of the score distribution, telling the questions apart, and within-question noise summed along the curve. At a single benchmark the last part sharpens to the variance of one answer's influence under the best allocation of answers to questions, which every…
PDF arXiv
arXiv cs.CLSep 30
Index-Translate: A Multilingual Translation Model Family -- Text, Speech, Controlled Dubbing, and Long-Document Translation
Tianjiao Li, Mengran Yu, Chenyu Shi, Lusheng Zhang, Qisi Chen +6
We introduce Index-Translate, a multilingual translation model family that combines a shared multilingual foundation with specialized training for general translation, instruction following, speech translation, controlled dubbing, and long-document translation. It includes three model sizes, 2B, 9B, and 35B-A3B, and supports translation in 150 languages, with multilingual instruction following. Evaluations on general translation and complex translation instructions show that Index-Translate outperforms translation models of comparable size and achieves performance comparable to 100B-scale translation models and frontier models. Image-text evaluation shows that translation remains possible without image examples in Index-Translate-specific training, although results vary by model size. Index-Echo provides end-to-end speech-to-text and speech-to-speech translation, outperforming existing end-to-end models and achieving performance comparable to frontier omni models. Index-Homura extends the family to syllable-controlled dubbing. Index-NativeLong introduces native long-document translation with a dedicated task formulation and benchmark. These capabilities support diverse…
PDF arXiv
arXiv cs.CLSep 30
Debias It Yourself: Teaching LLMs Cognitive Bias Mitigation Interventions
Chahat Raj, Sina Mansouri, Aylin Caliskan, Antonios Anastasopoulos, Ziwei Zhu
Bias has long been studied in social psychology and cognitive science, where decades of research have produced a body of validated interventions that reduce stereotypical thinking and prejudiced responses in humans. We propose Debias It Yourself (DIY), a cognitively grounded framework that translates five such interventions into debiasing procedures for large language models and delivers them through three established paradigms: Show (in-context examples), Train (instruction tuning), and Revise (guided self-revision). Across three models, five bias benchmarks, eleven debiasing baselines, and three reasoning benchmarks, Train+Revise and Revise alone attain the top two average ranks, lead the bias-reasoning tradeoff (mean bias as low as 2% at 90% reasoning accuracy), and reduce bias on unseen dimensions by up to 14.8%. Our code and data are publicly available.
PDF arXiv
arXiv cs.CLSep 30
On the (In)effectiveness of AMR Augmentation for Large Language Models
Hoa Quynh Nhung Nguyen, Jacopo Staiano, Michael Sullivan
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit---or lack thereof---of AMR augmentation for modern LLMs is thus far unclear. In this paper, we attempt to reproduce recent work that reported substantial downstream gains from AMR augmentation, finding that these are likely due to specific choices in the experimental settings used: using a consistent and unified protocol for hyperparameter selection, we observe that text-only baselines consistently match or exceed the performance of AMR-augmented models. To investigate this null result, we introduce a perplexity-based probe measuring the degree to which AMR provides an LLM with supplemental relational knowledge not already available to the model. We find that AMR augmentation does not help LLMs improve their understanding of relational content in the sentence, indicating that augmenting these models with AMR offers no clear benefit on downstream tasks.
PDF arXiv
arXiv cs.CLSep 30
Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Zhaoxuan Tan, Pei Zhou +4
As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compaction essential for staying within context windows and reducing prefill cost. Existing methods summarize the history or compress its KV cache, often adding model computation to preserve information for future requests. A new user request can change which history matters, but reassessing that history with the model requires re-encoding it if the KV cache has expired. Past attention provides signals of historical importance and dependencies between messages, while relevance to the current task must be assessed using the new user request. We introduce ReCAP, a memory compaction method that stores attention-derived importance scores and dependency links in a lightweight, persistent context graph. For each new request, ReCAP combines stored importance with relevance cues from the request and follows dependency links to select messages and their supporting context, without additional model calls for selection. Compared with Codex's default summarization-based compaction, ReCAP reduces estimated latency for compaction and cold restoration…
PDF arXiv
arXiv cs.CLSep 30
JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation
Mufeng Yang, Junwei Yu, Yepeng Ding
Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel of judges leaves a hard residue: when judges disagree, majority voting discards the conflict instead of resolving it. We present JuryFlow, a disagreement-guided, human-in-the-loop multi-agent evaluation framework that treats inter-judge disagreement not as noise to be averaged away, but as a precise, claim-level signal indicating where an evaluation is uncertain. JuryFlow decomposes each candidate response into atomic claims, has a panel of heterogeneous judges assign per-claim verdicts, and builds a disagreement graph whose nodes are scored by verdict entropy and whose edges encode structural similarity between claims. A human acts as a structural guide, selecting which disagreement to resolve through a single, minimal intervention rather than re-labeling the response, after which the focal claim is re-evaluated, the correction propagates along graph edges and to historically similar cases, and is crystallized into reusable rubric entries that all judges inherit, making the evaluator progressively self-refining. To enable…
PDF arXiv

Papers

Top tools here

All tools

jcode+233% downloads · 25.5k/wk+233.1%
billion-context+259% downloads · 1.1M/wk+258.8%
LMDeploy+136% downloads · 12.3k/wk+135.8%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
Vercel AI SDK300 releases this week

Recently funded

Market

KineDBPre-seed · Oct 29$946.3k
Jingshuo TechnologySeed · Oct 29undisclosed
Kaola YouranSeries B+ · Oct 29undisclosed
Chaojinhua IntelligenceSeed · Oct 28undisclosed
Chaoqiumian TechnologySeed · Oct 26undisclosed

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers36 this week
language models llms · large language models · language models301 papers32 this week
memory construction · persistent memory · long term memory285 papers28 this week
large language models · machine translation · multilingual236 papers24 this week
generative recommendation · sequential recommendation · personalized171 papers24 this week
deep research · scientific literature · citations155 papers20 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.LGSep 30
Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves
Sohail, Sarkar, Shakuntala Baichoo
Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time. Its effect is reported as a scaling curve: accuracy against the number $k$ of sampled answers. The curve is cheap to draw and expensive to trust. A budget read off it is chosen after looking at every point, so only a band that covers all budgets at once protects the choice, and on a 100-question benchmark a fixed exact-binomial design needs 192,000 generated answers to certify 64 budgets to within $\pm1/32$ at 95%. Most of that cost pays for the wrong uncertainty. A benchmark is a fixed list of questions; at budget 64, about three quarters of the variance of a selected answer's correctness lies between questions, and an audit that revisits every question need not pay for it. We derive the minimax cost of certifying the whole curve, up to logarithmic factors. It has three parts: calibrating the tail of the score distribution, telling the questions apart, and within-question noise summed along the curve. At a single benchmark the last part sharpens to the variance of one answer's influence under the best allocation of answers to questions, which every…
PDF arXiv
arXiv cs.CLSep 30
Index-Translate: A Multilingual Translation Model Family -- Text, Speech, Controlled Dubbing, and Long-Document Translation
Tianjiao Li, Mengran Yu, Chenyu Shi, Lusheng Zhang, Qisi Chen +6
We introduce Index-Translate, a multilingual translation model family that combines a shared multilingual foundation with specialized training for general translation, instruction following, speech translation, controlled dubbing, and long-document translation. It includes three model sizes, 2B, 9B, and 35B-A3B, and supports translation in 150 languages, with multilingual instruction following. Evaluations on general translation and complex translation instructions show that Index-Translate outperforms translation models of comparable size and achieves performance comparable to 100B-scale translation models and frontier models. Image-text evaluation shows that translation remains possible without image examples in Index-Translate-specific training, although results vary by model size. Index-Echo provides end-to-end speech-to-text and speech-to-speech translation, outperforming existing end-to-end models and achieving performance comparable to frontier omni models. Index-Homura extends the family to syllable-controlled dubbing. Index-NativeLong introduces native long-document translation with a dedicated task formulation and benchmark. These capabilities support diverse…
PDF arXiv
arXiv cs.CLSep 30
Debias It Yourself: Teaching LLMs Cognitive Bias Mitigation Interventions
Chahat Raj, Sina Mansouri, Aylin Caliskan, Antonios Anastasopoulos, Ziwei Zhu
Bias has long been studied in social psychology and cognitive science, where decades of research have produced a body of validated interventions that reduce stereotypical thinking and prejudiced responses in humans. We propose Debias It Yourself (DIY), a cognitively grounded framework that translates five such interventions into debiasing procedures for large language models and delivers them through three established paradigms: Show (in-context examples), Train (instruction tuning), and Revise (guided self-revision). Across three models, five bias benchmarks, eleven debiasing baselines, and three reasoning benchmarks, Train+Revise and Revise alone attain the top two average ranks, lead the bias-reasoning tradeoff (mean bias as low as 2% at 90% reasoning accuracy), and reduce bias on unseen dimensions by up to 14.8%. Our code and data are publicly available.
PDF arXiv
arXiv cs.CLSep 30
On the (In)effectiveness of AMR Augmentation for Large Language Models
Hoa Quynh Nhung Nguyen, Jacopo Staiano, Michael Sullivan
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit---or lack thereof---of AMR augmentation for modern LLMs is thus far unclear. In this paper, we attempt to reproduce recent work that reported substantial downstream gains from AMR augmentation, finding that these are likely due to specific choices in the experimental settings used: using a consistent and unified protocol for hyperparameter selection, we observe that text-only baselines consistently match or exceed the performance of AMR-augmented models. To investigate this null result, we introduce a perplexity-based probe measuring the degree to which AMR provides an LLM with supplemental relational knowledge not already available to the model. We find that AMR augmentation does not help LLMs improve their understanding of relational content in the sentence, indicating that augmenting these models with AMR offers no clear benefit on downstream tasks.
PDF arXiv
arXiv cs.CLSep 30
Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Zhaoxuan Tan, Pei Zhou +4
As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compaction essential for staying within context windows and reducing prefill cost. Existing methods summarize the history or compress its KV cache, often adding model computation to preserve information for future requests. A new user request can change which history matters, but reassessing that history with the model requires re-encoding it if the KV cache has expired. Past attention provides signals of historical importance and dependencies between messages, while relevance to the current task must be assessed using the new user request. We introduce ReCAP, a memory compaction method that stores attention-derived importance scores and dependency links in a lightweight, persistent context graph. For each new request, ReCAP combines stored importance with relevance cues from the request and follows dependency links to select messages and their supporting context, without additional model calls for selection. Compared with Codex's default summarization-based compaction, ReCAP reduces estimated latency for compaction and cold restoration…
PDF arXiv
arXiv cs.CLSep 30
JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation
Mufeng Yang, Junwei Yu, Yepeng Ding
Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel of judges leaves a hard residue: when judges disagree, majority voting discards the conflict instead of resolving it. We present JuryFlow, a disagreement-guided, human-in-the-loop multi-agent evaluation framework that treats inter-judge disagreement not as noise to be averaged away, but as a precise, claim-level signal indicating where an evaluation is uncertain. JuryFlow decomposes each candidate response into atomic claims, has a panel of heterogeneous judges assign per-claim verdicts, and builds a disagreement graph whose nodes are scored by verdict entropy and whose edges encode structural similarity between claims. A human acts as a structural guide, selecting which disagreement to resolve through a single, minimal intervention rather than re-labeling the response, after which the focal claim is re-evaluated, the correction propagates along graph edges and to historically similar cases, and is crystallized into reusable rubric entries that all judges inherit, making the evaluator progressively self-refining. To enable…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 1h ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

277papers this week−44.5% vs last week

All papers · 2.6k With code · 74

Newest Most relevant

1.8k papers this month

Open problems in Frontier LLMs

Evaluating long-horizon agents in real-world web environments1 paper raises it · 1 addresses it
Current benchmarks do not test agents on live, open, and heterogeneous web tasks with complex interactions.
- Deep Search in Real-World Web Environmentsraises · addresses
Net value of agent memory over direct retrieval1 paper raises it · 1 addresses it
Many memory systems underperform naive retrieval, and their trade-offs are poorly understood.
- Memory Tax: Dissecting the Net Value of Agent Memoryraises · addresses
Inferring user habits from weak longitudinal evidence1 paper raises it · 1 addresses it
Agents struggle to learn and apply user habits from sparse or noisy historical interactions.
- HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memoryraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models often lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs are accurate but slow and expensive for checking implicit assertions in reviews.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.