Papers

Field

Top tools here

All tools

jcode+233% downloads · 25.5k/wk+233.1%
billion-context+259% downloads · 1.1M/wk+258.8%
LMDeploy+136% downloads · 12.3k/wk+135.8%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
Vercel AI SDK300 releases this week

Top tools here

Open problems in Frontier LLMs

Evaluating long-horizon agents in real-world web environments1 paper raises it · 1 addresses it
Current benchmarks do not test agents on live, open, and heterogeneous web tasks with complex interactions.
- Deep Search in Real-World Web Environmentsraises · addresses
Net value of agent memory over direct retrieval1 paper raises it · 1 addresses it
Many memory systems underperform naive retrieval, and their trade-offs are poorly understood.
- Memory Tax: Dissecting the Net Value of Agent Memoryraises · addresses
Inferring user habits from weak longitudinal evidence1 paper raises it · 1 addresses it
Agents struggle to learn and apply user habits from sparse or noisy historical interactions.
- HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memoryraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models often lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs are accurate but slow and expensive for checking implicit assertions in reviews.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

VinciSeries B+ · Oct 29$250M
KineDBPre-seed · Oct 29$946k
Jingshuo TechnologySeed · Oct 29undisclosed
Kaola YouranSeries B+ · Oct 29undisclosed
Chaojinhua IntelligenceSeed · Oct 28undisclosed

Recently funded

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers33 this week
language models llms · large language models · language models301 papers31 this week
memory construction · persistent memory · long term memory285 papers24 this week
generative recommendation · sequential recommendation · personalized171 papers22 this week
large language models · machine translation · multilingual236 papers20 this week
deep research · scientific literature · citations155 papers20 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CLSep 30
RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection
Elia Onofri, Roberto Di Pietro
Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary frontier models, costly and ill-suited to high-throughput monitoring. We investigate whether a panel of cheap open-weight judges (4--9B) can be aggregated to stand in for a frontier one, what the substitution sacrifices, and when it is worth making. We propose RAIM, an aggregation scheme robust to the members' correlated errors, coupling a cross-fitted stacked logistic regression with an admissibility test that, read from the members' own outputs, identifies when aggregating them improves on their best member and stays within reach of the frontier judge. We instantiate RAIM with ten judges from disjoint families across eight faithfulness benchmarks. Against Claude Sonnet, the panel retains a median 93% of its Cohen's $κ$ and gives up only 2.9 points of balanced accuracy on average; read as paired differences, it clearly improves on one benchmark and clearly worsens on three (only two by a non-negligible margin), leaving four unresolved. At a sixty-fourth of the frontier's inference price, the operative expense is a one-time in-domain…
PDF arXiv
arXiv cs.CLSep 30
Argument Structure Prediction in Online Conversations: A Comparative Study of Modeling Paradigms and Task Architectures
Siddharth Bhargava, Sara Tonelli, Patricia Martín-Rodilla, Javier Parapar
Argument structure prediction (ASP) constructs complete argument structures from discourse by identifying argumentative units and their relations. While recent work has explored diverse approaches---including unified neural models, multi-step pipelines, and prompt-based large language models (LLMs)---their relative trade-offs remain under-explored, particularly in dialogical settings. We present a systematic evaluation of ASP under strict schema constraints, comparing supervised fine-tuning and prompt-based LLMs across single- and multi-step task architectures, generating complete argument structures from dialogical input end-to-end. We benchmark them on three diverse dialogical corpora adapted from Inference Anchoring Theory into bipolar argument structures. Under a shared evaluation framework, we assess predictive performance, cross-domain generalization, schema compliance, and computational efficiency. Our results show that ASP remains a challenging task, with identifying argumentative relations emerging as the primary bottleneck, largely due to the implicit and context-dependent nature of dialogical argumentation. To facilitate future research, we release our data processing…
PDF arXiv
arXiv cs.CLSep 30
ViLegalExpert: A Large-Scale Benchmark for Vietnamese Legal Retrieval and Question Answering from Real-World Consultations
Dat Tien Nguyen, Nghia Hieu Nguyen, Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Kiet Van Nguyen +1
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage of real-world legal consultations. We introduce \textbf{ViLegalExpert}, a large-scale benchmark constructed from authentic citizen--lawyer consultations, containing over \textbf{172K} questions across \textbf{34 legal domains}, together with professional answers and expert-verified legal evidence. ViLegalExpert supports legal information retrieval, extractive QA, and abstractive QA. Experiments with representative retrieval methods and language models reveal substantial challenges in evidence retrieval and grounded answer generation. While pretrained models perform strongly on QA, hybrid retrieval achieves the best retrieval performance. These results demonstrate the difficulty of mapping naturally expressed legal questions to authoritative provisions and establish ViLegalExpert as a challenging benchmark for reliable Vietnamese Legal AI.
PDF arXiv
arXiv cs.LGSep 30
When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls
Juli Huang
Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that this inference can be fragile unless the intervention semantics, evaluation metric, and controls are carefully validated. A natural post-projection implementation of "zeroing a head" is nearly uncorrelated with a corrected pre-projection ablation (Pearson r = 0.057) and selects a completely disjoint top-5 set of important heads. We also show that binary accuracy can hide effects at behavioral floors and near ceilings, whereas gold-token log-probability remains graded. Using a discovery/held-out split and 1,000 matched random-head and layer-matched-head control draws, the corrected per-head effect ranking is highly stable across splits (Spearman rho = 0.974), and the top-5 selected heads significantly exceed both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is not robust on GPT-2. Replication on DistilGPT2 preserves the intervention-semantic and matched-control findings. These results show that single-head…
PDF arXiv
arXiv cs.AISep 30
RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agent
Ubaidillah Ariq Prathama, Bo Liu, Yeo Boon Hong, Yu-Xuan Huang, Yangkai Ding +1
Long-horizon agent interactions generate useful but noisy experience, and retraining models to absorb it is expensive. Context-evolving agents therefore need memory extraction methods that improve with more test-time compute without relying on gold labels. We propose RefCon, which combines sequential self-refinement with parallel self-contrast to extract higher-quality memories without gold labels. Evaluated on AppWorld and BFCL-V3 across multiple context-evolving agent frameworks, RefCon delivers strong and consistent gains, including relative improvements of 21.6% on ACE and 16.6% on ReMe over no-scaling baselines, while a diversity-focused variant (DivCon) achieves a 35.5% gain on ReasoningBank. RefCon consistently outperforms existing baselines without ground-truth labels, and generalizes across model scales and to software engineering tasks, where it surpasses even ground-truth baselines. We further analyze the accuracy-token trade-off and scaling behavior, showing RefCon maintains favorable efficiency and continues to improve as more trajectories are used, unlike diversity-only scaling which saturates earlier.
PDF arXiv
arXiv cs.AISep 30
BELIEFRAG: Making Adaptive RAG State-Aware under Evolving Evidence
Hongji Pu
Adaptive RAG uses signals such as confidence, relevance, support, and retrieval quality to decide when to search or correct evidence. In multi-step retrieval, however, these local signals must be combined into a persistent view of what the current evidence supports, what remains missing, and which action should follow. Existing methods often use such signals as separate triggers, making it difficult to preserve a coherent evidence state across a trajectory; we call this problem evidence-state fragmentation. We introduce BELIEFRAG, a closed-loop controller that updates an explicit state over sufficiency, reliability, conflict, uncertainty, evidence gaps, and acquisition cost, then chooses among retrieval, query rewriting, verification, answering, stopping, and abstention. Across six QA benchmarks with gpt-oss-120b, BELIEFRAG reaches mean token F1 0.572 with 3.89k tokens per question, outperforming fixed iterative retrieval (0.555 F1) while using 39% fewer tokens. The same quality-cost pattern transfers to Qwen3-32B, where BELIEFRAG reaches 0.552 F1 versus 0.523 for iterative retrieval while using 35% fewer tokens. Analysis shows that the main gains come from corrective…
PDF arXiv

Papers

Top tools here

All tools

jcode+233% downloads · 25.5k/wk+233.1%
billion-context+259% downloads · 1.1M/wk+258.8%
LMDeploy+136% downloads · 12.3k/wk+135.8%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
Vercel AI SDK300 releases this week

Recently funded

Market

VinciSeries B+ · Oct 29$250M
KineDBPre-seed · Oct 29$946k
Jingshuo TechnologySeed · Oct 29undisclosed
Kaola YouranSeries B+ · Oct 29undisclosed
Chaojinhua IntelligenceSeed · Oct 28undisclosed

Topics in this field

retrieval augmented generation · knowledge graphs · retrieval augmented280 papers33 this week
language models llms · large language models · language models301 papers31 this week
memory construction · persistent memory · long term memory285 papers24 this week
generative recommendation · sequential recommendation · personalized171 papers22 this week
large language models · machine translation · multilingual236 papers20 this week
deep research · scientific literature · citations155 papers20 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.CLSep 30
RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection
Elia Onofri, Roberto Di Pietro
Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary frontier models, costly and ill-suited to high-throughput monitoring. We investigate whether a panel of cheap open-weight judges (4--9B) can be aggregated to stand in for a frontier one, what the substitution sacrifices, and when it is worth making. We propose RAIM, an aggregation scheme robust to the members' correlated errors, coupling a cross-fitted stacked logistic regression with an admissibility test that, read from the members' own outputs, identifies when aggregating them improves on their best member and stays within reach of the frontier judge. We instantiate RAIM with ten judges from disjoint families across eight faithfulness benchmarks. Against Claude Sonnet, the panel retains a median 93% of its Cohen's $κ$ and gives up only 2.9 points of balanced accuracy on average; read as paired differences, it clearly improves on one benchmark and clearly worsens on three (only two by a non-negligible margin), leaving four unresolved. At a sixty-fourth of the frontier's inference price, the operative expense is a one-time in-domain…
PDF arXiv
arXiv cs.CLSep 30
Argument Structure Prediction in Online Conversations: A Comparative Study of Modeling Paradigms and Task Architectures
Siddharth Bhargava, Sara Tonelli, Patricia Martín-Rodilla, Javier Parapar
Argument structure prediction (ASP) constructs complete argument structures from discourse by identifying argumentative units and their relations. While recent work has explored diverse approaches---including unified neural models, multi-step pipelines, and prompt-based large language models (LLMs)---their relative trade-offs remain under-explored, particularly in dialogical settings. We present a systematic evaluation of ASP under strict schema constraints, comparing supervised fine-tuning and prompt-based LLMs across single- and multi-step task architectures, generating complete argument structures from dialogical input end-to-end. We benchmark them on three diverse dialogical corpora adapted from Inference Anchoring Theory into bipolar argument structures. Under a shared evaluation framework, we assess predictive performance, cross-domain generalization, schema compliance, and computational efficiency. Our results show that ASP remains a challenging task, with identifying argumentative relations emerging as the primary bottleneck, largely due to the implicit and context-dependent nature of dialogical argumentation. To facilitate future research, we release our data processing…
PDF arXiv
arXiv cs.CLSep 30
ViLegalExpert: A Large-Scale Benchmark for Vietnamese Legal Retrieval and Question Answering from Real-World Consultations
Dat Tien Nguyen, Nghia Hieu Nguyen, Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Kiet Van Nguyen +1
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage of real-world legal consultations. We introduce \textbf{ViLegalExpert}, a large-scale benchmark constructed from authentic citizen--lawyer consultations, containing over \textbf{172K} questions across \textbf{34 legal domains}, together with professional answers and expert-verified legal evidence. ViLegalExpert supports legal information retrieval, extractive QA, and abstractive QA. Experiments with representative retrieval methods and language models reveal substantial challenges in evidence retrieval and grounded answer generation. While pretrained models perform strongly on QA, hybrid retrieval achieves the best retrieval performance. These results demonstrate the difficulty of mapping naturally expressed legal questions to authoritative provisions and establish ViLegalExpert as a challenging benchmark for reliable Vietnamese Legal AI.
PDF arXiv
arXiv cs.LGSep 30
When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls
Juli Huang
Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that this inference can be fragile unless the intervention semantics, evaluation metric, and controls are carefully validated. A natural post-projection implementation of "zeroing a head" is nearly uncorrelated with a corrected pre-projection ablation (Pearson r = 0.057) and selects a completely disjoint top-5 set of important heads. We also show that binary accuracy can hide effects at behavioral floors and near ceilings, whereas gold-token log-probability remains graded. Using a discovery/held-out split and 1,000 matched random-head and layer-matched-head control draws, the corrected per-head effect ranking is highly stable across splits (Spearman rho = 0.974), and the top-5 selected heads significantly exceed both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is not robust on GPT-2. Replication on DistilGPT2 preserves the intervention-semantic and matched-control findings. These results show that single-head…
PDF arXiv
arXiv cs.AISep 30
RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agent
Ubaidillah Ariq Prathama, Bo Liu, Yeo Boon Hong, Yu-Xuan Huang, Yangkai Ding +1
Long-horizon agent interactions generate useful but noisy experience, and retraining models to absorb it is expensive. Context-evolving agents therefore need memory extraction methods that improve with more test-time compute without relying on gold labels. We propose RefCon, which combines sequential self-refinement with parallel self-contrast to extract higher-quality memories without gold labels. Evaluated on AppWorld and BFCL-V3 across multiple context-evolving agent frameworks, RefCon delivers strong and consistent gains, including relative improvements of 21.6% on ACE and 16.6% on ReMe over no-scaling baselines, while a diversity-focused variant (DivCon) achieves a 35.5% gain on ReasoningBank. RefCon consistently outperforms existing baselines without ground-truth labels, and generalizes across model scales and to software engineering tasks, where it surpasses even ground-truth baselines. We further analyze the accuracy-token trade-off and scaling behavior, showing RefCon maintains favorable efficiency and continues to improve as more trajectories are used, unlike diversity-only scaling which saturates earlier.
PDF arXiv
arXiv cs.AISep 30
BELIEFRAG: Making Adaptive RAG State-Aware under Evolving Evidence
Hongji Pu
Adaptive RAG uses signals such as confidence, relevance, support, and retrieval quality to decide when to search or correct evidence. In multi-step retrieval, however, these local signals must be combined into a persistent view of what the current evidence supports, what remains missing, and which action should follow. Existing methods often use such signals as separate triggers, making it difficult to preserve a coherent evidence state across a trajectory; we call this problem evidence-state fragmentation. We introduce BELIEFRAG, a closed-loop controller that updates an explicit state over sufficiency, reliability, conflict, uncertainty, evidence gaps, and acquisition cost, then chooses among retrieval, query rewriting, verification, answering, stopping, and abstention. Across six QA benchmarks with gpt-oss-120b, BELIEFRAG reaches mean token F1 0.572 with 3.89k tokens per question, outperforming fixed iterative retrieval (0.555 F1) while using 39% fewer tokens. The same quality-cost pattern transfers to Qwen3-32B, where BELIEFRAG reaches 0.552 F1 versus 0.523 for iterative retrieval while using 35% fewer tokens. Analysis shows that the main gains come from corrective…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 52m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

254papers this week−48.3% vs last week

All papers · 2.6k With code · 74

Newest Most relevant

1.8k papers this month

Open problems in Frontier LLMs

Evaluating long-horizon agents in real-world web environments1 paper raises it · 1 addresses it
Current benchmarks do not test agents on live, open, and heterogeneous web tasks with complex interactions.
- Deep Search in Real-World Web Environmentsraises · addresses
Net value of agent memory over direct retrieval1 paper raises it · 1 addresses it
Many memory systems underperform naive retrieval, and their trade-offs are poorly understood.
- Memory Tax: Dissecting the Net Value of Agent Memoryraises · addresses
Inferring user habits from weak longitudinal evidence1 paper raises it · 1 addresses it
Agents struggle to learn and apply user habits from sparse or noisy historical interactions.
- HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memoryraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models often lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Efficient fact-checking of review comments1 paper raises it · 1 addresses it
Frontier LLMs are accurate but slow and expensive for checking implicit assertions in reviews.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses

Named by an AI from these papers' abstracts.