Papers

Field

Top tools here

All tools

QwenPaw+1266% downloads · 39k/wk+1266.0%
billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
TanStack AI+68% downloads · 1M/wk+67.7%

Top tools here

Open problems in Agents

Long-horizon task completion with exploration and verification3 papers raise it · 2 address it
Agents struggle to accumulate evidence, adjust strategies, and verify results over long, realistic workflows.
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often fail to deliver agents that truly meet the specified requirements rather than just appearing to.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Realistic evaluation of agent learning from experience1 paper raises it · 1 addresses it
Existing benchmarks often test reasoning with known rules rather than learning novel, counterintuitive rules through interaction.
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
Detecting when past context matters in long conversations1 paper raises it · 0 address it
Agents cannot reliably identify when historical context is relevant, leading to missed or incorrect responses.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
Open-source, reproducible GUI agent stacks2 papers raise it · 2 address it
Frontier GUI agents are often closed, limiting systematic study and community collaboration.
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

FirstWorkSeries A · Oct 21$15M
Asymptote LabsSeed · Oct 20$5.5M
LumilabPre-seed · Oct 20undisclosed
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M

Recently funded

Topics in this field

coding agents · code generation · coding agent318 papers81 this week
skills · tasks · ai321 papers47 this week
ai agents · indirect prompt injection · adversarial328 papers45 this week
recursive self improvement · harness evolution · adaptive278 papers44 this week
tool using agents · agent systems · multi agent223 papers40 this week
agent · agentic · llm agents179 papers40 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CLOct 12
CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds
Ang Li, Haofei Yu, Zhengwei Tao, Liang Chen, Kuan Li +5
The growing use of language models for forecasting motivates a broader evaluation of their world-model capabilities. Such evaluation must go beyond predictive accuracy to assess hidden-state inference and intervention prediction. Real-world forecasting benchmarks offer limited access to hidden-state ground truth and controlled experiments. We introduce CivWorldBench, a benchmark built on a multi-agent strategy game engine with recorded internal states, controlled observations, and paired intervention rollouts. Its 1,900 decision points from 475 worlds support three task levels: association forecasting, belief inference, and intervention prediction. Reference predictors support attributable evaluation of capability gaps, with an expert harness and empirical ceiling estimates contextualizing scores. Stored trajectories make intermediate predictions verifiable, while agent-play experiments test whether world-model capabilities support decision-making. The main panel comprises eleven frontier language models. Composite benchmark scores correlate with real-world forecasting leaderboards on matched model panels, more closely than with a general-capability index, supporting its use as a…
PDF arXiv
arXiv cs.AIOct 12
AuthGateBench: Benchmarking LLMs as Authorization Gate
Zhixiong Zhuang, Jin Jia, Zhiwei Shang, Ziyu Wang
Automated approval systems let coding agents execute high-risk actions with minimal human oversight by delegating the authorization decision to a classifier, typically a large language model (LLM). Such systems, adopted in tools like Claude Code and Codex, remain vulnerable to failure modes such as scope escalation, where the agent takes actions exceeding what the user authorized. Existing benchmarks focus mainly on malicious requests, evaluating refusal under an implicit policy. We introduce AuthGateBench, the first policy-guided benchmark for authorization gating under benign requests. We implement a novel pipeline where an LLM generates contrast pairs with identical action and trajectory but differing request, flipping the label between block and allow. This allows us to assess whether models reason over context rather than the action alone. We further examine whether auxiliary environment signals can help models make better decisions. The resulting benchmark comprises 1,400 instances spanning five failure modes drawn from real-world Claude Code incidents. Evaluations show accuracy degrades unevenly across failure modes as models over-rely on internal safety alignment, and…
PDF arXiv
arXiv cs.AICodeOct 12▲ 5
EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications
David Nukrai, Bar Alon, Nativ Levy, Ido Andrew Atad, Ido Cohen +4
Web application development often proceeds through successive requests that extend an existing implementation. Yet web-generation benchmarks primarily evaluate development toward fixed specifications. We introduce EvoWeb, a novel benchmark comprising 320 cumulative requests over 32 web applications, grounded in real-world product features and structured by functional dependency graphs. Agents iteratively modify a persistent codebase, while a multimodal web agent tests each intermediate application in a live browser, assessing newly requested functionality, cross-request integration, and previously requested functionality. To characterize reliability throughout development, we integrate trajectory performance across user-tolerance thresholds rather than impose a fixed stopping criterion. Experiments across open- and closed-source frontier models show that EvoWeb reveals clear performance differences between models, with cross-request integration emerging as a key challenge. Further analysis shows that later requests with deeper or more numerous dependencies are solved less accurately, while runtime errors become more frequent over successive requests. EvoWeb provides a testbed for…
PDF arXiv Code
arXiv cs.AIOct 12▲ 4
Recursive Environment Scaling for General Long-Horizon Agents
Lingxiao Jing, Zhengbo Jiao, Yichen Xiang, Bofan Zhu, Lingbei Meng +8
We study environment scaling as a way to increase agent-benchmark difficulty while holding core task objectives fixed. Our framework, Recursive Environment Scaling (RES), transforms existing benchmark environments while preserving their objectives and completion requirements. A proposer changes information access, resource relations, and execution dependencies; execution-grounded checks validate the resulting candidate, and accepted changes are inherited across rounds. This produces related environments for paired evaluation and training-trajectory collection. Experiments on 294 initial tasks from three benchmarks show lower task success for all six evaluation models, with decreases of 57.14--77.14 percentage points on Claw-Eval-Live. With equal trajectory counts and identical optimization settings, fine-tuning on RES trajectories outperforms original-trajectory SFT in all nine model--benchmark comparisons by 7.00--14.61 percentage points. Ablations identify recursive inheritance and dependency composition as important contributors to increased difficulty. Together, these results support environment scaling for both agent evaluation and training.
PDF arXiv
arXiv cs.LGOct 12
A Production-Grade, High-Efficiency Agentic RL Framework for Frontier LLMs with Validated Recipes
LoonSage Team, Bowei Song, Caiwei Zhao, Chuwen Liu, Dechuang Gu +43
Reinforcement learning (RL) has emerged as a critical phase in the post-training pipeline of large language models (LLMs). As the frontier of LLM capabilities shifts from static reasoning to agentic tasks--characterized by external tool invocation, multi-turn interactions, and long-horizon decision-making--existing mainstream RL frameworks have grown increasingly unwieldy, raising the overhead of adoption, usage, and debugging. To address these challenges, we present LoongSage, an agent-native, fully asynchronous RL framework with a lightweight core of roughly 14K lines of code. Built upon Megatron for distributed training and SGLang for high-throughput inference, LoongSage centralizes orchestration within a single controller process and encapsulates all GPU-resident components as Ray actors. It introduces AgentFlow, a unified execution paradigm that orchestrates both white-box and black-box agents. For asynchronous execution, it integrates sliding-window strategies that bound trajectory staleness, and employs TransferMesh, a topology-aware synchronization mechanism that routes weights after each step over the fastest transport available to each receiver. Beyond agentic RL,…
PDF arXiv
arXiv cs.AIOct 12
Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
Seungdong Yoa, Ye Seul Sim, Suhee Yoon, Sanghyu Yoon, Dongmin Kim +3
Multi-agent debate has shown promise for improving the reasoning of large language models, yet recent theory suggests its benefits are highly regime-dependent. While interaction can amplify informative signals under corrective conditions, symmetric debate dynamics are neutral in expectation, often making majority voting preferable. We reconcile these views by arguing that debate is effective only when invoked at the right time and with appropriate structure. Based on this insight, we propose LASE: Leader-Adaptive Structured Engagement, a leader-centric multi-agent debate framework that selectively engages interaction only in non-neutral regimes. LASE introduces an asymmetric leader-supporter architecture that enables directed information flow and selective signal amplification, while defaulting to simple aggregation otherwise. Experiments across diverse reasoning benchmarks show that LASE achieves multi-agent-level performance with near single-agent token cost, substantially improving efficiency over static debate and voting baselines.
PDF arXiv

Papers

Top tools here

All tools

QwenPaw+1266% downloads · 39k/wk+1266.0%
billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%
TanStack AI+68% downloads · 1M/wk+67.7%

Topics in this field

coding agents · code generation · coding agent318 papers81 this week
skills · tasks · ai321 papers47 this week
ai agents · indirect prompt injection · adversarial328 papers45 this week
recursive self improvement · harness evolution · adaptive278 papers44 this week
tool using agents · agent systems · multi agent223 papers40 this week
agent · agentic · llm agents179 papers40 this week

Recently funded

Market

FirstWorkSeries A · Oct 21$15M
Asymptote LabsSeed · Oct 20$5.5M
LumilabPre-seed · Oct 20undisclosed
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.CLOct 12
CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds
Ang Li, Haofei Yu, Zhengwei Tao, Liang Chen, Kuan Li +5
The growing use of language models for forecasting motivates a broader evaluation of their world-model capabilities. Such evaluation must go beyond predictive accuracy to assess hidden-state inference and intervention prediction. Real-world forecasting benchmarks offer limited access to hidden-state ground truth and controlled experiments. We introduce CivWorldBench, a benchmark built on a multi-agent strategy game engine with recorded internal states, controlled observations, and paired intervention rollouts. Its 1,900 decision points from 475 worlds support three task levels: association forecasting, belief inference, and intervention prediction. Reference predictors support attributable evaluation of capability gaps, with an expert harness and empirical ceiling estimates contextualizing scores. Stored trajectories make intermediate predictions verifiable, while agent-play experiments test whether world-model capabilities support decision-making. The main panel comprises eleven frontier language models. Composite benchmark scores correlate with real-world forecasting leaderboards on matched model panels, more closely than with a general-capability index, supporting its use as a…
PDF arXiv
arXiv cs.AIOct 12
AuthGateBench: Benchmarking LLMs as Authorization Gate
Zhixiong Zhuang, Jin Jia, Zhiwei Shang, Ziyu Wang
Automated approval systems let coding agents execute high-risk actions with minimal human oversight by delegating the authorization decision to a classifier, typically a large language model (LLM). Such systems, adopted in tools like Claude Code and Codex, remain vulnerable to failure modes such as scope escalation, where the agent takes actions exceeding what the user authorized. Existing benchmarks focus mainly on malicious requests, evaluating refusal under an implicit policy. We introduce AuthGateBench, the first policy-guided benchmark for authorization gating under benign requests. We implement a novel pipeline where an LLM generates contrast pairs with identical action and trajectory but differing request, flipping the label between block and allow. This allows us to assess whether models reason over context rather than the action alone. We further examine whether auxiliary environment signals can help models make better decisions. The resulting benchmark comprises 1,400 instances spanning five failure modes drawn from real-world Claude Code incidents. Evaluations show accuracy degrades unevenly across failure modes as models over-rely on internal safety alignment, and…
PDF arXiv
arXiv cs.AICodeOct 12▲ 5
EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications
David Nukrai, Bar Alon, Nativ Levy, Ido Andrew Atad, Ido Cohen +4
Web application development often proceeds through successive requests that extend an existing implementation. Yet web-generation benchmarks primarily evaluate development toward fixed specifications. We introduce EvoWeb, a novel benchmark comprising 320 cumulative requests over 32 web applications, grounded in real-world product features and structured by functional dependency graphs. Agents iteratively modify a persistent codebase, while a multimodal web agent tests each intermediate application in a live browser, assessing newly requested functionality, cross-request integration, and previously requested functionality. To characterize reliability throughout development, we integrate trajectory performance across user-tolerance thresholds rather than impose a fixed stopping criterion. Experiments across open- and closed-source frontier models show that EvoWeb reveals clear performance differences between models, with cross-request integration emerging as a key challenge. Further analysis shows that later requests with deeper or more numerous dependencies are solved less accurately, while runtime errors become more frequent over successive requests. EvoWeb provides a testbed for…
PDF arXiv Code
arXiv cs.AIOct 12▲ 4
Recursive Environment Scaling for General Long-Horizon Agents
Lingxiao Jing, Zhengbo Jiao, Yichen Xiang, Bofan Zhu, Lingbei Meng +8
We study environment scaling as a way to increase agent-benchmark difficulty while holding core task objectives fixed. Our framework, Recursive Environment Scaling (RES), transforms existing benchmark environments while preserving their objectives and completion requirements. A proposer changes information access, resource relations, and execution dependencies; execution-grounded checks validate the resulting candidate, and accepted changes are inherited across rounds. This produces related environments for paired evaluation and training-trajectory collection. Experiments on 294 initial tasks from three benchmarks show lower task success for all six evaluation models, with decreases of 57.14--77.14 percentage points on Claw-Eval-Live. With equal trajectory counts and identical optimization settings, fine-tuning on RES trajectories outperforms original-trajectory SFT in all nine model--benchmark comparisons by 7.00--14.61 percentage points. Ablations identify recursive inheritance and dependency composition as important contributors to increased difficulty. Together, these results support environment scaling for both agent evaluation and training.
PDF arXiv
arXiv cs.LGOct 12
A Production-Grade, High-Efficiency Agentic RL Framework for Frontier LLMs with Validated Recipes
LoonSage Team, Bowei Song, Caiwei Zhao, Chuwen Liu, Dechuang Gu +43
Reinforcement learning (RL) has emerged as a critical phase in the post-training pipeline of large language models (LLMs). As the frontier of LLM capabilities shifts from static reasoning to agentic tasks--characterized by external tool invocation, multi-turn interactions, and long-horizon decision-making--existing mainstream RL frameworks have grown increasingly unwieldy, raising the overhead of adoption, usage, and debugging. To address these challenges, we present LoongSage, an agent-native, fully asynchronous RL framework with a lightweight core of roughly 14K lines of code. Built upon Megatron for distributed training and SGLang for high-throughput inference, LoongSage centralizes orchestration within a single controller process and encapsulates all GPU-resident components as Ray actors. It introduces AgentFlow, a unified execution paradigm that orchestrates both white-box and black-box agents. For asynchronous execution, it integrates sliding-window strategies that bound trajectory staleness, and employs TransferMesh, a topology-aware synchronization mechanism that routes weights after each step over the fastest transport available to each receiver. Beyond agentic RL,…
PDF arXiv
arXiv cs.AIOct 12
Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
Seungdong Yoa, Ye Seul Sim, Suhee Yoon, Sanghyu Yoon, Dongmin Kim +3
Multi-agent debate has shown promise for improving the reasoning of large language models, yet recent theory suggests its benefits are highly regime-dependent. While interaction can amplify informative signals under corrective conditions, symmetric debate dynamics are neutral in expectation, often making majority voting preferable. We reconcile these views by arguing that debate is effective only when invoked at the right time and with appropriate structure. Based on this insight, we propose LASE: Leader-Adaptive Structured Engagement, a leader-centric multi-agent debate framework that selectively engages interaction only in non-neutral regimes. LASE introduces an asymmetric leader-supporter architecture that enables directed information flow and selective signal amplification, while defaulting to simple aggregation otherwise. Experiments across diverse reasoning benchmarks show that LASE achieves multi-agent-level performance with near single-agent token cost, substantially improving efficiency over static debate and voting baselines.
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 1h ago

Agents

LLM agents: planning, tool use, multi-agent systems, computer/web agents and agent benchmarks.

513papers this week−11.4% vs last week

All papers · 2.7k With code · 139

Newest Most relevant

2.5k papers this month

Open problems in Agents

Long-horizon task completion with exploration and verification3 papers raise it · 2 address it
Agents struggle to accumulate evidence, adjust strategies, and verify results over long, realistic workflows.
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often fail to deliver agents that truly meet the specified requirements rather than just appearing to.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Realistic evaluation of agent learning from experience1 paper raises it · 1 addresses it
Existing benchmarks often test reasoning with known rules rather than learning novel, counterintuitive rules through interaction.
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
Detecting when past context matters in long conversations1 paper raises it · 0 address it
Agents cannot reliably identify when historical context is relevant, leading to missed or incorrect responses.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
Open-source, reproducible GUI agent stacks2 papers raise it · 2 address it
Frontier GUI agents are often closed, limiting systematic study and community collaboration.
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses

Named by an AI from these papers' abstracts.