Papers

Field

Top tools here

Coming with more sources: stars history

All tools

Agno4 releases this week
FastMCP6 releases this week
n8n+25% downloads · 148.3k/wk+25.0%
DeepEval+15% downloads · 769.3k/wk+14.7%
Ragas+14% downloads · 319.9k/wk+13.6%

Top tools here

Open problems in Agents

Long-horizon task execution and evidence accumulation3 papers raise it · 2 address it
Agents struggle to sustain multi-step reasoning, adapt strategies, and turn intermediate results into final deliverables in real-world tasks.
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Realistic evaluation of agentic learning from experience1 paper raises it · 1 addresses it
Existing benchmarks often test reasoning with pre-existing knowledge rather than learning from novel, unfamiliar environments.
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often fail to deliver agents that truly meet the specified requirements, not just appear to.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Detecting when past context matters in long conversations1 paper raises it · 0 address it
AI systems cannot reliably identify when historical context is relevant for current interactions in long-term companionship.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
Scalable and realistic GUI agent training and evaluation1 paper raises it · 2 address it
High cost and scarcity of realistic interactive environments limit progress in GUI agent training and evaluation.
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useaddresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

Asymptote LabsSeed · Oct 20$5.5M
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M
SupaHumanSeries A · Oct 19$1.5M
BoxLiteSeed · Oct 19undisclosed

Recently funded

Topics in this field

coding agents · code generation · coding agent286 papers86 this week
skills · tasks · ai308 papers65 this week
recursive self improvement · harness evolution · adaptive260 papers52 this week
tool using agents · agent systems · multi agent209 papers50 this week
ai agents · indirect prompt injection · adversarial314 papers48 this week
vision language · safety evaluations · safety alignment115 papers38 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.CLOct 19
Co-Evolving Policy and Reward for Code Generation: A Mechanistic Study of Single-Adapter Drift
Amarsaikhan Batjargal, Noriko Tomuro
Reinforcement learning with verifiable rewards (RLVR) is particularly well suited to code generation because generated programs can be objectively evaluated through execution. Execution also provides behavioral information that can be reused as additional supervision. The Synergistic Policy and Reward Co-Evolving Framework (SPARK), originally proposed for reasoning tasks, exploits this idea by recycling training rollouts and their evaluation results to train a model not only to generate solutions but also to evaluate and improve them. We investigate whether this policy-reward co-evolution approach transfers effectively to code generation and whether execution-grounded reflection provides an additional source of self-improvement. We compare SPARK-style rollout recycling with execution-only Group Relative Policy Optimization (GRPO) using Qwen2.5-Coder-3B-Instruct. Our results do not support either hypothesis under the configuration tested. Although co-evolution improves training performance, these gains do not generalize to held-out problems. Compared at the same stage of training, rollout recycling produces roughly 34x greater KL divergence from the reference policy than…
PDF arXiv
arXiv cs.SEOct 19
When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories
Tianqi Liu, Yandong Guan, Songze Li, Wenkai Wang, Gang Yu +1
Coding agents increasingly undertake development tasks in mature enterprise repositories, yet existing benchmarks offer limited insight into whether they fulfill complete Product Requirement Documents (PRDs) and where delivery breaks down. Issue resolution and development from scratch leave incremental PRD fulfillment underexplored, while isolated tests provide incomplete evidence of business and integration requirements. We introduce PACE, a benchmark of 43 enterprise tasks with 750 expert-audited Code Points. It combines requirement reconstruction, criterion construction, and evidence-based evaluation to measure PRD fulfillment and connect delivery gaps to agent behavior. Across 11 model--scaffold configurations, most scores fall between 52.1 and 78.1; GPT-5.6-Sol achieves the highest complete-delivery rate of 14.0\%. Incomplete implementation and missing integration are the principal bottlenecks: producing individual components does not ensure a complete delivery. Evaluation achieves 95.2\% agreement with expert consensus, with lower case-level variation across judging records than repeated executions. Beyond model comparison, the benchmark provides actionable feedback for…
PDF arXiv
arXiv cs.SEOct 19
The EBC Proportionality Law: A Recurring Architectural-Role Pattern Across Java Software
Dzikri Rahadian Fudholi, Andrea Capiluppi
Architectural roles describe how responsibilities are distributed within software systems. Entity--Boundary--Control (EBC) separates domain information, external interaction, and workflow coordination, but its proportions across diverse software remain unclear. This study examines file-level classifier-derived EBC proportionality in 1,556,985 Java files from 732 project instances and 78 functional categories. A classifier trained on labels generated by a large language model assigned one representative role per file. The labels were aggregated at project and functional-category levels and analysed using descriptive statistics and density-based clustering. The results show that Entity was less common than Boundary and Control. The dominant cluster covered 48 categories, 472 project instances, and 47.75% of classified files; its mean composition was 13.25% Entity, 42.65% Boundary, and 44.10% Control. The remaining 30 categories form one alternative and 11 smaller specialised patterns. Some selected variations were consistent with category responsibilities, while others received mixed support. Agreement with independent expert labels was limited (31.37% accuracy; macro-F1 0.31;…
PDF arXiv
arXiv cs.AIOct 19
Autonomous intelligence for mining productivity: a multi-agent AI approach to operational value creation
N Guerin, A Dehoux, M Nakhla
Integrated Remote Operation Centres (IROCs) are emerging as the architectural frontier of Industry 4.0 for capital-intensive sectors such as mining, oil and gas, and energy. Yet two intertwined gaps constrain their evolution towards full autonomy. First, no integrated agentic architecture has been proposed for mining IROCs that orchestrates Supervisory Control and Data Acquisition systems, digital twins, predictive analytics and human-machine interfaces under safety-critical constraints. Second, no evaluation framework jointly addresses the real-time, Health-Safety-Environment, regulatory-auditability and cost-per-decision requirements that govern industrial agentic deployments. This paper makes four contributions. It synthesises industry needs and quantified gains documented in analogous industries that motivate an agentic approach to mining IROCs. It introduces EI2ROC, a three-tier multi-agent architecture combining a Large Language Model orchestrator with specialised monitoring and production-capacityprediction agents communicating through a Model Context Protocol-style tool interface. It provides a critical review of agentic evaluation along six orthogonal dimensions and…
PDF arXiv
arXiv cs.AIOct 19
VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement
Ruotian Peng, Yuanyi Ren, Shuaiqi Liu, Xin Guo, Fang Zhu +5
Recursive self-improvement (RSI) requires agents to diagnose system bottlenecks and coordinate improvements across training and inference. We refer to the agents' capability to jointly optimize training data, the trainer, and the inference harness as cross-component co-design. We introduce VibeTraining-Bench, a benchmark for evaluating this capability across tasks spanning terminal operation, software engineering, multi-turn tool use, and mathematical reasoning. Within each task, research agents start from an identical initial agent system under a fixed resource budget. Guided by validation feedback, they iteratively refine data, trainers, and inference harnesses, then select and submit a final agent system for evaluation on the test set. To prevent reward hacking, we enforce three safeguards: test data isolation, separate research and grading containers, and network restrictions for research agents. Our evaluation results show that frontier research agents substantially improve target agent systems. For example, on SWE-bench Verified, GPT-6-Astra raises the target agent system's performance from 32.9% to 53.7%, comparable to the human-developed instruct model's 55.4%. However,…
PDF arXiv
arXiv cs.AIOct 19
The Control Layer Behind Rule-Following Agents Trading Across Market Conditions
Saad Naja, Nasser Odtallah, Jake Van-Praag, Sarah May
Trading discipline often fails people. Our study tested whether four AI trading agents followed their strategies and risk rules. Offering traders the evidence and an agent evaluation toolkit to separate luck from following the rules, before giving agents meaningful autonomy or capital. Our study analysed 2,627 closed positions from 12 February to 27 August 2026 across the same asset universe and execution conditions. Most positions were simulated against live prices with a 0.09% round-trip fee; 18% involved real Hyperliquid orders. Each position declared a strategy, stop and target, with a limit of five open positions per agent. The fleet finished net positive after fees. Rule adherence was high across all four agents, including the unprofitable one. Profitability tracked how often agents won relative to their break-even win rates. Institutional-style controls ran by default and contained losses through a 29% market drawdown and recovery, with documented exceptions. The discipline held for all four agents. Agents kept 97.5% of gross loss within declared stops, and typical losses closed at declared levels. Declared risk remained stable after losses and broadly stable across calm…
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

Agno4 releases this week
FastMCP6 releases this week
n8n+25% downloads · 148.3k/wk+25.0%
DeepEval+15% downloads · 769.3k/wk+14.7%
Ragas+14% downloads · 319.9k/wk+13.6%

Topics in this field

coding agents · code generation · coding agent286 papers86 this week
skills · tasks · ai308 papers65 this week
recursive self improvement · harness evolution · adaptive260 papers52 this week
tool using agents · agent systems · multi agent209 papers50 this week
ai agents · indirect prompt injection · adversarial314 papers48 this week
vision language · safety evaluations · safety alignment115 papers38 this week

Recently funded

Market

Asymptote LabsSeed · Oct 20$5.5M
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M
SupaHumanSeries A · Oct 19$1.5M
BoxLiteSeed · Oct 19undisclosed

Active contributors · 90 days

Recent papers

All in Research →

arXiv cs.CLOct 19
Co-Evolving Policy and Reward for Code Generation: A Mechanistic Study of Single-Adapter Drift
Amarsaikhan Batjargal, Noriko Tomuro
Reinforcement learning with verifiable rewards (RLVR) is particularly well suited to code generation because generated programs can be objectively evaluated through execution. Execution also provides behavioral information that can be reused as additional supervision. The Synergistic Policy and Reward Co-Evolving Framework (SPARK), originally proposed for reasoning tasks, exploits this idea by recycling training rollouts and their evaluation results to train a model not only to generate solutions but also to evaluate and improve them. We investigate whether this policy-reward co-evolution approach transfers effectively to code generation and whether execution-grounded reflection provides an additional source of self-improvement. We compare SPARK-style rollout recycling with execution-only Group Relative Policy Optimization (GRPO) using Qwen2.5-Coder-3B-Instruct. Our results do not support either hypothesis under the configuration tested. Although co-evolution improves training performance, these gains do not generalize to held-out problems. Compared at the same stage of training, rollout recycling produces roughly 34x greater KL divergence from the reference policy than…
PDF arXiv
arXiv cs.SEOct 19
When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories
Tianqi Liu, Yandong Guan, Songze Li, Wenkai Wang, Gang Yu +1
Coding agents increasingly undertake development tasks in mature enterprise repositories, yet existing benchmarks offer limited insight into whether they fulfill complete Product Requirement Documents (PRDs) and where delivery breaks down. Issue resolution and development from scratch leave incremental PRD fulfillment underexplored, while isolated tests provide incomplete evidence of business and integration requirements. We introduce PACE, a benchmark of 43 enterprise tasks with 750 expert-audited Code Points. It combines requirement reconstruction, criterion construction, and evidence-based evaluation to measure PRD fulfillment and connect delivery gaps to agent behavior. Across 11 model--scaffold configurations, most scores fall between 52.1 and 78.1; GPT-5.6-Sol achieves the highest complete-delivery rate of 14.0\%. Incomplete implementation and missing integration are the principal bottlenecks: producing individual components does not ensure a complete delivery. Evaluation achieves 95.2\% agreement with expert consensus, with lower case-level variation across judging records than repeated executions. Beyond model comparison, the benchmark provides actionable feedback for…
PDF arXiv
arXiv cs.SEOct 19
The EBC Proportionality Law: A Recurring Architectural-Role Pattern Across Java Software
Dzikri Rahadian Fudholi, Andrea Capiluppi
Architectural roles describe how responsibilities are distributed within software systems. Entity--Boundary--Control (EBC) separates domain information, external interaction, and workflow coordination, but its proportions across diverse software remain unclear. This study examines file-level classifier-derived EBC proportionality in 1,556,985 Java files from 732 project instances and 78 functional categories. A classifier trained on labels generated by a large language model assigned one representative role per file. The labels were aggregated at project and functional-category levels and analysed using descriptive statistics and density-based clustering. The results show that Entity was less common than Boundary and Control. The dominant cluster covered 48 categories, 472 project instances, and 47.75% of classified files; its mean composition was 13.25% Entity, 42.65% Boundary, and 44.10% Control. The remaining 30 categories form one alternative and 11 smaller specialised patterns. Some selected variations were consistent with category responsibilities, while others received mixed support. Agreement with independent expert labels was limited (31.37% accuracy; macro-F1 0.31;…
PDF arXiv
arXiv cs.AIOct 19
Autonomous intelligence for mining productivity: a multi-agent AI approach to operational value creation
N Guerin, A Dehoux, M Nakhla
Integrated Remote Operation Centres (IROCs) are emerging as the architectural frontier of Industry 4.0 for capital-intensive sectors such as mining, oil and gas, and energy. Yet two intertwined gaps constrain their evolution towards full autonomy. First, no integrated agentic architecture has been proposed for mining IROCs that orchestrates Supervisory Control and Data Acquisition systems, digital twins, predictive analytics and human-machine interfaces under safety-critical constraints. Second, no evaluation framework jointly addresses the real-time, Health-Safety-Environment, regulatory-auditability and cost-per-decision requirements that govern industrial agentic deployments. This paper makes four contributions. It synthesises industry needs and quantified gains documented in analogous industries that motivate an agentic approach to mining IROCs. It introduces EI2ROC, a three-tier multi-agent architecture combining a Large Language Model orchestrator with specialised monitoring and production-capacityprediction agents communicating through a Model Context Protocol-style tool interface. It provides a critical review of agentic evaluation along six orthogonal dimensions and…
PDF arXiv
arXiv cs.AIOct 19
VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement
Ruotian Peng, Yuanyi Ren, Shuaiqi Liu, Xin Guo, Fang Zhu +5
Recursive self-improvement (RSI) requires agents to diagnose system bottlenecks and coordinate improvements across training and inference. We refer to the agents' capability to jointly optimize training data, the trainer, and the inference harness as cross-component co-design. We introduce VibeTraining-Bench, a benchmark for evaluating this capability across tasks spanning terminal operation, software engineering, multi-turn tool use, and mathematical reasoning. Within each task, research agents start from an identical initial agent system under a fixed resource budget. Guided by validation feedback, they iteratively refine data, trainers, and inference harnesses, then select and submit a final agent system for evaluation on the test set. To prevent reward hacking, we enforce three safeguards: test data isolation, separate research and grading containers, and network restrictions for research agents. Our evaluation results show that frontier research agents substantially improve target agent systems. For example, on SWE-bench Verified, GPT-6-Astra raises the target agent system's performance from 32.9% to 53.7%, comparable to the human-developed instruct model's 55.4%. However,…
PDF arXiv
arXiv cs.AIOct 19
The Control Layer Behind Rule-Following Agents Trading Across Market Conditions
Saad Naja, Nasser Odtallah, Jake Van-Praag, Sarah May
Trading discipline often fails people. Our study tested whether four AI trading agents followed their strategies and risk rules. Offering traders the evidence and an agent evaluation toolkit to separate luck from following the rules, before giving agents meaningful autonomy or capital. Our study analysed 2,627 closed positions from 12 February to 27 August 2026 across the same asset universe and execution conditions. Most positions were simulated against live prices with a 0.09% round-trip fee; 18% involved real Hyperliquid orders. Each position declared a strategy, stop and target, with a limit of five open positions per agent. The fleet finished net positive after fees. Rule adherence was high across all four agents, including the unprofitable one. Profitability tracked how often agents won relative to their break-even win rates. Institutional-style controls ran by default and contained losses through a 29% market drawdown and recovery, with documented exceptions. The discipline held for all four agents. Agents kept 97.5% of gross loss within declared stops, and typical losses closed at declared levels. Declared risk remained stable after losses and broadly stable across calm…
PDF arXiv

Activity · 12 weeks

Coming with more sources: repos

Open problems in Agents

Long-horizon task execution and evidence accumulation3 papers raise it · 2 address it
Agents struggle to sustain multi-step reasoning, adapt strategies, and turn intermediate results into final deliverables in real-world tasks.
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Realistic evaluation of agentic learning from experience1 paper raises it · 1 addresses it
Existing benchmarks often test reasoning with pre-existing knowledge rather than learning from novel, unfamiliar environments.
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often fail to deliver agents that truly meet the specified requirements, not just appear to.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Detecting when past context matters in long conversations1 paper raises it · 0 address it
AI systems cannot reliably identify when historical context is relevant for current interactions in long-term companionship.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
Scalable and realistic GUI agent training and evaluation1 paper raises it · 2 address it
High cost and scarcity of realistic interactive environments limit progress in GUI agent training and evaluation.
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentsraises · addresses
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useaddresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 1h ago

Agents

LLM agents: planning, tool use, multi-agent systems, computer/web agents and agent benchmarks.

560papers this week+8.9% vs last week

All papers · 2.5k With code · 112

Newest Most relevant

2.4k papers this month