Papers

Field

Top tools here

All tools

Agno5 releases this week
FastMCP6 releases this week
Pydantic AI+17% downloads · 1.5M/wk+17.0%
n8n+25% downloads · 148.3k/wk+25.0%
smolagents+19% downloads · 125.9k/wk+18.7%

Top tools here

Open problems in Agents

Long-horizon task execution and memory3 papers raise it · 3 address it
Agents fail to retain, retrieve, and use past information over long interactions, limiting real-world deployment.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
- Agent-Reflex: Teaching Language Agents to Act Reflectivelyaddresses
Real-world task ambiguity and unstated constraints3 papers raise it · 2 address it
Agents struggle with ambiguous, incomplete, or evolving requirements in live systems, unlike controlled benchmarks.
- OR-License: Evaluating LLM Agents as Real-World Operations Research Practitionersraises · addresses
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Faithful agent development from requirements1 paper raises it · 1 addresses it
Coding agents often produce systems that appear correct but fail to meet the actual requirements.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Sequential distraction in multimodal GUI agents1 paper raises it · 1 addresses it
Omni-modal inputs cause agents to focus on irrelevant signals, leading to compounding errors.
- Focused Omni-Modal GUI Agentsraises · addresses
Open, reproducible agent stacks and evaluation1 paper raises it · 1 addresses it
Closed systems and lack of open benchmarks hinder systematic study and improvement of agent designs.
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

FirstWorkSeries A · Oct 21$15M
Asymptote LabsSeed · Oct 20$5.5M
LumilabPre-seed · Oct 20undisclosed
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M

Recently funded

Topics in this field

coding agents · code generation · coding agent303 papers83 this week
skills · tasks · ai315 papers56 this week
recursive self improvement · harness evolution · adaptive271 papers50 this week
ai agents · indirect prompt injection · adversarial319 papers42 this week
tool using agents · agent systems · multi agent216 papers41 this week
agent · agentic · llm agents172 papers40 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CROct 14
RedDarwin: Evolving Jailbreak Skills against Frontier LLMs
Zhouyuan Ma, Yutao Wu, Hanxun Huang, Xiang Zheng, Yi Liu +1
Red teaming must continually adapt as frontier LLM safeguards evolve, yet existing approaches largely search for adversarial prompts or optimize within fixed attack procedures. We observe that even when an attack becomes ineffective, its underlying mechanisms can remain useful and can be revised or recombined to construct stronger attacks. Based on this insight, we introduce RedDarwin, an agentic framework that shifts the unit of jailbreak evolution from prompts and fixed attacks to executable attack skills. RedDarwin consists of 47 reusable base skills and autonomously revises, composes, executes, and selects them using target-model feedback, allowing successful discoveries to become building blocks for subsequent evolution. RedDarwin produces 65 evolved skills, improving 44 of 47 base attacks and increasing their mean attack success rate (ASR) on AdvBench from 18.8% to 47.3%. These evolved skills consistently outperform baselines on the held-out StrongREJECT and frontier LLMs unseen during evolution; with only 10 target-model queries, RedDarwin achieves 100% ASR on GPT-5.6-Sol, 92% on Claude-Sonnet-5, and 80% on Gemini-3.8-Flash. Analysis of the evolution trajectories further…
PDF arXiv
arXiv cs.PLOct 14
Verifying deployed bytecode against executable EVM semantics, with exact gas and no added axioms
Seulkee Baek
Much of Ethereum's most consequential code is immutable bytecode: old output of unverified compilers, and protocol logic run as contract code. We show that such bytecode can be verified as it is, without trusting what produced it, without simplifying the machine it runs on, and with no axioms beyond Lean's own. Blanc, a Lean 4 verification stack, reads fixed runtime bytes via kernel-checked certificates and states every theorem against Jaune, an executable semantics of the EVM and block processing that passes all 34,205 blockchain-test cases of Ethereum's fixture release v20.0.2 that involve only forks that have governed mainnet since May 2025. Eight artifacts, deployed from 2017 to 2026, come from seven compiler versions and an assembler. For the EIP-7002 contract, we prove that the end-of-block call to it, whose failure would invalidate the block, succeeds whatever its storage holds, and that each block's withdrawal requests are exactly the oldest pending submissions; mainnet met these theorems' starting premises when Prague began. For a Uniswap V2 pair, we prove that after any valid history whose hashed slots stay apart, from any checkpoint representing its Solidity model's…
PDF arXiv
arXiv cs.AIOct 14
Runtime Monitoring of Supermartingale Certificates
Mirco Giacobbe, Thomas A. Henzinger, Konstantin Kueffner, Emily Yu, Đorđe Žikelić
Supermartingale certificates formally guarantee temporal properties of discrete and continuous-time stochastic dynamical systems. However, these guarantees rely on a mathematical model of the world, leaving a sim-to-real gap that can invalidate them for the deployed system. We formulate the runtime monitoring problem for supermartingale certificates, in which a monitor observes a single system trajectory and raises an alarm when there is sufficient statistical evidence that the supermartingale certificate has been violated during execution. We present an algorithm to automatically construct a monitor using betting-based sequential tests that bound the probability of ever raising a false alarm at a user-specified level. Within this betting framework, we formulate strategies that allow the monitor to exploit spatial regularity to improve detection power while preserving this statistical guarantee. We demonstrate the efficacy of our approach on supermartingale certificate benchmarks from the literature.
PDF arXiv
arXiv cs.AICodeOct 14▲ 28
RuleTrek: Benchmarking Agentic Rule Reasoning in Complex Scenarios
Chenhao Yuan, Bohan Yu, Yinhao Xu, Chao Yin, Chenxi Zhou +8
Large language models (LLMs) increasingly tackle complex real-world tasks in specialized domains. However, identifying and correctly applying domain-specific rules remains challenging when scenario information is distributed across multiple files. Existing rule-reasoning benchmarks typically represent scenarios using only a few factual statements, simplifying the identification and application of relevant rules. To address this gap, this paper introduces \textbf{RuleTrek}, a multi-domain benchmark for agentic rule reasoning in complex scenarios. Its rule-grounded construction pipeline extracts structured rule graphs, generates and incrementally composes rule-specific sub-scenarios while ensuring consistency and solvability. To systematically evaluate agentic rule reasoning in complex scenarios, RuleTrek assesses three capabilities in interactive workspaces: Rule Identification (RI), Rule Deduction (RD), and End-to-End Rule Reasoning (E2E). Experiments on 11 frontier LLMs show that the highest RI and RD scores are only 62.47\% and 60.43\%, respectively, while E2E scores range from 26.97\% to 55.29\%. These results demonstrate that current LLM agents face substantial challenges in…
PDF arXiv Code
arXiv cs.CLOct 14▲ 41
Interlock: From Local Correctness to Task Completion in Long-Horizon Agents
Linji Hao, Yanji He, Zihan Wang, Hansi Yang, Jiayuan Zhang +4
Large language model agents increasingly undertake long-horizon tasks in complex environments characterized by fragmented evidence, partially completed states, and dependencies across resources and artifacts. Existing agents may satisfy individual requirements while failing to maintain these relationships, producing locally correct but incomplete outcomes. These failures arise because task completion depends on relationships among environmental evidence, consequential decisions, and required effects. To address this completion gap, we propose Interlock, a framework for task synthesis that represents each task as a typed completion graph. Grounding edges connect public evidence to decisions, coupling hyperedges impose joint constraints across components, and realization edges connect decisions to all required downstream effects. Interlock compiles this graph into a resettable workspace, a public request, and an evaluation rubric. A source-coverage review checks whether the original requirements survive generation, while a public-input review checks whether the rubric is supported by the materials available to the solver. An agent-as-judge inspects execution artifacts and runs…
PDF arXiv
arXiv cs.AIOct 14
MCAIR: Monitoring and Correcting Agent Unfaithfulness via Internal Representations
Yan Xu, Dandan Song, Luan Zhang, Jun Yang, Ke Duan +6
Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in completing complex tasks through multi-step reasoning and tool interactions. However, recent studies have revealed that LLM-based agents can exhibit unfaithful behaviors, where their final reports fail to faithfully reflect their actual execution trajectories. Specifically, when agents fail to accomplish tasks through the intended sources or procedures, they may resort to alternative sources or strategies while failing to disclose such deviations, leading users to incorrectly believe that the tasks have been completed according to the original requirements. We refer to this specific form of agent unfaithfulness as Agent Completion Unfaithfulness. Detecting and mitigating agent completion unfaithfulness is critical yet remains largely unexplored. In this work, we propose MCAIR, a runtime monitoring and correction framework that leverages internal representations of LLM-based agents. MCAIR uses multi-step internal representations as inputs to train a risk-aware probe, enabling online detection of potential completion unfaithfulness during execution. Furthermore, MCAIR introduces a risk-guided…
PDF arXiv

Papers

Top tools here

All tools

Agno5 releases this week
FastMCP6 releases this week
Pydantic AI+17% downloads · 1.5M/wk+17.0%
n8n+25% downloads · 148.3k/wk+25.0%
smolagents+19% downloads · 125.9k/wk+18.7%

Topics in this field

coding agents · code generation · coding agent303 papers83 this week
skills · tasks · ai315 papers56 this week
recursive self improvement · harness evolution · adaptive271 papers50 this week
ai agents · indirect prompt injection · adversarial319 papers42 this week
tool using agents · agent systems · multi agent216 papers41 this week
agent · agentic · llm agents172 papers40 this week

Recently funded

Market

FirstWorkSeries A · Oct 21$15M
Asymptote LabsSeed · Oct 20$5.5M
LumilabPre-seed · Oct 20undisclosed
GreptileSeries B+ · Oct 19$50M
VesslaPre-seed · Oct 19$3.3M

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.CROct 14
RedDarwin: Evolving Jailbreak Skills against Frontier LLMs
Zhouyuan Ma, Yutao Wu, Hanxun Huang, Xiang Zheng, Yi Liu +1
Red teaming must continually adapt as frontier LLM safeguards evolve, yet existing approaches largely search for adversarial prompts or optimize within fixed attack procedures. We observe that even when an attack becomes ineffective, its underlying mechanisms can remain useful and can be revised or recombined to construct stronger attacks. Based on this insight, we introduce RedDarwin, an agentic framework that shifts the unit of jailbreak evolution from prompts and fixed attacks to executable attack skills. RedDarwin consists of 47 reusable base skills and autonomously revises, composes, executes, and selects them using target-model feedback, allowing successful discoveries to become building blocks for subsequent evolution. RedDarwin produces 65 evolved skills, improving 44 of 47 base attacks and increasing their mean attack success rate (ASR) on AdvBench from 18.8% to 47.3%. These evolved skills consistently outperform baselines on the held-out StrongREJECT and frontier LLMs unseen during evolution; with only 10 target-model queries, RedDarwin achieves 100% ASR on GPT-5.6-Sol, 92% on Claude-Sonnet-5, and 80% on Gemini-3.8-Flash. Analysis of the evolution trajectories further…
PDF arXiv
arXiv cs.PLOct 14
Verifying deployed bytecode against executable EVM semantics, with exact gas and no added axioms
Seulkee Baek
Much of Ethereum's most consequential code is immutable bytecode: old output of unverified compilers, and protocol logic run as contract code. We show that such bytecode can be verified as it is, without trusting what produced it, without simplifying the machine it runs on, and with no axioms beyond Lean's own. Blanc, a Lean 4 verification stack, reads fixed runtime bytes via kernel-checked certificates and states every theorem against Jaune, an executable semantics of the EVM and block processing that passes all 34,205 blockchain-test cases of Ethereum's fixture release v20.0.2 that involve only forks that have governed mainnet since May 2025. Eight artifacts, deployed from 2017 to 2026, come from seven compiler versions and an assembler. For the EIP-7002 contract, we prove that the end-of-block call to it, whose failure would invalidate the block, succeeds whatever its storage holds, and that each block's withdrawal requests are exactly the oldest pending submissions; mainnet met these theorems' starting premises when Prague began. For a Uniswap V2 pair, we prove that after any valid history whose hashed slots stay apart, from any checkpoint representing its Solidity model's…
PDF arXiv
arXiv cs.AIOct 14
Runtime Monitoring of Supermartingale Certificates
Mirco Giacobbe, Thomas A. Henzinger, Konstantin Kueffner, Emily Yu, Đorđe Žikelić
Supermartingale certificates formally guarantee temporal properties of discrete and continuous-time stochastic dynamical systems. However, these guarantees rely on a mathematical model of the world, leaving a sim-to-real gap that can invalidate them for the deployed system. We formulate the runtime monitoring problem for supermartingale certificates, in which a monitor observes a single system trajectory and raises an alarm when there is sufficient statistical evidence that the supermartingale certificate has been violated during execution. We present an algorithm to automatically construct a monitor using betting-based sequential tests that bound the probability of ever raising a false alarm at a user-specified level. Within this betting framework, we formulate strategies that allow the monitor to exploit spatial regularity to improve detection power while preserving this statistical guarantee. We demonstrate the efficacy of our approach on supermartingale certificate benchmarks from the literature.
PDF arXiv
arXiv cs.AICodeOct 14▲ 28
RuleTrek: Benchmarking Agentic Rule Reasoning in Complex Scenarios
Chenhao Yuan, Bohan Yu, Yinhao Xu, Chao Yin, Chenxi Zhou +8
Large language models (LLMs) increasingly tackle complex real-world tasks in specialized domains. However, identifying and correctly applying domain-specific rules remains challenging when scenario information is distributed across multiple files. Existing rule-reasoning benchmarks typically represent scenarios using only a few factual statements, simplifying the identification and application of relevant rules. To address this gap, this paper introduces \textbf{RuleTrek}, a multi-domain benchmark for agentic rule reasoning in complex scenarios. Its rule-grounded construction pipeline extracts structured rule graphs, generates and incrementally composes rule-specific sub-scenarios while ensuring consistency and solvability. To systematically evaluate agentic rule reasoning in complex scenarios, RuleTrek assesses three capabilities in interactive workspaces: Rule Identification (RI), Rule Deduction (RD), and End-to-End Rule Reasoning (E2E). Experiments on 11 frontier LLMs show that the highest RI and RD scores are only 62.47\% and 60.43\%, respectively, while E2E scores range from 26.97\% to 55.29\%. These results demonstrate that current LLM agents face substantial challenges in…
PDF arXiv Code
arXiv cs.CLOct 14▲ 41
Interlock: From Local Correctness to Task Completion in Long-Horizon Agents
Linji Hao, Yanji He, Zihan Wang, Hansi Yang, Jiayuan Zhang +4
Large language model agents increasingly undertake long-horizon tasks in complex environments characterized by fragmented evidence, partially completed states, and dependencies across resources and artifacts. Existing agents may satisfy individual requirements while failing to maintain these relationships, producing locally correct but incomplete outcomes. These failures arise because task completion depends on relationships among environmental evidence, consequential decisions, and required effects. To address this completion gap, we propose Interlock, a framework for task synthesis that represents each task as a typed completion graph. Grounding edges connect public evidence to decisions, coupling hyperedges impose joint constraints across components, and realization edges connect decisions to all required downstream effects. Interlock compiles this graph into a resettable workspace, a public request, and an evaluation rubric. A source-coverage review checks whether the original requirements survive generation, while a public-input review checks whether the rubric is supported by the materials available to the solver. An agent-as-judge inspects execution artifacts and runs…
PDF arXiv
arXiv cs.AIOct 14
MCAIR: Monitoring and Correcting Agent Unfaithfulness via Internal Representations
Yan Xu, Dandan Song, Luan Zhang, Jun Yang, Ke Duan +6
Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in completing complex tasks through multi-step reasoning and tool interactions. However, recent studies have revealed that LLM-based agents can exhibit unfaithful behaviors, where their final reports fail to faithfully reflect their actual execution trajectories. Specifically, when agents fail to accomplish tasks through the intended sources or procedures, they may resort to alternative sources or strategies while failing to disclose such deviations, leading users to incorrectly believe that the tasks have been completed according to the original requirements. We refer to this specific form of agent unfaithfulness as Agent Completion Unfaithfulness. Detecting and mitigating agent completion unfaithfulness is critical yet remains largely unexplored. In this work, we propose MCAIR, a runtime monitoring and correction framework that leverages internal representations of LLM-based agents. MCAIR uses multi-step internal representations as inputs to train a risk-aware probe, enabling online detection of potential completion unfaithfulness during execution. Furthermore, MCAIR introduces a risk-guided…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 51m ago

Agents

LLM agents: planning, tool use, multi-agent systems, computer/web agents and agent benchmarks.

528papers this week−5.2% vs last week

All papers · 2.6k With code · 123

Newest Most relevant

2.4k papers this month

Open problems in Agents

Long-horizon task execution and memory3 papers raise it · 3 address it
Agents fail to retain, retrieve, and use past information over long interactions, limiting real-world deployment.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
- Agent-Reflex: Teaching Language Agents to Act Reflectivelyaddresses
Real-world task ambiguity and unstated constraints3 papers raise it · 2 address it
Agents struggle with ambiguous, incomplete, or evolving requirements in live systems, unlike controlled benchmarks.
- OR-License: Evaluating LLM Agents as Real-World Operations Research Practitionersraises · addresses
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Faithful agent development from requirements1 paper raises it · 1 addresses it
Coding agents often produce systems that appear correct but fail to meet the actual requirements.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Sequential distraction in multimodal GUI agents1 paper raises it · 1 addresses it
Omni-modal inputs cause agents to focus on irrelevant signals, leading to compounding errors.
- Focused Omni-Modal GUI Agentsraises · addresses
Open, reproducible agent stacks and evaluation1 paper raises it · 1 addresses it
Closed systems and lack of open benchmarks hinder systematic study and improvement of agent designs.
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses

Named by an AI from these papers' abstracts.