Papers

Field

Top tools here

Open problems in Agents & indirect prompt injection

Lack of harness-specific evaluation benchmarks3 papers raise it · 2 address it
Agent performance depends on both models and harnesses, but most benchmarks focus only on model capabilities, leaving harness reliability and failure attribution underexplored.
- Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attributionraises · addresses
- Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agentsraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Outcome-capability gap in evolving environments1 paper raises it · 1 addresses it
Evaluating agents by outcomes alone obscures the underlying capabilities, especially in dynamic settings where outcomes and mechanisms diverge.
- LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Marketsraises · addresses
Security of vibe-coded agent outputs1 paper raises it · 1 addresses it
Agents often produce functional but insecure code, as security is not explicitly addressed in requirements or evaluations.
- Last Secure Code Benchmark: Is the Code You Vibe Secure?raises · addresses
Evidence-to-action chain failures in tool use1 paper raises it · 1 addresses it
Agents may act without sufficient evidence or fail to complete multi-step workflows, leading to unreliable or unsafe outcomes.
- From Evidence to Action: How Tool-Using Agents Failraises · addresses
Long-horizon program synthesis and verification2 papers raise it · 2 address it
Agents struggle with scalable, verifiable training environments for long-horizon tasks like program reproduction or checker synthesis.
- ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agentsraises · addresses
- CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?raises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

agents · tool · multi agent441 papers55 this week
indirect prompt injection · prompt injection · ai agents433 papers54 this week
coding agents · coding agent · code generation246 papers40 this week
code generation · language models llms · compiler163 papers35 this week
specification generation · formal specifications · formalization91 papers19 this week
proof assistant · proof search · theorem provers106 papers18 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.SEOct 22
SWE-Clar: Can Coding Agents Clarify Incomplete Client Requirements and Deliver Greenfield Software?
Jialei Li, Dingxian Wang, Xiaoyan Zhao, Yimeng Bai, Xuanyi Zhu +9
Greenfield software development in freelance settings requires coding agents to turn incomplete client requests into working software. This makes requirement clarification essential, yet clients may be unable to provide continuous feedback over long development cycles, and receiving their answers does not guarantee correct understanding of complex requirements. These challenges motivate two benchmark design choices: clarification precedes code implementation, and its effectiveness is assessed through explicit tests of requirement understanding. Guided by these choices, we introduce SWE-Clar, a benchmark grounded in real, high-value client commissions to evaluate requirement clarification and code implementation in this setting. It comprises 100 tasks based on real software development jobs on Upwork, each with at least US\$4,000 in total payments. Each task begins with multi-round interaction between a coding agent and a client agent, followed by separate runs for code implementation with acceptance testing and clarification assessment. For the latter, we construct 512 questions paired with reference answers from real client-freelancer interactions, covering both explicitly…
PDF arXiv
arXiv cs.CROct 22
Ulysses: A Security Harness For Synthesizing Verifiable Web Exploits
Aditya Oke, André Vicente Duarte, Rui Melo, Shubham Gandhi, Nachiket Kotalwar +4
How can an AI agent automatically discover security vulnerabilities in any software repository? Existing approaches use LLMs to generate software vulnerability exploits. However, it is challenging for an LLM to reliably verify its own generated exploits. In this paper, we propose Ulysses, a multi-agent security harness to synthesize verifiable vulnerability exploits for web applications. Ulysses converts candidate vulnerability findings into verified Proofs of Exploitability by iteratively exploring exploit hypotheses and establishing success in independently observed execution effects. We evaluated Ulysses on WebExploitGym, a curated benchmark of exploitable instances from 25 real-world web applications that span 5 languages and 7 frameworks. Ulysses produces verified proofs for 10 of 25 real-world advisories. It also discovered 4 new proofs of exploitability that have never been previously reported.
PDF arXiv
arXiv cs.SEOct 22
ToolContractGate: Versioned Trusted Contracts for Drift-Safe Agent Tool Invocation
Sachin Gupta
Agent tool approval and input validation do not, by themselves, bind a request to an approved source artifact. ToolContractGate provides a deterministic runtime contract evaluator for OpenAPI-derived agent tools. Each versioned ToolContract combines a stored OpenAPI fingerprint with access, schema, and behavioral declarations. The full evaluator applies ten checks, including a cached boot-time provenance result; four execution modes and five evaluation configurations support baseline, ablation, and non-blocking shadow analysis. On a synthetic benchmark of 45 cases (35 unsafe cases across 12 drift classes and 10 safe cases), the full pipeline detects all 35 faults: 29 BLOCK, 3 REVIEW, and 3 WARN decisions, with no false blocks on the safe cases. The archived run reports a mean gate latency of 0.18 ms and a 95th percentile (p95) of 2 ms using integer-millisecond measurements. The open-source artifact includes seven OpenAPI specifications, twelve enterprise contracts, 29 unit tests, and a 900-record evaluation. Results characterize contract-decision coverage; downstream enforcement requires a host that honors the returned decision. Production traffic, semantic backend drift, and…
PDF arXiv
arXiv cs.AIOct 22▲ 8
PaperFormalizer: Author-in-the-Loop Autoformalization of Research Papers in Lean
Jiyuan Tan, Vasilis Syrgkanis, Jose Blanchet
Even a carefully proofread paper can leave small gaps in its proofs or its stated assumptions. We introduce PaperFormalizer, a Lean-based agentic pipeline that helps authors find and close such gaps in their own papers and turn each paper into a Lean-verified version. Starting from the LaTeX source, PaperFormalizer transcribes the paper into a dependency graph of statements, audits each statement in natural language for faithfulness, ambiguity and correctness, and then formalizes and proves the statements in Lean against Mathlib, optional domain libraries and the author's personal toolkit. Only the author knows what a statement is meant to say and how a formalization gap in a result is best fixed. The pipeline therefore stops and asks the author whenever a statement admits more than one reading, presents every repair, in natural language and TeX symbols, as a proposal that the author can accept, decline or redirect, and leaves to the author every decision that could potentially change what the paper claims. The output is an annotated and revised version of the paper, a report of the changes, and Lean code in which each declaration is linked to the statement it formalizes. The…
PDF arXiv
arXiv cs.LGOct 22
CuraLoc: Repository-Level Fault Localization via Planned Code Inspection
Sangeeta Warrier, Anush Sankaran, Ryen White
A bug report describes what went wrong, but rarely where the code should change. Existing repository-level localization methods often let language models iteratively decide what code to inspect next, requiring repeated model calls and substantial token usage. We investigate whether these inspection decisions can instead be planned before source retrieval. In this research, we introduce CuraLoc, which first selects candidate files and represents their symbols using compact behavioral and structural descriptions. In a single planning decision, the model jointly selects starting symbols, chooses code relationships to inspect from each, and forms hypotheses about likely edits. Deterministic retrieval collects the requested source for a final ranking informed by those hypotheses, concentrating model reasoning on choosing and comparing code rather than directing each retrieval step. We demonstrate the efficacy of our proposed approach on the SWE-Bench Lite and Verified datasets. On SWE Bench Verified with GPT-5.2, CuraLoc raises Func@1 from LocAgent's 53.8% to 59.3% while using 90.3% fewer recorded tokens, and also outperforms RepoSearcher (56.2%) and Agentless (39.6%) on Func@1. On…
PDF arXiv
arXiv stat.MLOct 22
Reliable AI-Assisted Decisions via Selective Auditing
Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text…
PDF arXiv

Papers

Topics in this field

agents · tool · multi agent441 papers55 this week
indirect prompt injection · prompt injection · ai agents433 papers54 this week
coding agents · coding agent · code generation246 papers40 this week
code generation · language models llms · compiler163 papers35 this week
specification generation · formal specifications · formalization91 papers19 this week
proof assistant · proof search · theorem provers106 papers18 this week

Recent papers

All papers in Research

arXiv cs.SEOct 22
SWE-Clar: Can Coding Agents Clarify Incomplete Client Requirements and Deliver Greenfield Software?
Jialei Li, Dingxian Wang, Xiaoyan Zhao, Yimeng Bai, Xuanyi Zhu +9
Greenfield software development in freelance settings requires coding agents to turn incomplete client requests into working software. This makes requirement clarification essential, yet clients may be unable to provide continuous feedback over long development cycles, and receiving their answers does not guarantee correct understanding of complex requirements. These challenges motivate two benchmark design choices: clarification precedes code implementation, and its effectiveness is assessed through explicit tests of requirement understanding. Guided by these choices, we introduce SWE-Clar, a benchmark grounded in real, high-value client commissions to evaluate requirement clarification and code implementation in this setting. It comprises 100 tasks based on real software development jobs on Upwork, each with at least US\$4,000 in total payments. Each task begins with multi-round interaction between a coding agent and a client agent, followed by separate runs for code implementation with acceptance testing and clarification assessment. For the latter, we construct 512 questions paired with reference answers from real client-freelancer interactions, covering both explicitly…
PDF arXiv
arXiv cs.CROct 22
Ulysses: A Security Harness For Synthesizing Verifiable Web Exploits
Aditya Oke, André Vicente Duarte, Rui Melo, Shubham Gandhi, Nachiket Kotalwar +4
How can an AI agent automatically discover security vulnerabilities in any software repository? Existing approaches use LLMs to generate software vulnerability exploits. However, it is challenging for an LLM to reliably verify its own generated exploits. In this paper, we propose Ulysses, a multi-agent security harness to synthesize verifiable vulnerability exploits for web applications. Ulysses converts candidate vulnerability findings into verified Proofs of Exploitability by iteratively exploring exploit hypotheses and establishing success in independently observed execution effects. We evaluated Ulysses on WebExploitGym, a curated benchmark of exploitable instances from 25 real-world web applications that span 5 languages and 7 frameworks. Ulysses produces verified proofs for 10 of 25 real-world advisories. It also discovered 4 new proofs of exploitability that have never been previously reported.
PDF arXiv
arXiv cs.SEOct 22
ToolContractGate: Versioned Trusted Contracts for Drift-Safe Agent Tool Invocation
Sachin Gupta
Agent tool approval and input validation do not, by themselves, bind a request to an approved source artifact. ToolContractGate provides a deterministic runtime contract evaluator for OpenAPI-derived agent tools. Each versioned ToolContract combines a stored OpenAPI fingerprint with access, schema, and behavioral declarations. The full evaluator applies ten checks, including a cached boot-time provenance result; four execution modes and five evaluation configurations support baseline, ablation, and non-blocking shadow analysis. On a synthetic benchmark of 45 cases (35 unsafe cases across 12 drift classes and 10 safe cases), the full pipeline detects all 35 faults: 29 BLOCK, 3 REVIEW, and 3 WARN decisions, with no false blocks on the safe cases. The archived run reports a mean gate latency of 0.18 ms and a 95th percentile (p95) of 2 ms using integer-millisecond measurements. The open-source artifact includes seven OpenAPI specifications, twelve enterprise contracts, 29 unit tests, and a 900-record evaluation. Results characterize contract-decision coverage; downstream enforcement requires a host that honors the returned decision. Production traffic, semantic backend drift, and…
PDF arXiv
arXiv cs.AIOct 22▲ 8
PaperFormalizer: Author-in-the-Loop Autoformalization of Research Papers in Lean
Jiyuan Tan, Vasilis Syrgkanis, Jose Blanchet
Even a carefully proofread paper can leave small gaps in its proofs or its stated assumptions. We introduce PaperFormalizer, a Lean-based agentic pipeline that helps authors find and close such gaps in their own papers and turn each paper into a Lean-verified version. Starting from the LaTeX source, PaperFormalizer transcribes the paper into a dependency graph of statements, audits each statement in natural language for faithfulness, ambiguity and correctness, and then formalizes and proves the statements in Lean against Mathlib, optional domain libraries and the author's personal toolkit. Only the author knows what a statement is meant to say and how a formalization gap in a result is best fixed. The pipeline therefore stops and asks the author whenever a statement admits more than one reading, presents every repair, in natural language and TeX symbols, as a proposal that the author can accept, decline or redirect, and leaves to the author every decision that could potentially change what the paper claims. The output is an annotated and revised version of the paper, a report of the changes, and Lean code in which each declaration is linked to the statement it formalizes. The…
PDF arXiv
arXiv cs.LGOct 22
CuraLoc: Repository-Level Fault Localization via Planned Code Inspection
Sangeeta Warrier, Anush Sankaran, Ryen White
A bug report describes what went wrong, but rarely where the code should change. Existing repository-level localization methods often let language models iteratively decide what code to inspect next, requiring repeated model calls and substantial token usage. We investigate whether these inspection decisions can instead be planned before source retrieval. In this research, we introduce CuraLoc, which first selects candidate files and represents their symbols using compact behavioral and structural descriptions. In a single planning decision, the model jointly selects starting symbols, chooses code relationships to inspect from each, and forms hypotheses about likely edits. Deterministic retrieval collects the requested source for a final ranking informed by those hypotheses, concentrating model reasoning on choosing and comparing code rather than directing each retrieval step. We demonstrate the efficacy of our proposed approach on the SWE-Bench Lite and Verified datasets. On SWE Bench Verified with GPT-5.2, CuraLoc raises Func@1 from LocAgent's 53.8% to 59.3% while using 90.3% fewer recorded tokens, and also outperforms RepoSearcher (56.2%) and Agentless (39.6%) on Func@1. On…
PDF arXiv
arXiv stat.MLOct 22
Reliable AI-Assisted Decisions via Selective Auditing
Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text…
PDF arXiv

Active contributors · 90 days

Activity · 12 weeks

Fields · AI · updated 7m ago

Agents & indirect prompt injection

Found by the topic model: agents, indirect prompt injection, code generation.

232papers this week−11.8% vs last week

All papers · 1.6k With code · 41

Newest Most relevant

1k papers this month

Open problems in Agents & indirect prompt injection

Lack of harness-specific evaluation benchmarks3 papers raise it · 2 address it
Agent performance depends on both models and harnesses, but most benchmarks focus only on model capabilities, leaving harness reliability and failure attribution underexplored.
- Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attributionraises · addresses
- Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agentsraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Outcome-capability gap in evolving environments1 paper raises it · 1 addresses it
Evaluating agents by outcomes alone obscures the underlying capabilities, especially in dynamic settings where outcomes and mechanisms diverge.
- LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Marketsraises · addresses
Security of vibe-coded agent outputs1 paper raises it · 1 addresses it
Agents often produce functional but insecure code, as security is not explicitly addressed in requirements or evaluations.
- Last Secure Code Benchmark: Is the Code You Vibe Secure?raises · addresses
Evidence-to-action chain failures in tool use1 paper raises it · 1 addresses it
Agents may act without sufficient evidence or fail to complete multi-step workflows, leading to unreliable or unsafe outcomes.
- From Evidence to Action: How Tool-Using Agents Failraises · addresses
Long-horizon program synthesis and verification2 papers raise it · 2 address it
Agents struggle with scalable, verifiable training environments for long-horizon tasks like program reproduction or checker synthesis.
- ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agentsraises · addresses
- CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?raises · addresses

Named by an AI from these papers' abstracts.