Fields · AI · updated 53m ago

Agents & indirect prompt injection

Found by the topic model: agents, indirect prompt injection, code generation.

220papers this week−16.7% vs last week

All papers · 1.6k With code · 41

Newest Most relevant

1k papers this month

Papers

Agents & indirect prompt injection · ByteTrail

Fields · AI · updated 53m ago

Agents & indirect prompt injection

Found by the topic model: agents, indirect prompt injection, code generation.

220papers this week−16.7% vs last week

All papers · 1.6k With code · 41

Newest Most relevant

1k papers this month

Recent papers

All papers in Research

arXiv cs.SEOct 21
Metamodel-Guided LLM Decoding for Consistent DSL Generation
Boqi Chen, Aren A. Babikian, José Antonio Hernández López
Large language models (LLMs) are increasingly used to generate software engineering artifacts written in domain-specific languages (DSLs). Correct DSL instances usually must comply with not only the concrete syntax but also the abstract syntax defined by a metamodel augmented with semantic well-formedness constraints. Existing constrained decoding approaches typically guarantee compliance only with concrete syntax or with a limited set of semantic constraints expressed over the generated text, making metamodel-level constraint consistency difficult to enforce during generation. We present MeMoDecode: a decoding framework that enforces a DSL's concrete syntax, abstract syntax, and well-formedness constraints jointly. Specifically, whenever the LLM generates a complete metamodel concept, MeMoDecode incrementally maps the current output to a partial model and verifies whether it can still be completed into a consistent instance. We combine MeMoDecode with a simple retry mechanism that resumes decoding from the most recent completable snapshot whenever an inconsistency is detected. Under a sound backend verifier, MeMoDecode never performs worse than grammar-constrained decoding on…
PDF arXiv
arXiv cs.SEOct 21
Benchmarking Coding Agents for System-Level Performance Optimization
Yanze Zhang, Maggie Yongqi Guan, Shudong Liu, Qi Hao, Kanye Ye Wang +2
Coding agents are becoming an increasingly important paradigm for automating software engineering tasks. However, existing coding benchmarks primarily evaluate agents at the function or repository level, leaving their capabilities in more complex distributed software systems largely unexplored. In practice, many modern applications adopt microservice architectures, where end-to-end performance emerges from interactions among multiple independently running services, requiring agents to reason beyond local code changes. To study this capability, we introduce MicroPerf-Bench, a benchmark for evaluating coding agents on end-to-end performance optimization in microservice systems. Constructing such a benchmark is challenging because reproducible system-level performance fixes are sparse and difficult to attribute. We address this challenge by injecting empirically grounded performance anti-patterns into real-world open-source systems while preserving functionality and ensuring measurable performance degradation. The resulting benchmark contains 65 optimization instances across seven systems and nine anti-patterns, each with an executable environment, functional tests, and…
PDF arXiv
arXiv cs.DBOct 21
Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement
Kardelen Subasi, Jarosław A. Chudziak
LLM-generated records increasingly populate production information systems where they are treated as data assets subject to the same quality expectations as records from operational databases. A single such record can simultaneously satisfy and violate a quality rule: a grounded fact adjacent to a hallucination, a useful answer adjacent to a stereotype. Classical data cleaning cannot resolve this because the two judgments are incompatible over the same span; a single LLM judge cannot either, since it shares the training biases that produced the conflict. Existing interventions such as prompt engineering, chain-of-thought, and RLHF reduce surface error rates but provide no audit trail and cannot distinguish genuine remediation from blanket suppression. The question is whether a deterministic check over structured inter-agent records can catch the defects that referential checks alone miss. We present BiasGuard-D: two adversarial roles drawn from distinct model families exchange typed claim documents, a three-model quorum adjudicates by majority vote, and a combined referential-plus-semantic invariant validates the verdict. The semantic component fires on verdicts referential…
PDF arXiv
arXiv cs.AIOct 21
Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail?
Esmail Gumaan
When an LLM agent's move fails, the harness writes the move and the failure message into the transcript and asks again. In tool calling, this record makes small models more likely to repeat the failed action. We test whether the same holds in games (Sokoban, Connect Four, Othello), where a move can fail by being illegal or by being legal but losing, both verified exactly. On fixed probe items built by perturbing a correct move, the record backfires: pooled over 8 instruction-tuned models (135M-1.7B) it makes the failed move more likely (corrective gain G = -1.38 nats per token), through the move's surface form, for illegal moves but not for losing ones, and not for every model. An exploratory analysis shows what sets the sign: after the record, the failed move's log-probability barely depends on how likely it was before (slope 0.18). Moves the model would not have played become more likely, and greedy decoding starts to produce them; moves it was about to play become less likely, but greedy decoding still repeats 38% of them. A description of the failure that omits the move string keeps the model's preference instead. Along the OLMo 2 1B post-training ladder the inversion is…
PDF arXiv
arXiv cs.SEOct 21
Who Verifies the Agent's Success Claim? Self-Report Leakage in GUI-Agent Completion Judging
Yunfeng Zhu, Shengcheng Yu, Zihan Li, Zhenyu Chen
In automation pipelines, a GUI agent's confident success report can let downstream stages proceed even when the recorded execution trace does not show that the agent produced the requested outcome. We call this case a dangerous misreport: an unhedged success claim that the trace does not support as causal accomplishment during the run. We study report--trace auditing, which checks an agent's completion claim against an independently recorded trace. A controlled aware/blind intervention reveals self-report leakage: the agent's own self-report changes how completion is judged. The same judge reads the same task and trace; only that self-report is included or withheld. Withholding it improves trace-only dangerous-misreport recall by 43.4 percentage points with Qwen3.5-35B-A3B and 13.3 percentage points with DeepSeek-284B-A13B, with the same trend on held-out Batch 2. This motivates report isolation. The report records what the agent claims, but it must not influence the trace-state judgment. We implement this separation with a structured auditor that separates claim interpretation from claim-blind trace analysis and applies deterministic rules to extracted evidence. Because evidence…
PDF arXiv
arXiv cs.SEOct 21
On the Effectiveness of Snapshot Testing: An Empirical Study
Shun Fujita, Satoru Tomizawa, Yutaro Kashiwa, Bin Lin, Hajimu Iida
Software testing is essential for ensuring quality, yet testing complex front-end user interfaces is often time-consuming and difficult. Snapshot testing has emerged as a low-cost regression testing method that detects unexpected changes by comparing program behavior before and after modifications. Despite its growing adoption, empirical evidence regarding its actual effectiveness remains scarce. To fill this gap, this study presents a large-scale empirical investigation into the effectiveness of snapshot testing involving 340 GitHub repositories and a survey of 60 experienced practitioners. Our quantitative results demonstrate that combining snapshot and traditional non-snapshot tests significantly enhances code coverage, achieving a median statement coverage of 86.7%. We find that the two approaches are highly complementary, typically exercising distinct regions of the production code. Furthermore, test suites incorporating snapshot tests detected 1.3 times more mutants than those relying solely on traditional tests. Our qualitative insights from the developer survey reveal that practitioners value snapshot testing for its ease of implementation, ability to prevent regressions,…
PDF arXiv

Papers

Open problems in Agents & indirect prompt injection

Lack of harness-specific evaluation benchmarks3 papers raise it · 2 address it
Agent performance depends on both models and harnesses, but most benchmarks focus only on model capabilities, leaving harness reliability and failure attribution underexplored.
- Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attributionraises · addresses
- Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agentsraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises
Outcome-capability gap in evolving environments1 paper raises it · 1 addresses it
Evaluating agents by outcomes alone obscures the underlying capabilities, especially in dynamic settings where outcomes and mechanisms diverge.
- LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Marketsraises · addresses
Security of vibe-coded agent outputs1 paper raises it · 1 addresses it
Agents often produce functional but insecure code, as security is not explicitly addressed in requirements or evaluations.
- Last Secure Code Benchmark: Is the Code You Vibe Secure?raises · addresses
Evidence-to-action chain failures in tool use1 paper raises it · 1 addresses it
Agents may act without sufficient evidence or fail to complete multi-step workflows, leading to unreliable or unsafe outcomes.
- From Evidence to Action: How Tool-Using Agents Failraises · addresses
Long-horizon program synthesis and verification2 papers raise it · 2 address it
Agents struggle with scalable, verifiable training environments for long-horizon tasks like program reproduction or checker synthesis.
- ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agentsraises · addresses
- CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?raises · addresses

Named by an AI from these papers' abstracts.

Open problems

Topics in this field

agents · tool · multi agent441 papers54 this week
indirect prompt injection · prompt injection · ai agents433 papers49 this week
coding agents · coding agent · code generation246 papers38 this week
code generation · language models llms · compiler163 papers33 this week
proof assistant · proof search · theorem provers106 papers18 this week
specification generation · formal specifications · formalization91 papers18 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.SEOct 21
Metamodel-Guided LLM Decoding for Consistent DSL Generation
Boqi Chen, Aren A. Babikian, José Antonio Hernández López
Large language models (LLMs) are increasingly used to generate software engineering artifacts written in domain-specific languages (DSLs). Correct DSL instances usually must comply with not only the concrete syntax but also the abstract syntax defined by a metamodel augmented with semantic well-formedness constraints. Existing constrained decoding approaches typically guarantee compliance only with concrete syntax or with a limited set of semantic constraints expressed over the generated text, making metamodel-level constraint consistency difficult to enforce during generation. We present MeMoDecode: a decoding framework that enforces a DSL's concrete syntax, abstract syntax, and well-formedness constraints jointly. Specifically, whenever the LLM generates a complete metamodel concept, MeMoDecode incrementally maps the current output to a partial model and verifies whether it can still be completed into a consistent instance. We combine MeMoDecode with a simple retry mechanism that resumes decoding from the most recent completable snapshot whenever an inconsistency is detected. Under a sound backend verifier, MeMoDecode never performs worse than grammar-constrained decoding on…
PDF arXiv
arXiv cs.SEOct 21
Benchmarking Coding Agents for System-Level Performance Optimization
Yanze Zhang, Maggie Yongqi Guan, Shudong Liu, Qi Hao, Kanye Ye Wang +2
Coding agents are becoming an increasingly important paradigm for automating software engineering tasks. However, existing coding benchmarks primarily evaluate agents at the function or repository level, leaving their capabilities in more complex distributed software systems largely unexplored. In practice, many modern applications adopt microservice architectures, where end-to-end performance emerges from interactions among multiple independently running services, requiring agents to reason beyond local code changes. To study this capability, we introduce MicroPerf-Bench, a benchmark for evaluating coding agents on end-to-end performance optimization in microservice systems. Constructing such a benchmark is challenging because reproducible system-level performance fixes are sparse and difficult to attribute. We address this challenge by injecting empirically grounded performance anti-patterns into real-world open-source systems while preserving functionality and ensuring measurable performance degradation. The resulting benchmark contains 65 optimization instances across seven systems and nine anti-patterns, each with an executable environment, functional tests, and…
PDF arXiv
arXiv cs.DBOct 21
Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement
Kardelen Subasi, Jarosław A. Chudziak
LLM-generated records increasingly populate production information systems where they are treated as data assets subject to the same quality expectations as records from operational databases. A single such record can simultaneously satisfy and violate a quality rule: a grounded fact adjacent to a hallucination, a useful answer adjacent to a stereotype. Classical data cleaning cannot resolve this because the two judgments are incompatible over the same span; a single LLM judge cannot either, since it shares the training biases that produced the conflict. Existing interventions such as prompt engineering, chain-of-thought, and RLHF reduce surface error rates but provide no audit trail and cannot distinguish genuine remediation from blanket suppression. The question is whether a deterministic check over structured inter-agent records can catch the defects that referential checks alone miss. We present BiasGuard-D: two adversarial roles drawn from distinct model families exchange typed claim documents, a three-model quorum adjudicates by majority vote, and a combined referential-plus-semantic invariant validates the verdict. The semantic component fires on verdicts referential…
PDF arXiv
arXiv cs.AIOct 21
Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail?
Esmail Gumaan
When an LLM agent's move fails, the harness writes the move and the failure message into the transcript and asks again. In tool calling, this record makes small models more likely to repeat the failed action. We test whether the same holds in games (Sokoban, Connect Four, Othello), where a move can fail by being illegal or by being legal but losing, both verified exactly. On fixed probe items built by perturbing a correct move, the record backfires: pooled over 8 instruction-tuned models (135M-1.7B) it makes the failed move more likely (corrective gain G = -1.38 nats per token), through the move's surface form, for illegal moves but not for losing ones, and not for every model. An exploratory analysis shows what sets the sign: after the record, the failed move's log-probability barely depends on how likely it was before (slope 0.18). Moves the model would not have played become more likely, and greedy decoding starts to produce them; moves it was about to play become less likely, but greedy decoding still repeats 38% of them. A description of the failure that omits the move string keeps the model's preference instead. Along the OLMo 2 1B post-training ladder the inversion is…
PDF arXiv
arXiv cs.SEOct 21
Who Verifies the Agent's Success Claim? Self-Report Leakage in GUI-Agent Completion Judging
Yunfeng Zhu, Shengcheng Yu, Zihan Li, Zhenyu Chen
In automation pipelines, a GUI agent's confident success report can let downstream stages proceed even when the recorded execution trace does not show that the agent produced the requested outcome. We call this case a dangerous misreport: an unhedged success claim that the trace does not support as causal accomplishment during the run. We study report--trace auditing, which checks an agent's completion claim against an independently recorded trace. A controlled aware/blind intervention reveals self-report leakage: the agent's own self-report changes how completion is judged. The same judge reads the same task and trace; only that self-report is included or withheld. Withholding it improves trace-only dangerous-misreport recall by 43.4 percentage points with Qwen3.5-35B-A3B and 13.3 percentage points with DeepSeek-284B-A13B, with the same trend on held-out Batch 2. This motivates report isolation. The report records what the agent claims, but it must not influence the trace-state judgment. We implement this separation with a structured auditor that separates claim interpretation from claim-blind trace analysis and applies deterministic rules to extracted evidence. Because evidence…
PDF arXiv
arXiv cs.SEOct 21
On the Effectiveness of Snapshot Testing: An Empirical Study
Shun Fujita, Satoru Tomizawa, Yutaro Kashiwa, Bin Lin, Hajimu Iida
Software testing is essential for ensuring quality, yet testing complex front-end user interfaces is often time-consuming and difficult. Snapshot testing has emerged as a low-cost regression testing method that detects unexpected changes by comparing program behavior before and after modifications. Despite its growing adoption, empirical evidence regarding its actual effectiveness remains scarce. To fill this gap, this study presents a large-scale empirical investigation into the effectiveness of snapshot testing involving 340 GitHub repositories and a survey of 60 experienced practitioners. Our quantitative results demonstrate that combining snapshot and traditional non-snapshot tests significantly enhances code coverage, achieving a median statement coverage of 86.7%. We find that the two approaches are highly complementary, typically exercising distinct regions of the production code. Furthermore, test suites incorporating snapshot tests detected 1.3 times more mutants than those relying solely on traditional tests. Our qualitative insights from the developer survey reveal that practitioners value snapshot testing for its ease of implementation, ability to prevent regressions,…
PDF arXiv