Spotlight

Papers

Field

Top tools here

Open problems in AI agents

Long-horizon reasoning and memory in real-world agent tasks3 papers raise it · 3 address it
Agents fail to retain, retrieve, and use past context over long interactions, limiting their ability to solve multi-step or evolving tasks.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
- Agent-Reflex: Teaching Language Agents to Act Reflectivelyaddresses
Ambiguous or unstated requirements in real deployments2 papers raise it · 2 address it
Agents struggle with open-ended, real-world tasks where constraints and goals are unclear or must be discovered through interaction.
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
Agents cannot consistently suppress internal reasoning when instructed, leading to unpredictable behavior and wasted computation.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Verification of agent actions and transitions1 paper raises it · 1 addresses it
Agents lack robust methods to detect unsafe or incorrect transitions between states, especially in GUI or interactive environments.
- Right Screen, Wrong Transition: World Models as Verifiers for GUI Agentsraises · addresses
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often produce systems that appear to meet requirements but fail to deliver the intended behavior in practice.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

code generation · coding agents · software engineering366 papers83 this week
ai agents · agents · multi agent450 papers56 this week
reasoning tasks · reasoning paths · reasoning models360 papers47 this week
harness optimization · agent harnesses · harness evolution264 papers35 this week
ai safety · safety reasoning · safety alignment152 papers25 this week
agent skills · language model agents · skill evolution244 papers23 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

★ Spotlight this month

arXiv cs.AICodeOct 15▲ 25

MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao +5

Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes. We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE…

PDF arXiv page Details & code

Spotlight

Recent papers

All papers in Research

arXiv cs.AIOct 22
Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning
Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn
Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix, committing further computation only to the surviving candidates. Existing pruning strategies divide broadly into two categories: Quality-based methods assign a score to each partial solution and discard those predicted to ultimately lead to an incorrect final solution, while diversity-based methods select partial solutions that follow different reasoning trajectories and are therefore likely to cover different final solutions. The tension between these two objectives is an instance of exploration versus exploitation, yet little to no prior work on pruning exposes this trade-off directly, let alone optimizes it. In this paper, we introduce a pruning procedure that addresses this trade-off directly by optimizing pass@$k$ over the selected solution pool. In particular, it chooses the pool to maximize the probability that at least one…
PDF arXiv
arXiv stat.MLOct 22
Reliable AI-Assisted Decisions via Selective Auditing
Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text…
PDF arXiv
arXiv cs.AIOct 22
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Qijin Zhang, Hanzhen Wang, Yuehan Zhao, Haoyu Dai, Yao Shu +2
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but…
PDF arXiv
arXiv cs.LGOct 22
Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation
Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski
Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in…
PDF arXiv
arXiv cs.CLOct 22
Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
Marek Šuppa, Ivan Vykopal, Jaroslav Kopčan
Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. We use an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only, budgeted API and compiles each finding into an executable test, which a native speaker then validates. On the 26.5M-document Slovak portion of FineWeb2, three runs costing \$2.21-\$2.84 each found seven findings over four phenomena. The largest is a machine-generated content farm of 1,408,437 documents (5.32%) that passed the pipeline's filters, 86.6% of it passing even stricter thresholds. Blinded labels support 6 of the 7 findings (defect rates .53-1.00 against .08 in a random control) and do not support one. The compiled suite scans the full corpus in 0.7 s of wall-clock time without a GPU, against \$12k-\$90k per LLM classification pass. The best static sampling baseline validates more selectors than any agent run, at under half the cost. The agent contributes exact prevalence, a precision estimate checked against sampled documents, an account of why the pipeline missed each slice, and a reusable test.…
PDF arXiv
arXiv cs.LGOct 22
OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation
Qi Li, Xinchao Wang
On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures…
PDF arXiv

Papers

Topics in this field

code generation · coding agents · software engineering366 papers83 this week
ai agents · agents · multi agent450 papers56 this week
reasoning tasks · reasoning paths · reasoning models360 papers47 this week
harness optimization · agent harnesses · harness evolution264 papers35 this week
ai safety · safety reasoning · safety alignment152 papers25 this week
agent skills · language model agents · skill evolution244 papers23 this week

Active contributors · 90 days

★ Spotlight this month

arXiv cs.AICodeOct 15▲ 25

MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao +5

Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes. We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE…

PDF arXiv page Details & code

Activity · 12 weeks

Fields · AI · updated 41m ago

AI agents

Found by the topic model: ai agents, code generation, reasoning tasks.

388papers this week−41.1% vs last week

All papers · 3k With code · 146

Newest Most relevant

2.6k papers this month

Open problems in AI agents

Long-horizon reasoning and memory in real-world agent tasks3 papers raise it · 3 address it
Agents fail to retain, retrieve, and use past context over long interactions, limiting their ability to solve multi-step or evolving tasks.
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversationsraises
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
- Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?raises · addresses
- Agent-Reflex: Teaching Language Agents to Act Reflectivelyaddresses
Ambiguous or unstated requirements in real deployments2 papers raise it · 2 address it
Agents struggle with open-ended, real-world tasks where constraints and goals are unclear or must be discovered through interaction.
- FDE-Gym: The Last Mile of AI Deliveryraises · addresses
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
Agents cannot consistently suppress internal reasoning when instructed, leading to unpredictable behavior and wasted computation.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Verification of agent actions and transitions1 paper raises it · 1 addresses it
Agents lack robust methods to detect unsafe or incorrect transitions between states, especially in GUI or interactive environments.
- Right Screen, Wrong Transition: World Models as Verifiers for GUI Agentsraises · addresses
Faithful agent development from natural-language requirements1 paper raises it · 1 addresses it
Coding agents often produce systems that appear to meet requirements but fail to deliver the intended behavior in practice.
- AgentSWE: Can Coding Agents Build the Agent You Actually Want?raises · addresses

Named by an AI from these papers' abstracts.

Recent papers

All papers in Research

arXiv cs.AIOct 22
Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning
Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn
Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix, committing further computation only to the surviving candidates. Existing pruning strategies divide broadly into two categories: Quality-based methods assign a score to each partial solution and discard those predicted to ultimately lead to an incorrect final solution, while diversity-based methods select partial solutions that follow different reasoning trajectories and are therefore likely to cover different final solutions. The tension between these two objectives is an instance of exploration versus exploitation, yet little to no prior work on pruning exposes this trade-off directly, let alone optimizes it. In this paper, we introduce a pruning procedure that addresses this trade-off directly by optimizing pass@$k$ over the selected solution pool. In particular, it chooses the pool to maximize the probability that at least one…
PDF arXiv
arXiv stat.MLOct 22
Reliable AI-Assisted Decisions via Selective Auditing
Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text…
PDF arXiv
arXiv cs.AIOct 22
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Qijin Zhang, Hanzhen Wang, Yuehan Zhao, Haoyu Dai, Yao Shu +2
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but…
PDF arXiv
arXiv cs.LGOct 22
Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation
Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski
Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in…
PDF arXiv
arXiv cs.CLOct 22
Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
Marek Šuppa, Ivan Vykopal, Jaroslav Kopčan
Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. We use an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only, budgeted API and compiles each finding into an executable test, which a native speaker then validates. On the 26.5M-document Slovak portion of FineWeb2, three runs costing \$2.21-\$2.84 each found seven findings over four phenomena. The largest is a machine-generated content farm of 1,408,437 documents (5.32%) that passed the pipeline's filters, 86.6% of it passing even stricter thresholds. Blinded labels support 6 of the 7 findings (defect rates .53-1.00 against .08 in a random control) and do not support one. The compiled suite scans the full corpus in 0.7 s of wall-clock time without a GPU, against \$12k-\$90k per LLM classification pass. The best static sampling baseline validates more selectors than any agent run, at under half the cost. The agent contributes exact prevalence, a precision estimate checked against sampled documents, an account of why the pipeline missed each slice, and a reusable test.…
PDF arXiv
arXiv cs.LGOct 22
OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation
Qi Li, Xinchao Wang
On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures…
PDF arXiv