Papers

Field

Top tools here

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting supervision from diverse teachers degrades joint optimization and capability retention
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable policy-shift transfer in recursive improvement1 paper raises it · 1 addresses it
Endpoint policy shifts may hide inconsistent local action evolution, reducing transfer quality
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often lack justified, learnable observation-action dependencies for agents
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Scaling long-horizon agentic data synthesis1 paper raises it · 1 addresses it
Environment synthesis often omits key work, failing to capture exploration, feedback, and verification patterns
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
Harness self-evolution reliability1 paper raises it · 1 addresses it
Persistent harness revisions require continual validation to avoid regression and ensure safety
- Self-Evolving Agent Harnessraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

agent collaboration · multi agent · based multi agent274 papers25 this week
teacher policy distillation · policy distillation opd · teacher supervision409 papers23 this week
recursive self improvement · ai · agent588 papers22 this week
multi agent debate · ai agents · agent debate278 papers21 this week
conversational agents · user simulators · user simulation220 papers20 this week
scientific agents · research agents · ai agents243 papers19 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.AIOct 7
Agentic AI-Assisted Modeling for Production Scheduling: Assessment in Constraint Programming
Ángel Sánchez-Fernández, Javier Pernas-Álvarez, Diego Crespo-Pereira
Developing optimization models for production scheduling requires substantial expert effort. Research on large language models (LLMs) has followed two directions: specialized approaches for automated modeling, mostly for mixed-integer linear programming, which often rely on dedicated training or problem-specific architectures that limit industrial deployment; and agentic artificial intelligence for operational decision support, which generally assumes that the optimization model already exists. This study bridges both directions by assessing whether general-purpose LLMs, orchestrated as agents without task-specific training, can formulate and implement constraint programming models from natural-language problem descriptions. Singleagent and multi-agent architectures are integrated with a Model Context Protocol server that provides context-aware retrieval of solver documentation to mitigate hallucinations during implementation. Both are compared with a direct LLM baseline on six industry-oriented problems covering flow-shop, job-shop, flexible job-shop and resource-constrained warehouse scheduling, using three LLMs and assessing modeling accuracy, execution success, latency and…
PDF arXiv
arXiv cs.CLOct 7
Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis, Ruofei Lai, Jeff Z. Pan
Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising approach for post-training such agents, but its reliance on sparse, outcome-based supervision can make credit assignment difficult and limit learning efficiency. In this paper, we systematically investigate how intermediate supervision can improve reinforcement learning for search agents. We study a range of reward-shaping and credit-assignment strategies that provide learning signals from intermediate retrieval steps. Building on these insights, we develop a training framework that combines intermediate signals with final outcome rewards to improve learning from multi-step search trajectories. Experiments across multiple benchmarks under matched training conditions demonstrate improvements in aggregate search-agent performance and show that both the choice of intermediate signal and where its credit is assigned affect training behaviour. These findings show that reward design and credit assignment are important design dimensions for training effective search agents.
PDF arXiv
arXiv cs.AICodeOct 7▲ 8
UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Policy
Yifei Lu, Cheng Liu, Dianzhi Yu, Hui Xiang, Ji Zhang +2
Large language model agents can improve across tasks by retaining reusable skills distilled from prior interactions. Recent work jointly optimizes task execution and skill extraction, enabling the policy and skillbank to co-evolve. However, as the actor continues learning, rewarding skill proposals through their reuse in subsequent training steps may conflate skill benefits with actor improvement, while directly testing each proposed skill requires costly additional actor rollouts. In this paper, we introduce UniSkill, which uses a shared policy to interact with the environment and propose skillbank edits (Add, Update, or No Edit) from the resulting trajectories. Specifically, the actor learns from environment rewards, while contrastive action feedback guides skill proposal learning. This feedback provides an actor-alignment signal by measuring how replacing the retrieved skill with a proposed skill changes the current actor's action log-likelihood gap between previously collected successful and failed trajectories from the same task, thereby avoiding new rollouts for each proposal. Since proposal-level feedback may suppress an otherwise appropriate edit operation when the…
PDF arXiv Code
arXiv cs.AROct 7
When Algorithmic Exploration Becomes Cheap: A Case Study of Agentic Research in EDA
Keren Zhu, Yu Deng, Xiaoyu Hao, Liwen Jiang, Zijian Jiang +4
As EDA researchers, we conducted eight deliberate trials of agentic algorithm exploration, selecting several topics outside our areas of depth. One faculty member and seven students participated, including students without publication experience. With limited intervention in the algorithms, agents developed mathematical constructions, analyzed existing tools, and implemented improvements; some efforts fell short of their practical goals. We also used AI to collect, classify, and analyze 8,420 papers from four EDA conferences and two journals over 2022-2026. Among 2,380 primary-core papers, we classified 97.7% from titles and abstracts as computationally closed, including work on new formulations. Together, these observations suggest that much of EDA offers an executable environment for increasingly accessible algorithm research. We see an opportunity for tool developers to investigate ideas they previously lacked time to pursue. We also ask how EDA should validate and reward research when results become easier to produce than to examine, and what papers and venue labels will continue to tell us about a contribution.
PDF arXiv
arXiv cs.AIOct 7
RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation
Hengbo Xiao, Boyao Zhang, Purui Liu, Yuxuan Zheng, Haoran Yin +2
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in domains where task outcomes can be reliably evaluated, but long-horizon interaction remains challenging due to sparse terminal feedback and difficult credit assignment. Process rewards provide denser supervision, yet the capabilities most relevant for training can change as the policy evolves: a behavior that is easy to evaluate or frequently deficient need not be the bottleneck currently limiting task success. We introduce RewardWeaver, a self-evolving reward adaptation framework for language agents in long-horizon interaction. RewardWeaver maintains a validated capability space in which the semantics of admitted Rubrics remain fixed, and closes the loop between policy optimization, task evaluation, failure attribution, and reward adaptation. After each training stage, it performs outcome-grounded backward attribution on low-outcome trajectories, aggregates recurrent and policy-controlled capability bottlenecks, and dynamically selects the corresponding process rewards for the next stage. Recurrent failures not covered by the existing capability space trigger a separate, controlled expansion…
PDF arXiv
arXiv cs.AIOct 7
SkillSandbox: Skill Verification via Dynamic Scenario Synthesis
Serin Kim, Kwangwook Seo, Dokyung Song, Jinyoung Yeo, Dongha Lee
Self-evolving agents distill task-solving experience into skills for future reuse, but these skills can encode incorrect procedures or non-transferable knowledge. It is therefore critical to verify each skill's reusability: whether its guidance remains useful beyond the experience from which it was distilled. Such verification requires observing how a skill affects execution in new tasks, yet existing tasks may not expose the situations where the target skill can actually be exercised. To construct such situations, we propose SkillSandbox, a framework that dynamically synthesizes a task and its environment for each skill that are skill-relevant yet novel. A Proposer specifies the conditions to preserve and the source-specific details to vary, a Builder constructs an executable scenario, and a Verifier compares executions with and without the skill. The Verifier assesses executability, utility, and efficiency to assign a Keep or Reject verdict, determining whether the skill enters the library. Across ALFWorld and WebShop with three models, SkillSandbox consistently yields the strongest downstream performance and improved execution efficiency. Further analyses examine whether these…
PDF arXiv

Papers

Topics in this field

agent collaboration · multi agent · based multi agent274 papers25 this week
teacher policy distillation · policy distillation opd · teacher supervision409 papers23 this week
recursive self improvement · ai · agent588 papers22 this week
multi agent debate · ai agents · agent debate278 papers21 this week
conversational agents · user simulators · user simulation220 papers20 this week
scientific agents · research agents · ai agents243 papers19 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.AIOct 7
Agentic AI-Assisted Modeling for Production Scheduling: Assessment in Constraint Programming
Ángel Sánchez-Fernández, Javier Pernas-Álvarez, Diego Crespo-Pereira
Developing optimization models for production scheduling requires substantial expert effort. Research on large language models (LLMs) has followed two directions: specialized approaches for automated modeling, mostly for mixed-integer linear programming, which often rely on dedicated training or problem-specific architectures that limit industrial deployment; and agentic artificial intelligence for operational decision support, which generally assumes that the optimization model already exists. This study bridges both directions by assessing whether general-purpose LLMs, orchestrated as agents without task-specific training, can formulate and implement constraint programming models from natural-language problem descriptions. Singleagent and multi-agent architectures are integrated with a Model Context Protocol server that provides context-aware retrieval of solver documentation to mitigate hallucinations during implementation. Both are compared with a direct LLM baseline on six industry-oriented problems covering flow-shop, job-shop, flexible job-shop and resource-constrained warehouse scheduling, using three LLMs and assessing modeling accuracy, execution success, latency and…
PDF arXiv
arXiv cs.CLOct 7
Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis, Ruofei Lai, Jeff Z. Pan
Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising approach for post-training such agents, but its reliance on sparse, outcome-based supervision can make credit assignment difficult and limit learning efficiency. In this paper, we systematically investigate how intermediate supervision can improve reinforcement learning for search agents. We study a range of reward-shaping and credit-assignment strategies that provide learning signals from intermediate retrieval steps. Building on these insights, we develop a training framework that combines intermediate signals with final outcome rewards to improve learning from multi-step search trajectories. Experiments across multiple benchmarks under matched training conditions demonstrate improvements in aggregate search-agent performance and show that both the choice of intermediate signal and where its credit is assigned affect training behaviour. These findings show that reward design and credit assignment are important design dimensions for training effective search agents.
PDF arXiv
arXiv cs.AICodeOct 7▲ 8
UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Policy
Yifei Lu, Cheng Liu, Dianzhi Yu, Hui Xiang, Ji Zhang +2
Large language model agents can improve across tasks by retaining reusable skills distilled from prior interactions. Recent work jointly optimizes task execution and skill extraction, enabling the policy and skillbank to co-evolve. However, as the actor continues learning, rewarding skill proposals through their reuse in subsequent training steps may conflate skill benefits with actor improvement, while directly testing each proposed skill requires costly additional actor rollouts. In this paper, we introduce UniSkill, which uses a shared policy to interact with the environment and propose skillbank edits (Add, Update, or No Edit) from the resulting trajectories. Specifically, the actor learns from environment rewards, while contrastive action feedback guides skill proposal learning. This feedback provides an actor-alignment signal by measuring how replacing the retrieved skill with a proposed skill changes the current actor's action log-likelihood gap between previously collected successful and failed trajectories from the same task, thereby avoiding new rollouts for each proposal. Since proposal-level feedback may suppress an otherwise appropriate edit operation when the…
PDF arXiv Code
arXiv cs.AROct 7
When Algorithmic Exploration Becomes Cheap: A Case Study of Agentic Research in EDA
Keren Zhu, Yu Deng, Xiaoyu Hao, Liwen Jiang, Zijian Jiang +4
As EDA researchers, we conducted eight deliberate trials of agentic algorithm exploration, selecting several topics outside our areas of depth. One faculty member and seven students participated, including students without publication experience. With limited intervention in the algorithms, agents developed mathematical constructions, analyzed existing tools, and implemented improvements; some efforts fell short of their practical goals. We also used AI to collect, classify, and analyze 8,420 papers from four EDA conferences and two journals over 2022-2026. Among 2,380 primary-core papers, we classified 97.7% from titles and abstracts as computationally closed, including work on new formulations. Together, these observations suggest that much of EDA offers an executable environment for increasingly accessible algorithm research. We see an opportunity for tool developers to investigate ideas they previously lacked time to pursue. We also ask how EDA should validate and reward research when results become easier to produce than to examine, and what papers and venue labels will continue to tell us about a contribution.
PDF arXiv
arXiv cs.AIOct 7
RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation
Hengbo Xiao, Boyao Zhang, Purui Liu, Yuxuan Zheng, Haoran Yin +2
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in domains where task outcomes can be reliably evaluated, but long-horizon interaction remains challenging due to sparse terminal feedback and difficult credit assignment. Process rewards provide denser supervision, yet the capabilities most relevant for training can change as the policy evolves: a behavior that is easy to evaluate or frequently deficient need not be the bottleneck currently limiting task success. We introduce RewardWeaver, a self-evolving reward adaptation framework for language agents in long-horizon interaction. RewardWeaver maintains a validated capability space in which the semantics of admitted Rubrics remain fixed, and closes the loop between policy optimization, task evaluation, failure attribution, and reward adaptation. After each training stage, it performs outcome-grounded backward attribution on low-outcome trajectories, aggregates recurrent and policy-controlled capability bottlenecks, and dynamically selects the corresponding process rewards for the next stage. Recurrent failures not covered by the existing capability space trigger a separate, controlled expansion…
PDF arXiv
arXiv cs.AIOct 7
SkillSandbox: Skill Verification via Dynamic Scenario Synthesis
Serin Kim, Kwangwook Seo, Dokyung Song, Jinyoung Yeo, Dongha Lee
Self-evolving agents distill task-solving experience into skills for future reuse, but these skills can encode incorrect procedures or non-transferable knowledge. It is therefore critical to verify each skill's reusability: whether its guidance remains useful beyond the experience from which it was distilled. Such verification requires observing how a skill affects execution in new tasks, yet existing tasks may not expose the situations where the target skill can actually be exercised. To construct such situations, we propose SkillSandbox, a framework that dynamically synthesizes a task and its environment for each skill that are skill-relevant yet novel. A Proposer specifies the conditions to preserve and the source-specific details to vary, a Builder constructs an executable scenario, and a Verifier compares executions with and without the skill. The Verifier assesses executability, utility, and efficiency to assign a Keep or Reject verdict, determining whether the skill enters the library. Across ALFWorld and WebShop with three models, SkillSandbox consistently yields the strongest downstream performance and improved execution efficiency. Further analyses examine whether these…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 12m ago

Recursive self improvement

Found by the topic model: recursive self improvement, teacher policy distillation, multi agent debate.

250papers this week−29.4% vs last week

All papers · 3.3k With code · 243

Newest Most relevant

1.5k papers this month

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting supervision from diverse teachers degrades joint optimization and capability retention
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable policy-shift transfer in recursive improvement1 paper raises it · 1 addresses it
Endpoint policy shifts may hide inconsistent local action evolution, reducing transfer quality
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often lack justified, learnable observation-action dependencies for agents
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Scaling long-horizon agentic data synthesis1 paper raises it · 1 addresses it
Environment synthesis often omits key work, failing to capture exploration, feedback, and verification patterns
- Scaling Agentic Data for Long-Horizon Terminal Intelligenceraises · addresses
Harness self-evolution reliability1 paper raises it · 1 addresses it
Persistent harness revisions require continual validation to avoid regression and ensure safety
- Self-Evolving Agent Harnessraises · addresses

Named by an AI from these papers' abstracts.