Papers

Field

Top tools here

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting teacher signals degrade joint optimization and cause capability loss when training a single student from multiple teachers.
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable supervision in cross-tokenizer distillation1 paper raises it · 1 addresses it
Vocabulary mismatch and alignment coverage issues make teacher feedback noisy or harmful for student training.
- Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliabilityraises · addresses
Inconsistent policy shifts during teacher training1 paper raises it · 1 addresses it
Endpoint policy shifts may hide early and late preference changes that cancel out, leading to poor transfer.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often include actions unsupported by the agent's visible history, reducing learning effectiveness.
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Credit assignment in long-horizon agent training1 paper raises it · 1 addresses it
Critic values vary across turns, making fine-grained credit allocation and policy updates difficult.
- Multiscale Policy Optimization for Long-Horizon LLM Agentsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

recursive self improvement · ai · agent520 papers43 this week
multi agent debate · ai agents · agent debate231 papers22 this week
teacher policy distillation · policy distillation opd · teacher supervision354 papers20 this week
scientific agents · research agents · ai agents196 papers20 this week
agent collaboration · multi agent · based multi agent206 papers19 this week
conversational agents · user simulators · user simulation173 papers19 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.MAOct 28
MAS-Loop: Beyond One-Shot Orchestration towards Self-Evolving Multi-Agent Systems
Hehai Lin, Qi Yang, Chengwei Qin
Training-time multi-agent system (MAS) design seeks to learn reusable orchestration policies, yet prevailing approaches remain largely one-shot: an Orchestrator generates a system but only observes and learns from its terminal outcome. This paradigm discards execution trajectories that expose system-level failures and contain fine-grained information beyond the terminal outcome, leaving no direct basis for revising the source MAS, evaluating the updated MAS, or learning from the resulting contrast. To this end, we introduce MAS-Loop, an execution-grounded framework that closes this feedback loop by coupling an Orchestrator with a trajectory-conditioned Refiner. For each task, the Orchestrator constructs and executes a source MAS to obtain an outcome score and an intermediate trajectory. Conditioned on the trajectory, the Refiner localizes an actionable failure, formulates a trajectory-grounded diagnosis, and proposes a bounded patch to the agents, information flow, or execution configuration. The executable updated MAS is re-executed on the same task to measure its realized outcome under stochastic execution. This pair-MAS comparison provides complementary learning signals: the…
PDF arXiv
arXiv cs.CYOct 28
TrussTutor-Bench: Benchmarking large language models on solution accuracy, sycophancy, and error detection in statics
Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune
Large language models (LLMs) have rapidly become a default source of help for students in science and engineering courses. In foundational engineering mechanics courses such as statics, students routinely use these tools to check answers, work through partial solutions, and ask for explanations. Effective tutoring in this setting requires more than a correct final answer. A model must also confirm correct work when a student doubts it, and identify errors in flawed work. Existing benchmarks evaluate LLMs primarily on final-answer accuracy, so it is unclear whether models that can solve mechanics problems also have the other technical capabilities that tutoring depends on. Here we introduce TrussTutor-Bench, a benchmark of 200 verified two-dimensional truss problems paired with automatically generated student solutions, and use it to evaluate thirteen LLMs on solution accuracy, sycophancy, and error detection. Baseline accuracy ranged from 64.5% to 97.0%. Every model detected nearly all injected errors in synthetic flawed student work (90.4% to 100%), so error detection did not distinguish models under these clean conditions. Sycophancy did. When a student insisted that correct…
PDF arXiv
arXiv cs.AIOct 28
Aligning Autoresearch: Co-Improvement between Agents and Humans
Tianjian Li, Chuanyang Jin, Seungone Kim, Tong Chen, John Hoffman +8
Autoresearch agents can now attempt long-horizon AI research end-to-end, i.e., self-improvement. However, best results are still obtained by research agents interacting with human researchers, and improving AI's ability to perform this co-research will only reinforce this effect---a process termed co-improvement (Weston and Foerster, 2025). Further, aligning with human expectations is likely a safer path for research agents, especially given recent agent reward hacking phenomena (OpenAI, 2026b). In this work we address these issues in four ways: (i) we empirically study the role of humans in directing autoresearch agents, focusing on the domain of benchmark creation, and find humans still play a pivotal role particularly in the ideation phase; (ii) we also develop a benchmark, HackBench, that shows current autoresearch agents are not aligned and are unable to achieve hack localization sufficiently; (iii) we present a method to train agents for improved human interaction by using human simulators as a proxy during reinforcement learning (RL); and finally (iv) we develop a method to improve model text generation, and hence agent communication, by RL with rubrics aligned with human…
PDF arXiv
arXiv cs.AIOct 28
Learning to Reason from Reference Samples Alone: Reinforced Adversarial Learning for Modern LLM Post-Training
Albert Bou, Javier Valdes-Aleman, Jon M. Laurent, Samuel G. Rodriques, Andrew D. White +2
In many settings where LLM post-training is desirable, high-quality answers are recognizable but difficult to score with explicit rewards. Reinforcement learning with verifiable rewards (RLVR) is limited to tasks where an explicit correctness signal is available. Prior adversarial approaches to training reasoning LLMs either complement verifiable rewards with co-trained critics or require reference reasoning traces. We ask whether a co-trained LLM judge can turn reference answers alone into the sole reward for on-policy post-training of reasoning LLMs. We call this method Reinforced Adversarial Learning (RAL). We first validate RAL on three verifiable tasks (integer multiplication, code-based arithmetic-expression synthesis, and SMILES-to-formula translation), using the verifiers only for evaluation. With sufficiently capable judges and enough training, RAL matches direct RLVR. We then apply RAL to scientific abstract generation in two settings. In the first setting, RAL guides the generator to produce abstracts in the scientific domain of the reference abstracts, even though the prompt leaves that domain unspecified. In the second setting, we generate abstracts for U.S. National…
PDF arXiv
arXiv stat.MLOct 28
Pinpointing Pivot Points: The DECIDE Framework for Data Analysis Decision Importances
Zachary T. Rewolinski, Jan Simson, Frauke Kreuter, Christoph Kern, Bin Yu
Human judgment calls made throughout the data science life cycle can change the conclusions drawn from data. Independent analysts often make different choices when given the same data, leading to often unacceptable variations in results and contributing to the ongoing reproducibility crisis across scientific disciplines. The rapid adoption of agentic tools threatens to amplify this crisis. Several approaches have been developed to address this variation by following the same strategy: repeating an analysis under different combinations of analytical decisions and inspecting the distribution of outcomes. Examples include multiverse analysis, specification curve analysis, and vibration of effects. More broadly, the Predictability-Computability-Stability (PCS) framework assesses how documented judgment calls across the data science life cycle affect conclusions, with predictability serving as a reality-check that a model captures signal. Each of these approaches results in a "multiverse" object. While the distribution of outcomes reflects the overall variability of conclusions, estimating the influence of each individual decision requires additional tools. A common heuristic is to…
PDF arXiv
arXiv cs.ROOct 28
AtomTree: A Hierarchical Framework for State-Aware Embodied Instruction Following with LLMs
Haotian Zhai, Tianming Sha, Junnan Li
Embodied instruction following in interactive environments requires agents to interpret natural language commands and complete long-horizon tasks. Existing planners either rely on large labeled trajectories, or use few-shot and zero-shot LLM prompting that produces a whole plan in a single serial reasoning process. These approaches depend on linguistic common sense, commit to a plan from the initial instruction, and only update after failures, which weakens robustness and state awareness. We present AtomTree, a hierarchical framework for state-aware task planning with LLMs. AtomTree incrementally builds a tree of subgoals and actions, where the LLM decides at each step, from the current symbolic state, whether to further decompose or to output a grounded low-level action. A lightweight memory module summarizes past outcomes for reuse in later decisions. When a low-level action stalls, AtomTree triggers hierarchical backtracking replanning to return to the parent node and regenerate the subtree plan from the latest state, preventing continued drift along an incorrect trajectory. In zero-shot evaluation on the ALFRED benchmark, AtomTree outperforms prior LLM-based planners. Project…
PDF arXiv

Papers

Topics in this field

recursive self improvement · ai · agent520 papers43 this week
multi agent debate · ai agents · agent debate231 papers22 this week
teacher policy distillation · policy distillation opd · teacher supervision354 papers20 this week
scientific agents · research agents · ai agents196 papers20 this week
agent collaboration · multi agent · based multi agent206 papers19 this week
conversational agents · user simulators · user simulation173 papers19 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.MAOct 28
MAS-Loop: Beyond One-Shot Orchestration towards Self-Evolving Multi-Agent Systems
Hehai Lin, Qi Yang, Chengwei Qin
Training-time multi-agent system (MAS) design seeks to learn reusable orchestration policies, yet prevailing approaches remain largely one-shot: an Orchestrator generates a system but only observes and learns from its terminal outcome. This paradigm discards execution trajectories that expose system-level failures and contain fine-grained information beyond the terminal outcome, leaving no direct basis for revising the source MAS, evaluating the updated MAS, or learning from the resulting contrast. To this end, we introduce MAS-Loop, an execution-grounded framework that closes this feedback loop by coupling an Orchestrator with a trajectory-conditioned Refiner. For each task, the Orchestrator constructs and executes a source MAS to obtain an outcome score and an intermediate trajectory. Conditioned on the trajectory, the Refiner localizes an actionable failure, formulates a trajectory-grounded diagnosis, and proposes a bounded patch to the agents, information flow, or execution configuration. The executable updated MAS is re-executed on the same task to measure its realized outcome under stochastic execution. This pair-MAS comparison provides complementary learning signals: the…
PDF arXiv
arXiv cs.CYOct 28
TrussTutor-Bench: Benchmarking large language models on solution accuracy, sycophancy, and error detection in statics
Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune
Large language models (LLMs) have rapidly become a default source of help for students in science and engineering courses. In foundational engineering mechanics courses such as statics, students routinely use these tools to check answers, work through partial solutions, and ask for explanations. Effective tutoring in this setting requires more than a correct final answer. A model must also confirm correct work when a student doubts it, and identify errors in flawed work. Existing benchmarks evaluate LLMs primarily on final-answer accuracy, so it is unclear whether models that can solve mechanics problems also have the other technical capabilities that tutoring depends on. Here we introduce TrussTutor-Bench, a benchmark of 200 verified two-dimensional truss problems paired with automatically generated student solutions, and use it to evaluate thirteen LLMs on solution accuracy, sycophancy, and error detection. Baseline accuracy ranged from 64.5% to 97.0%. Every model detected nearly all injected errors in synthetic flawed student work (90.4% to 100%), so error detection did not distinguish models under these clean conditions. Sycophancy did. When a student insisted that correct…
PDF arXiv
arXiv cs.AIOct 28
Aligning Autoresearch: Co-Improvement between Agents and Humans
Tianjian Li, Chuanyang Jin, Seungone Kim, Tong Chen, John Hoffman +8
Autoresearch agents can now attempt long-horizon AI research end-to-end, i.e., self-improvement. However, best results are still obtained by research agents interacting with human researchers, and improving AI's ability to perform this co-research will only reinforce this effect---a process termed co-improvement (Weston and Foerster, 2025). Further, aligning with human expectations is likely a safer path for research agents, especially given recent agent reward hacking phenomena (OpenAI, 2026b). In this work we address these issues in four ways: (i) we empirically study the role of humans in directing autoresearch agents, focusing on the domain of benchmark creation, and find humans still play a pivotal role particularly in the ideation phase; (ii) we also develop a benchmark, HackBench, that shows current autoresearch agents are not aligned and are unable to achieve hack localization sufficiently; (iii) we present a method to train agents for improved human interaction by using human simulators as a proxy during reinforcement learning (RL); and finally (iv) we develop a method to improve model text generation, and hence agent communication, by RL with rubrics aligned with human…
PDF arXiv
arXiv cs.AIOct 28
Learning to Reason from Reference Samples Alone: Reinforced Adversarial Learning for Modern LLM Post-Training
Albert Bou, Javier Valdes-Aleman, Jon M. Laurent, Samuel G. Rodriques, Andrew D. White +2
In many settings where LLM post-training is desirable, high-quality answers are recognizable but difficult to score with explicit rewards. Reinforcement learning with verifiable rewards (RLVR) is limited to tasks where an explicit correctness signal is available. Prior adversarial approaches to training reasoning LLMs either complement verifiable rewards with co-trained critics or require reference reasoning traces. We ask whether a co-trained LLM judge can turn reference answers alone into the sole reward for on-policy post-training of reasoning LLMs. We call this method Reinforced Adversarial Learning (RAL). We first validate RAL on three verifiable tasks (integer multiplication, code-based arithmetic-expression synthesis, and SMILES-to-formula translation), using the verifiers only for evaluation. With sufficiently capable judges and enough training, RAL matches direct RLVR. We then apply RAL to scientific abstract generation in two settings. In the first setting, RAL guides the generator to produce abstracts in the scientific domain of the reference abstracts, even though the prompt leaves that domain unspecified. In the second setting, we generate abstracts for U.S. National…
PDF arXiv
arXiv stat.MLOct 28
Pinpointing Pivot Points: The DECIDE Framework for Data Analysis Decision Importances
Zachary T. Rewolinski, Jan Simson, Frauke Kreuter, Christoph Kern, Bin Yu
Human judgment calls made throughout the data science life cycle can change the conclusions drawn from data. Independent analysts often make different choices when given the same data, leading to often unacceptable variations in results and contributing to the ongoing reproducibility crisis across scientific disciplines. The rapid adoption of agentic tools threatens to amplify this crisis. Several approaches have been developed to address this variation by following the same strategy: repeating an analysis under different combinations of analytical decisions and inspecting the distribution of outcomes. Examples include multiverse analysis, specification curve analysis, and vibration of effects. More broadly, the Predictability-Computability-Stability (PCS) framework assesses how documented judgment calls across the data science life cycle affect conclusions, with predictability serving as a reality-check that a model captures signal. Each of these approaches results in a "multiverse" object. While the distribution of outcomes reflects the overall variability of conclusions, estimating the influence of each individual decision requires additional tools. A common heuristic is to…
PDF arXiv
arXiv cs.ROOct 28
AtomTree: A Hierarchical Framework for State-Aware Embodied Instruction Following with LLMs
Haotian Zhai, Tianming Sha, Junnan Li
Embodied instruction following in interactive environments requires agents to interpret natural language commands and complete long-horizon tasks. Existing planners either rely on large labeled trajectories, or use few-shot and zero-shot LLM prompting that produces a whole plan in a single serial reasoning process. These approaches depend on linguistic common sense, commit to a plan from the initial instruction, and only update after failures, which weakens robustness and state awareness. We present AtomTree, a hierarchical framework for state-aware task planning with LLMs. AtomTree incrementally builds a tree of subgoals and actions, where the LLM decides at each step, from the current symbolic state, whether to further decompose or to output a grounded low-level action. A lightweight memory module summarizes past outcomes for reuse in later decisions. When a low-level action stalls, AtomTree triggers hierarchical backtracking replanning to return to the parent node and regenerate the subtree plan from the latest state, preventing continued drift along an incorrect trajectory. In zero-shot evaluation on the ALFRED benchmark, AtomTree outperforms prior LLM-based planners. Project…
PDF arXiv

Activity · 12 weeks

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting teacher signals degrade joint optimization and cause capability loss when training a single student from multiple teachers.
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable supervision in cross-tokenizer distillation1 paper raises it · 1 addresses it
Vocabulary mismatch and alignment coverage issues make teacher feedback noisy or harmful for student training.
- Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliabilityraises · addresses
Inconsistent policy shifts during teacher training1 paper raises it · 1 addresses it
Endpoint policy shifts may hide early and late preference changes that cancel out, leading to poor transfer.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often include actions unsupported by the agent's visible history, reducing learning effectiveness.
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Credit assignment in long-horizon agent training1 paper raises it · 1 addresses it
Critic values vary across turns, making fine-grained credit allocation and policy updates difficult.
- Multiscale Policy Optimization for Long-Horizon LLM Agentsraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 21m ago

Recursive self improvement

Found by the topic model: recursive self improvement, teacher policy distillation, multi agent debate.

251papers this week−34.5% vs last week

All papers · 2.7k With code · 205

Newest Most relevant

1.7k papers this month