Papers

Field

Top tools here

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting teacher signals degrade joint optimization and cause capability loss when training a single student from multiple teachers.
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable supervision in cross-tokenizer distillation1 paper raises it · 1 addresses it
Vocabulary mismatch and alignment coverage issues make teacher feedback noisy or harmful for student training.
- Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliabilityraises · addresses
Inconsistent policy shifts during teacher training1 paper raises it · 1 addresses it
Endpoint policy shifts may hide early and late preference changes that cancel out, leading to poor transfer.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often include actions unsupported by the agent's visible history, reducing learning effectiveness.
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Credit assignment in long-horizon agent training1 paper raises it · 1 addresses it
Critic values vary across turns, making fine-grained credit allocation and policy updates difficult.
- Multiscale Policy Optimization for Long-Horizon LLM Agentsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

recursive self improvement · ai · agent520 papers40 this week
teacher policy distillation · policy distillation opd · teacher supervision354 papers20 this week
multi agent debate · ai agents · agent debate231 papers20 this week
conversational agents · user simulators · user simulation173 papers19 this week
agent collaboration · multi agent · based multi agent206 papers18 this week
scientific agents · research agents · ai agents196 papers18 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGOct 28
Floor It, Then Fill It: Simulating Creator Ecosystems for Fair Video-Content Recommendation Systems at Billion User Scale
Sohini Roychowdhury, Long Sha, Sunand Agarwal, Payal Kamboj, Shuyang Chen +3
Engagement-optimized content recommendation systems exhibit a well-documented rich-get-richer dynamic that structurally disadvantages emerging creators on large-scale platforms. Evaluating debiasing interventions at scale demands faithful simulation of creator-viewer ecosystems, yet the fidelity, diversity, and validity of such simulators remain underexplored. We present a grounded user-simulation framework that models a heterogeneous population of ${\sim}42$K creators and their coupled viewer interactions over 12~weeks, and use it to evaluate equitable view-allocation policies formulated as a budgeted multi-armed bandit (MAB). The simulator captures three behavioral dimensions: (i)~\textbf{creator diversity}, modeled via a clustering approach grounded in real platform creator-level distributions; (ii)~\textbf{behavioral fidelity}, captured through coupled creator-content state machines that represent growth trajectories, engagement accumulation, and graduation dynamics; and (iii)~\textbf{population-level validity}, measured by calibration against observed platform metrics. We evaluate simulative-configurations that vary content-viewership target definitions, reward functions,…
PDF arXiv
arXiv cs.LGOct 28
MASS: Multi-mask Selective Self-Distillation for Leakage-Resilient Reasoning
Xiuqi Liu, Oveys Delafrooz Noroozi, Upamanyu Madhow
On-policy self-distillation enables the same model to act as student and teacher, with dense token-level supervision obtained by the teacher being given access to privileged information. However, conditioning the teacher on a single reference can concentrate its predictions on one reasoning trajectory and limit transfer to the student. We propose Multi-Mask Selective Self-Distillation (MASS), which approximates the unavailable marginal over diverse reference realizations using multiple independently masked views of one verified solution. MASS aggregates these complementary teacher predictions, selectively transfers reliable token-level corrections, and adaptively focuses masking on influential reference spans. Across mathematical reasoning benchmarks and model families, MASS consistently outperforms strong self-distillation baselines.
PDF arXiv
arXiv cs.AIOct 28
SkillCT: A Multidimensional Benchmark and Execution-Free Baseline for Agent Skill Assessment
Mengyu Sun, Zunlong Zhou, Ziyuan Yang, Yi Zhang
Agent skills package instructions, scripts, and resources for specialized tasks and are increasingly distributed through public Web repositories. However, existing skill benchmarks primarily focus on downstream task execution, insufficiently assessing specification quality, behavioral robustness, semantic alignment, and responsibility boundaries. Their reliance on task-specific tools, environments, and repeated executions limits scalability and pre-deployment assessment. To address these challenges, we introduce SkillCT, a benchmark that broadens skill assessment from downstream utility to a multidimensional characterization of skill quality. SkillCT evaluates skills across nine complementary dimensions spanning three aspects: Static Assessment, which captures intrinsic properties of the skill specification; Dynamic Assessment, which measures whether the induced behavior remains stable across backbone models and request variations; and Human-Assisted Assessment, which examines behavior-chain consistency and task granularity. These dimensions provide a broader view of skill quality than execution success. Applying SkillCT to a public skill corpus establishes a reference benchmark…
PDF arXiv
arXiv cs.AIOct 28
AI-Driven Digital Twin Framework for Warehouse Logistics Automation Optimization
Md Mahinur Alam, Kanita Jerin Tanha, Md Tanjin Alam, Shadman Sakib, Min Gyu Kim
In recent years, warehouse logistics optimization has increasingly focused on intelligent, data-driven systems to enhance operational efficiency. Conventional static product placement strategies are being progressively replaced by dynamic slotting approaches that can adapt to real-time variations in warehouse conditions. In this work, we present a novel framework that employs a reinforcement learning agent integrated within a FlexSim-based Digital Twin environment for real-time product placement optimization. The proposed agent continuously learns from streaming operational data, enabling it to reduce picking time while improving overall system throughput. The simulation environment models a four-zone warehouse layout with forklift-based material handling and leverages the Asset Administration Shell framework to ensure seamless interaction between the learning agent, physical asset, and the digital twin. Experimental evaluation demonstrates substantial performance gains, with the proposed approach achieving a 29.66\% reduction in average picking time, a 35.42\% increase in throughput, and an 18.54\% decrease in travel distance.
PDF arXiv
arXiv cs.CLOct 28
Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI
NVIDIA, :, Aastha Jhunjhunwala, Aayushi Ashokkumar Srivastava, Abhibha Gupta +554
We introduce Nemotron 3.5 Super and Nemotron 3.5 Lightning, open and efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, and agentic workloads. Super is a 120B-parameter, 12B-active multimodal model, while Lightning is a smaller 30B-parameter, 3B-active text model; both designed for efficient deployment. Both models combine large-scale distillation, long-context training, and native speculative decoding, while Super further incorporates Indic continued pretraining and agentic software-engineering mid-training. We develop a comprehensive post-training pipeline spanning supervised fine-tuning and reinforcement learning (RLVR and RLHF), complemented in Super by specialized teacher training and multi- teacher on-policy distillation (MOPD). This transforms Super from a text-only base model into a multimodal agent capable of reasoning over text, images, video, and interactive tool environments. We further explore test-time scaling through parallel agent execution and self-verification. Across training and deployment, we jointly optimize capability and efficiency. Indic continued pretraining improves Indian-language benchmarks…
PDF arXiv
arXiv cs.GTOct 28
What Can Firms Do in the Face of Model Distillation?
Diptangshu Sen, Jibang Wu, Juba Ziani
Model distillation has become a major concern in the rapidly growing generative AI discourse. In its simplest form, distillation is the phenomenon where smaller firms can use data distilled from transcripts of conversations with powerful proprietary closed-weight models to train their own models, thus achieving reasonably high model quality but at much lower costs. Besides the legitimate safety and security concerns associated with distillation, another major issue is how it can also distort marketplace outcomes. This motivates us to initiate an exploration of the problem from an economic theory lens. We consider a setting with two players: an incumbent and a distilling competitor and characterize how the incumbent's model characteristics drive the incentives around how much the competitor should distill. We also consider the impact of distillation on key market outcomes like user-side surplus and the incumbent's utility. Our key contribution, however, is to study interventions for preventing or limiting distillation attacks. In this context, we study two settings: i) when it is possible, with a certain probability (which grows in the extent of distillation conducted), to detect…
PDF arXiv

Papers

Topics in this field

recursive self improvement · ai · agent520 papers40 this week
teacher policy distillation · policy distillation opd · teacher supervision354 papers20 this week
multi agent debate · ai agents · agent debate231 papers20 this week
conversational agents · user simulators · user simulation173 papers19 this week
agent collaboration · multi agent · based multi agent206 papers18 this week
scientific agents · research agents · ai agents196 papers18 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.LGOct 28
Floor It, Then Fill It: Simulating Creator Ecosystems for Fair Video-Content Recommendation Systems at Billion User Scale
Sohini Roychowdhury, Long Sha, Sunand Agarwal, Payal Kamboj, Shuyang Chen +3
Engagement-optimized content recommendation systems exhibit a well-documented rich-get-richer dynamic that structurally disadvantages emerging creators on large-scale platforms. Evaluating debiasing interventions at scale demands faithful simulation of creator-viewer ecosystems, yet the fidelity, diversity, and validity of such simulators remain underexplored. We present a grounded user-simulation framework that models a heterogeneous population of ${\sim}42$K creators and their coupled viewer interactions over 12~weeks, and use it to evaluate equitable view-allocation policies formulated as a budgeted multi-armed bandit (MAB). The simulator captures three behavioral dimensions: (i)~\textbf{creator diversity}, modeled via a clustering approach grounded in real platform creator-level distributions; (ii)~\textbf{behavioral fidelity}, captured through coupled creator-content state machines that represent growth trajectories, engagement accumulation, and graduation dynamics; and (iii)~\textbf{population-level validity}, measured by calibration against observed platform metrics. We evaluate simulative-configurations that vary content-viewership target definitions, reward functions,…
PDF arXiv
arXiv cs.LGOct 28
MASS: Multi-mask Selective Self-Distillation for Leakage-Resilient Reasoning
Xiuqi Liu, Oveys Delafrooz Noroozi, Upamanyu Madhow
On-policy self-distillation enables the same model to act as student and teacher, with dense token-level supervision obtained by the teacher being given access to privileged information. However, conditioning the teacher on a single reference can concentrate its predictions on one reasoning trajectory and limit transfer to the student. We propose Multi-Mask Selective Self-Distillation (MASS), which approximates the unavailable marginal over diverse reference realizations using multiple independently masked views of one verified solution. MASS aggregates these complementary teacher predictions, selectively transfers reliable token-level corrections, and adaptively focuses masking on influential reference spans. Across mathematical reasoning benchmarks and model families, MASS consistently outperforms strong self-distillation baselines.
PDF arXiv
arXiv cs.AIOct 28
SkillCT: A Multidimensional Benchmark and Execution-Free Baseline for Agent Skill Assessment
Mengyu Sun, Zunlong Zhou, Ziyuan Yang, Yi Zhang
Agent skills package instructions, scripts, and resources for specialized tasks and are increasingly distributed through public Web repositories. However, existing skill benchmarks primarily focus on downstream task execution, insufficiently assessing specification quality, behavioral robustness, semantic alignment, and responsibility boundaries. Their reliance on task-specific tools, environments, and repeated executions limits scalability and pre-deployment assessment. To address these challenges, we introduce SkillCT, a benchmark that broadens skill assessment from downstream utility to a multidimensional characterization of skill quality. SkillCT evaluates skills across nine complementary dimensions spanning three aspects: Static Assessment, which captures intrinsic properties of the skill specification; Dynamic Assessment, which measures whether the induced behavior remains stable across backbone models and request variations; and Human-Assisted Assessment, which examines behavior-chain consistency and task granularity. These dimensions provide a broader view of skill quality than execution success. Applying SkillCT to a public skill corpus establishes a reference benchmark…
PDF arXiv
arXiv cs.AIOct 28
AI-Driven Digital Twin Framework for Warehouse Logistics Automation Optimization
Md Mahinur Alam, Kanita Jerin Tanha, Md Tanjin Alam, Shadman Sakib, Min Gyu Kim
In recent years, warehouse logistics optimization has increasingly focused on intelligent, data-driven systems to enhance operational efficiency. Conventional static product placement strategies are being progressively replaced by dynamic slotting approaches that can adapt to real-time variations in warehouse conditions. In this work, we present a novel framework that employs a reinforcement learning agent integrated within a FlexSim-based Digital Twin environment for real-time product placement optimization. The proposed agent continuously learns from streaming operational data, enabling it to reduce picking time while improving overall system throughput. The simulation environment models a four-zone warehouse layout with forklift-based material handling and leverages the Asset Administration Shell framework to ensure seamless interaction between the learning agent, physical asset, and the digital twin. Experimental evaluation demonstrates substantial performance gains, with the proposed approach achieving a 29.66\% reduction in average picking time, a 35.42\% increase in throughput, and an 18.54\% decrease in travel distance.
PDF arXiv
arXiv cs.CLOct 28
Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI
NVIDIA, :, Aastha Jhunjhunwala, Aayushi Ashokkumar Srivastava, Abhibha Gupta +554
We introduce Nemotron 3.5 Super and Nemotron 3.5 Lightning, open and efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, and agentic workloads. Super is a 120B-parameter, 12B-active multimodal model, while Lightning is a smaller 30B-parameter, 3B-active text model; both designed for efficient deployment. Both models combine large-scale distillation, long-context training, and native speculative decoding, while Super further incorporates Indic continued pretraining and agentic software-engineering mid-training. We develop a comprehensive post-training pipeline spanning supervised fine-tuning and reinforcement learning (RLVR and RLHF), complemented in Super by specialized teacher training and multi- teacher on-policy distillation (MOPD). This transforms Super from a text-only base model into a multimodal agent capable of reasoning over text, images, video, and interactive tool environments. We further explore test-time scaling through parallel agent execution and self-verification. Across training and deployment, we jointly optimize capability and efficiency. Indic continued pretraining improves Indian-language benchmarks…
PDF arXiv
arXiv cs.GTOct 28
What Can Firms Do in the Face of Model Distillation?
Diptangshu Sen, Jibang Wu, Juba Ziani
Model distillation has become a major concern in the rapidly growing generative AI discourse. In its simplest form, distillation is the phenomenon where smaller firms can use data distilled from transcripts of conversations with powerful proprietary closed-weight models to train their own models, thus achieving reasonably high model quality but at much lower costs. Besides the legitimate safety and security concerns associated with distillation, another major issue is how it can also distort marketplace outcomes. This motivates us to initiate an exploration of the problem from an economic theory lens. We consider a setting with two players: an incumbent and a distilling competitor and characterize how the incumbent's model characteristics drive the incentives around how much the competitor should distill. We also consider the impact of distillation on key market outcomes like user-side surplus and the incumbent's utility. Our key contribution, however, is to study interventions for preventing or limiting distillation attacks. In this context, we study two settings: i) when it is possible, with a certain probability (which grows in the extent of distillation conducted), to detect…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 20m ago

Recursive self improvement

Found by the topic model: recursive self improvement, teacher policy distillation, multi agent debate.

237papers this week−39.2% vs last week

All papers · 2.7k With code · 206

Newest Most relevant

1.7k papers this month

Open problems in Recursive self improvement

Gradient conflicts in multi-teacher distillation1 paper raises it · 1 addresses it
Conflicting teacher signals degrade joint optimization and cause capability loss when training a single student from multiple teachers.
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
Unreliable supervision in cross-tokenizer distillation1 paper raises it · 1 addresses it
Vocabulary mismatch and alignment coverage issues make teacher feedback noisy or harmful for student training.
- Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliabilityraises · addresses
Inconsistent policy shifts during teacher training1 paper raises it · 1 addresses it
Endpoint policy shifts may hide early and late preference changes that cancel out, leading to poor transfer.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Lack of decision-grounded trajectory supervision1 paper raises it · 1 addresses it
Successful trajectories often include actions unsupported by the agent's visible history, reducing learning effectiveness.
- Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agentsraises · addresses
Credit assignment in long-horizon agent training1 paper raises it · 1 addresses it
Critic values vary across turns, making fine-grained credit allocation and policy updates difficult.
- Multiscale Policy Optimization for Long-Horizon LLM Agentsraises · addresses

Named by an AI from these papers' abstracts.