Research

• Sensing communication16+23.1%
• Federated learning40+14.3%
• Agents484−3.8%
• Frontier LLMs417−11.5%
• Forecasting time75−16.7%
• Learning verifiable rewards164−17.6%
• Inference & serving244−18.9%
• Quantum machine learning23−20.7%

• arXiv cs.LGOct 19

  A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

  Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan, Gu Yannian, Song Mingfei +2

  Consolidating specialized foundation models into a unified generalist is a central goal of post-training. Two approaches are in common use: model merging, which is fast but treated as a heuristic, and multi-teacher on-policy distillation, which matches distributions but demands heavy rollout compute and suffers from cross-task gradient conflict. We show that the two are one objective seen at different orders. Within a shared basin of homologous specialists, the multi-teacher on-policy distillation objective is, to second order, exactly the Fisher-merging objective. Its minimizer is therefore the Fisher merge, a classic model-merging method given by a closed-form combination of the specialists, so merging is optimal, to second order, as a closed-form stage of distillation. Building on this identification, we propose Residual-Governed Branch-and-Merge(RGBM), a two-phase consolidation procedure. Concretely, a full-parameter Fisher merge is intractable on foundation models, so Phase~1 restricts the merge to the task-vector subspace. Extending tangent-space analyses of merging, we show that the same coefficients that mix task vectors also mix implicit rewards, which lets a residual…

  PDF
• arXiv cs.CLOct 19

  Bridged Self-Distillation for On-Policy Learning

  Runsheng Chen, Yihong Liu, Raoyuan Zhao, Michael A. Hedderich, Hinrich Schütze +1

  On-policy distillation can transfer the reasoning capabilities of a strong teacher to a smaller student. However, distillation from a stronger teacher model faces two mismatches: its solutions may be difficult for a smaller student to imitate, and its token-level probabilities may be incompatible with, or even unavailable to, the student. Existing methods handle these settings through different forms of supervision. To this end, we propose Bridged Self-Distillation (BSD), a unified framework for distilling from any teacher that provides reasoning traces. The model to be distilled from, termed the external teacher, first provides a reasoning trace as privileged context, which can be generated in an offline fashion. The student generates rollouts without this context, while a copy of the student conditioned on it, serving as the self-teacher, scores the same rollouts in the student's own token space. BSD uses the resulting token-level guidance to adjust updates on unsuccessful rollouts, with a verifier providing the outcome signal. As the external teacher is queried only to produce text, the same training procedure applies to all teacher scenarios: shared-tokenizer,…

  PDF
• arXiv cs.AIOct 19

  Selective Transfer through Sequence Distillation: From Benchmark Knowledge to Backdoor Behaviour

  Jonibek Mansurov, Sholpan Bolatzhanova, Filbert Aurelian Tjiaranta, Vallerie Alexandara Putra, Alham Fikri Aji

  Knowledge distillation is widely used to transfer capabilities from teacher models to student models, but recent work shows that it can also transfer unintended teacher properties. We study whether different hidden teacher properties survive the same distillation channels. Specifically, we compare two properties: benchmark-associated knowledge and refusal backdoors, and show that distillation treats them very differently. Teachers fine-tuned on benchmark data successfully pass that knowledge to students through all four distillation channels we test, including when students are trained only on teacher-generated number sequences containing no natural language or domain content whatsoever. Refusal backdoors tell the opposite story: backdoored teachers achieve near-perfect triggered refusal, yet sequence-distilled students inherit essentially none of this behaviour. Logit-based distillation transfers it substantially, reaching up to 85.5\% triggered refusal in students that saw zero backdoor examples. Distillation is therefore selective, and the selection criterion is not what data the student sees, but what structure the teacher's property has. The code is available at…

  PDF
• arXiv cs.LGOct 19

  Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation

  Siyuan Gan, Bo Yang, Linjian Meng, Boyan Wang, Xiaolong Qi +2

  Multi-teacher on-policy distillation (MOPD) aggregates the capabilities of multiple domain-expert teachers into a single student model. However, all domains in MOPD share the student's finite budget of optimization steps and rollouts. Fixed uses constant domain sampling ratios based on corpus sizes; by contrast, D$^3$-MOPD adjusts the ratios using the teacher-student reverse-KL divergence and its recent decrease. However, favoring large recent decreases can direct more training toward domains that already improve quickly, while domains that improve more slowly may receive too few examples. To address inflexible allocation and the risk of giving insufficient training to domains that improve more slowly, we propose Nash-MOPD, an adaptive domain scheduler based on a local minimax game using existing reverse-KL statistics. The equilibrium maximizes the smallest predicted proportional reduction under the local model and gives more prompts to domains that need more examples for comparable improvement. Consequently, Nash-MOPD responds to changing learning progress and supports domains that improve more slowly without extra teacher queries or rollouts. At step 100, Nash-MOPD exceeds both…

  PDF
• arXiv cs.CVOct 19

  After the Federation: Benchmarking Knowledge Portability to Heterogeneous Newcomers

  Xinyuan Zhao

  Once historical clients become unavailable, a federation must serve newcomers through the artifact it leaves behind. How much of its knowledge remains useful in an architecture chosen afterward? We introduce FEDNBO, a seven-dataset benchmark that separates knowledge supply, policy identification, and private-model compilation under disjoint data roles. Controlled studies expose inflated oracle gains, architecture-dependent transfer reversals, and a gap between teacher accuracy and private-model utility. Ordinary public-response distillation provides a competitive reference without public task labels or newcomer samples: gains over strengthened local training are 23.52 and 16.62 percentage points on 300 CelebA identities and 300 FedEMNIST writers, respectively. In a complete-pipeline FedEMNIST comparison, this reference reaches 97.13% accuracy with 12.4 seconds of compilation per architecture, versus 97.03% and 67.5 seconds for a HeteroFedTTA reconstruction at the same epoch budget. Equal-count public-input interventions extend support sensitivity to trainable visual backbones. Together, these results establish an auditable benchmark and a strong reference for evaluating the…

  PDF
• arXiv cs.LGOct 19

  Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization

  Suwan Yoon, Seongryong Jung, Changhee Lee

  Self-distillation policy optimization provides dense token-level supervision for LLM post-training through feedback-conditioned self-teachers. However, directly matching the self-teacher can disrupt successful behaviors on correct rollouts while providing insufficient correction on incorrect ones. We propose Asymmetric Policy Optimization via Self-Distillation (APSD), which constructs correctness-dependent targets by mixing student and self-teacher distributions: interpolation preserves successful behaviors on correct rollouts, while extrapolation beyond the self-teacher strengthens correction on incorrect ones. We analyze APSD through divergence decomposition, reverse-KL barycentric geometry, and gradient analysis, highlighting the importance of gradient flow through student-dependent targets. Experiments under reinforcement learning with verifiable rewards and rich feedback show that APSD achieves strong overall performance against reinforcement-learning and self-distillation baselines and matches SDPO performance with 2.6x fewer training steps on average across Qwen3 model scales.

  PDF
• arXiv cs.LGOct 19

  Asynchronous On-Policy Self-Distillation

  Xidong Wu, Qi Qi, Yuqiong Ji, Shangqian Gao

  Unlike on-policy distillation (OPD), which depends on a separate teacher model, on-policy self-distillation (OPSD) leverages a single language model acting simultaneously as the student and the teacher, with the teacher conditioned on privileged information. Despite this architectural simplicity, synchronous rollout generation dominates overall training time, creating a severe systems bottleneck when scaling OPSD. To address this, we introduce Asynchronous On-Policy Self-Distillation (AsyncOPSD), a framework that decouples rollout generation from learner updates to improve training throughput. Since the teacher and student share parameters, AsyncOPSD eliminates the multi-model coordination overhead inherent to OPD architectures in asynchronous training. As the student policy evolves, buffered trajectories generated by older policies become stale, potentially degrading performance. To mitigate this, we introduce a distillation objective based on staleness-aware policy control and token-level entropy weighting. Across three model sizes and mathematical reasoning benchmarks, AsyncOPSD improves the training throughput by $1.96$--$2.25\times$ over synchronous OPSD while increasing…

  PDF
• arXiv cs.AIOct 19

  Not Every Rollout Needs Distillation: Correctness-Filtered On-Policy Self-Distillation

  Harrison Zhao, Lei Huang, Fei Liu, Wen Shi, Zijian Jin

  On-policy self-distillation (OPSD) supervises student-generated trajectories with a reference-conditioned version of the same base model. We ask whether every trajectory supplies distinct supervision that warrants its distillation cost. Gradient measurements on Qwen3-1.7B at initialization reveal that correct and incorrect responses have similar individual gradient magnitudes, but incorrect responses have approximately three times the coherent gradient share. Their estimated population-mean directions also largely overlap. A signal--noise model fitted at a small group size tracks alignment at larger group sizes without refitting. Motivated by these measurements, we introduce correctness-filtered OPSD (C-OPSD), which generates one response per problem and distills only a subset of incorrect responses. Removing unselected rows before both distillation forwards avoids their teacher/student forward and student backward work. Across Qwen3-1.7B, 4B, and 8B on AIME24, AIME25, and HMMT25, C-OPSD achieves on-par performance with full OPSD. In a measured micro-batch-size-four implementation, executed rows decrease by approximately 68%, and with retained rows packed into full micro-batches,…

  PDF
• arXiv cs.AIOct 19

  ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents

  Chu Zhao, Lei Tang, Yanming Wang, Jianzhe Zhao, Guibing Guo +3

  On-policy distillation (OPD) enhances the capabilities of small language agents by providing dense, token-level supervision from a teacher model on student-generated trajectories. However, standard OPD implicitly assumes that credit is uniformly distributed across long-horizon trajectories, assigning the same strength of teacher supervision to different actions. Through credit-distribution analysis and outcome-reversal counterfactual experiments, we find that trajectory credit is concentrated on a small number of critical actions. Moreover, even when the trajectory is held fixed, reversing only the final outcome substantially changes the credit assigned to multiple interaction turns. These findings show that the uniform-credit assumption fails to identify critical decisions and ultimately degrades the model's reasoning accuracy. To address these challenges, we propose ToolOPD, an outcome-aware on-policy distillation framework designed for long-horizon tool use. ToolOPD first summarizes each completed trajectory and its outcome into a reusable hindsight skill. It then measures how this skill changes the teacher's probability of each student action token and converts the change…

  PDF
• arXiv cs.LGOct 19

  OPDArena: Understanding Learning and Forgetting in On-Policy Distillation

  Yuanze Xu, Tianzhu Ye, Hui Sun, Wenhui Wang, Li Dong +1

  On-policy distillation (OPD) is increasingly used for language model post-training, yet how teacher supervision shapes learning and forgetting remains unclear. We present OPDArena, a framework comparing supervised fine-tuning (SFT), reinforcement learning with verifiable rewards (RLVR), and OPD across five pretrained models and diverse teacher sources. It jointly evaluates mathematical learning, general capability retention, and predictive distribution changes. Teacher choice strongly shapes retention: students of RLVR-trained teachers improve on the target task while retaining prior capabilities, whereas students of SFT-trained teachers incur greater capability loss. Across post-training methods, out-of-domain (OOD) distribution change shows a more consistent positive association with forgetting than in-domain change. To understand how teacher choice affects these changes, we examine the additional information teachers provide relative to the student's pretrained model. Controlled teacher comparisons show that the task-specific preferences teachers acquire through post-training make student updates more concentrated in-domain.

  PDF
• arXiv cs.CLOct 19

  MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models

  Zhengao Li, Chuxu Zhang, Yushun Dong

  Knowledge acquisition from black-box large language models (LLMs) offers a practical approach to adapting smaller student models. However, many existing methods rely on predefined queries or student feedback to guide acquisition, leaving autonomous expansion of knowledge targets an important challenge. We propose \textbf{MapSteal}, a student-independent framework for structured recursive knowledge acquisition. Starting from a small set of initial roots, MapSteal represents acquisition targets using structured states and recursively expands them through teacher-proposed directions. Question realization, proposal admission, and deduplication constrain the acquisition process, while root-aware scheduling organizes exploration under a fixed QA budget. The resulting question--answer pairs are used to adapt student models without requiring student feedback during acquisition. Experiments with Qwen3.5 students at 2B, 4B, and 9B scales show competitive downstream performance across six medical and cybersecurity benchmarks. On the 4B medical evaluation, Full MapSteal improves average accuracy by 1.17 percentage points over a simplified recursive acquisition variant. Cross-domain…

  PDF
• arXiv cs.CLOct 18

  Self-Distillation for Supervised Fine-Tuning

  Wenxuan Zhu, Bang An, Shuming Liu, Wenqing Cui, Tianqi Xu +4

  Supervised fine-tuning treats the expert demonstration or gold response as a one-hot target, suppressing valid alternative continuations that the model would otherwise assign probability to. We propose Gold-Conditioned Distillation (GCD), a method that turns expert demonstrations into model-native supervision. GCD uses one model as both teacher and student: the teacher conditions on a prompt containing the gold response in context, the student conditions on the original prompt as at inference, and the student is trained to minimize the forward KL divergence to the teacher's next-token distribution at each gold response token. This lets the model use its prior knowledge to teach itself the new task: correctness is anchored by the gold response, while the soft target preserves plausible alternatives. Across various training tasks on Qwen and Llama models, GCD improves new-task accuracy over SFT and generally reduces degradation on general benchmarks.

  PDF
• arXiv cs.LGOct 18

  CD-OPD: Contextual Digest for On-Policy Distillation

  Mingyuan Li, Yucheng Li, Xiaxia Wang, Gong Cheng

  Teacher takeover in on-policy distillation (OPD) can produce incorrect responses even when the student and teacher independently answer the same problem correctly, a failure we quantify using Collaboration Failure Rate (CFR). To address it, we propose Contextual Digest for On-Policy Distillation (CD-OPD). CD-OPD structures the earlier context of teacher guidance through a teacher-generated dependency graph, curating the required problem conditions and preceding reasoning as node summaries alongside the unchanged guidance in a Contextual Digest before student continuation. We further develop online meta-reweighting, which compares Digest segment gradients with a task gradient from independent student responses to improve student-only performance, favoring updates predicted to reduce task loss. Across eight benchmarks, CD-OPD improves average accuracy by 1.32--7.05 percentage points over representative existing OPD methods. It also reduces CFR by 27.4\%--56.4\% relative to existing takeover-based OPD methods.

  PDF
• arXiv cs.LGOct 18

  Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation

  Jike Zhong, Ming Li, Yuxiang Lai, Xi Xiao, Sheng Zhang +7

  On-policy distillation (OPD) trains agents on their own trajectories using supervision from a stronger teacher. We uncover a counterintuitive phenomenon in multi-turn agentic tasks: a smaller, initially weaker student can substantially outperform its teacher during training, only to regress as distillation continues. We term this phenomenon distillation overshoot. Through decision- and task-level analyses, we trace overshoot to two competing effects of OPD: repairing weak student decisions and eroding successful student choices that the teacher disfavors. Early in training, repair outpaces erosion, allowing the student to combine the teacher's corrections with its own strengths and surpass the teacher. As training continues, erosion overtakes repair, causing these gains to vanish. Motivated by this finding, we propose Success-Preserving On-Policy Distillation (SPOD), which selectively masks conflicting teacher supervision on successful student choices and introduces a soft probability floor to counteract their erosion, while retaining teacher guidance elsewhere. SPOD requires only binary episode success and model probabilities already available during OPD. Across two model…

  PDF
• arXiv cs.CLOct 16

  SLA-LENS: From Classroom Transcripts to CSLA-Informed and Evidence-Linked L2 Interaction Analytics

  Nemika Tyagi, Olga Kellert, Phanos Kareen, Nelvin Licona-Guevara

  What happens after a learner makes an error, and how does the instructor's response shape the next turn? Investigating these exchanges across classrooms requires context-sensitive annotation that is difficult to scale manually. We introduce SLA-LENS, an end-to-end LLM-assisted framework that connects classroom transcripts to structured pedagogical annotations and evidence-linked descriptive analytics. Grounded in interactionist Second Language Acquisition (SLA) theory, its annotation schema captures second language learner errors, pedagogical functions, and uptake or repair while accounting for speaker roles, conversational context, and hierarchical labels. We apply the framework to 40 authentic online L2 Spanish classroom transcripts comprising over 21,000 sentence-level instances. We evaluate proprietary and open-weight LLMs, including task-specific QLoRA adaptation, using expert-reviewed gold annotations, held-out silver data, and metrics sensitive to different types of annotation error. A deterministic reporting layer generates reproducible visualizations of participation, feedback, learner errors, uptake, and temporal interaction patterns, linked to supporting transcript…

  PDF
• arXiv cs.AIOct 16

  Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning

  Anagha Aneesh, Sebastian Ibarraran, Frank Hu, Jason Liu, Grant Rotskoff

  Automating synthesis planning for small-molecule drug design requires chemical reasoners that are accurate as well as open to inspection and adaptation, yet frontier language models remain unreliable at predicting reaction outcomes and difficult to deploy, while small open-weight models lack the necessary domain reasoning. We show that frontier supervision can serve as a one-time bootstrap for open models on forward reaction prediction. Our models, Resonance, distill an open-weight teacher's reasoning traces and then learn from their own verified outputs, turning a small base model with near-zero accuracy into a capable reaction predictor, while a stronger student surpasses its teacher through distillation alone. Accuracy, however, does not bring robust reasoning: the small student inherits its teacher's single-pass style, self-training mainly sharpens answers it could already reach, and it readily abandons correct answers under pushback, suggesting that chemical reasoning models should be evaluated on the robustness of their reasoning as well as on the accuracy of their answers.

  PDF
• arXiv cs.CLOct 16

  Beyond the Answer: Student-Conditioned Trace Construction for Reasoning Distillation

  Shaoang Li, Yanhang Shi, Yufei Li, Mingfu Liang, Frank Shyu +4

  A reasoning trace can accompany a correct answer without providing effective supervision for a particular student. We study how to construct traces that improve learning beyond answer-only supervision by considering both their intermediate content and their accessibility to the target learner. We introduce ReDiST, a student-conditioned trace-construction and distillation framework for evidence-rich reasoning. Starting from the student's own attempts, ReDiST revises answer-correct drafts against the supplied evidence and repairs unsuccessful drafts with additional guidance from the verified answer. A frozen student-matched reader screens and ranks original and edited candidates using answer recovery and verified-answer probability, given the question, evidence, and candidate trace. We distill the selected targets while balancing trace and final-answer supervision with an auxiliary answer loss. Construction and selection occur offline, and inference uses only the distilled student. Experiments on question-answering benchmarks show that ReDiST can improve both source-task and cross-benchmark performance over answer-only, original-trace, and the evaluated trace-selection and…

  PDF
• arXiv cs.CLOct 16

  EduWild: How Teacher-Configured Chatbots Fail in Real-World Use and How to Fix Them

  Ziqi Liu, Ruiwei Xiao, Yuxuan Li, John Stamper

  Teachers increasingly configure educational chatbots through natural-language instructions, but how these bots fail in real-world use remains poorly understood. We introduce EduWild, a dataset of 1,726 conversations with 217 teacher-configured chatbots, each paired with the instructions and configuration behind it, including 900 conversations from the intended end users of 35 deployed chatbots. We develop a 29-code taxonomy covering factual, workflow, diagnostic, and pedagogical failures and apply it using a human-calibrated LLM judge. The most frequent failures are lapses in following teacher intent --- bots skip required workflow steps (53.8% of end-user conversations), omit expectation setup (33.4%), and overload learners with information or questions (37.1%). Guided by these findings, we design a prompt-based harness targeting the six most frequent failure modes. In a study with 200 college students across ten bots, the harness reduces the mean number of target failure codes per conversation from 1.406 to 0.489, a 65.2% relative reduction with bots weighted equally. These findings show how real-world failure analysis can guide practical improvements to educational chatbots.

  PDF
• arXiv cs.AIOct 16

  SimOPD: Relational Knowledge Transfer for On-Policy Distillation

  Kai Tang, Xing Zheng, Haobo Wang, Guangcheng Zhu, Shenzhi Yang +4

  A key challenge in on-policy distillation (OPD) is extracting effective supervision from teacher knowledge. Supervision from token probabilities may underutilize internal knowledge, while direct hidden-state matching forces the student to shift to the teacher's feature space and complicates cross-architecture transfer. Inspired by contrastive learning's use of relationships between instances, we propose \textbf{SimOPD} to transfer teacher internal knowledge through response similarities. Specifically, SimOPD uses paired memory banks of student-generated responses to extract supervision from the teacher's internal representations as similarity distributions. The student then matches these distributions within its own representation space, retaining representational freedom and removing the need for cross-model projection. Extensive experiments show that SimOPD improves the overall average over the strongest baselines by up to 2.6 and 3.1 percentage points in same- and cross-architecture settings.

  PDF
• arXiv cs.CLOct 16

  MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation

  Yuhan Sun, Jiaxin Zhao, Yongdong Luo, Yibo Hu, Meiguang Jin +4

  In a hierarchical multi-agent system, a Leader assigns tasks and subagents carry them out. Jointly training the Leader and the subagents is difficult because a final answer gives only indirect feedback to the subagents. An LLM judge can score the performance of the subagents, but these scores depend on the judge's own assessment of the quality of subagents' responses. We present MultiAgent-OPD, which combines on-policy distillation with reinforcement learning to jointly train a shared-parameter Leader-Searcher-Answerer team end to end. We first train a Teacher in a Searcher-Answerer workflow using answer correctness. During team training, the frozen Teacher scores student-generated search and summary tokens to provide local process supervision for the Searchers, while the Leader learns from team success and the Answerer from candidate correctness. Across seven datasets, the trained system reaches 38.68% macro avg@4, exceeding Prompted Critic by 4.06 percentage points and C-MAS + RL, the trained cascaded multiagent system, by 2.61 percentage points. The team also solves more questions within four attempts.

  PDF

• Teacher recoverability on incorrect student rollouts1 paper raises it · 1 addresses it

  Teachers often follow student errors instead of correcting them, degrading supervision quality in on-policy distillation.

  • Recovering Reliable Supervision via Critique-Guided On-Policy Distillationraises · addresses
• Gradient conflicts in multi-teacher on-policy distillation1 paper raises it · 2 address it

  Divergent teacher styles and expertise cause conflicting gradients, harming joint optimization and student performance.

  • Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationraises · addresses
  • Latent-MOPD: Latent Multi-Teacher On-Policy Distillationaddresses
• Policy shift opacity in teacher distillation1 paper raises it · 1 addresses it

  Endpoint policy shifts hide whether local action preferences evolved consistently during teacher training, limiting selective transfer.

  • Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
• Cross-tokenizer supervision reliability1 paper raises it · 1 addresses it

  Vocabulary mismatch and alignment coverage issues reduce the effectiveness of token-level supervision in on-policy distillation.

  • Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliabilityraises · addresses
• Collapse in iterative agent self-distillation1 paper raises it · 1 addresses it

  Successive distillation cycles degrade deployment performance and privileged-information-conditioned behavior, hindering recursive self-improvement.

  • ReSAIL: Mitigating Collapse in Iterative Agent Self-Distillationraises · addresses

Named by an AI from these papers' abstracts.