Papers

Field

Top tools here

Coming with more sources: stars history

All tools

Transformers3 releases this week
faster-whisper+39% downloads · 3.7M/wk+38.6%
Whisper+38% downloads · 1.7M/wk+38.5%
Tokenizers+14% downloads · 46.6M/wk+14.2%
DeepSpeed+23% downloads · 319.1k/wk+23.1%

Top tools here

Open problems in Training & fine-tuning

Teacher recoverability in on-policy distillation1 paper raises it · 1 addresses it
Teachers often fail to correct student errors in rollouts, limiting the effectiveness of on-policy distillation.
- Recovering Reliable Supervision via Critique-Guided On-Policy Distillationraises · addresses
Policy shift consistency during RL transfer1 paper raises it · 1 addresses it
Endpoint policy shifts may hide inconsistent local action preferences, reducing the reliability of distillation.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Memory consolidation without performance loss1 paper raises it · 1 addresses it
Gains from context-based improvements are often lost after context removal, limiting long-term retention.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Automated data curation for supervised fine-tuning1 paper raises it · 1 addresses it
Existing systems lack the full range of data engineering actions, limiting their ability to optimize training data.
- AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuningraises · addresses
Cross-capability gradient conflicts in multimodal RL1 paper raises it · 1 addresses it
Joint optimization of heterogeneous objectives in VLMs leads to interference, degrading specialized skills.
- Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Modelsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.AIOct 15
Learning from Unverified Peers via Shared Residual Distillation
Feiyang Chen, Runze Ma, Zichen Wang, Jiong Lou, Zihan Wang +2
A language model can generate several solutions to the same problem, some leading to correct answers and others to incorrect ones. Many post-training methods turn these trajectories into learning signals using supervision from reference answers, verifiable feedback, or external teachers. Without such supervision, learning from the model's own generations becomes less straightforward, as their correctness cannot be reliably assessed. An incorrect solution may nevertheless contain useful intermediate reasoning, so its value as supervision need not be determined solely by its final answer. Can a model improve by learning from its unverified peer solutions without first determining which are correct? We propose Shared Residual Distillation (SRD), an on-policy self-distillation method that learns from the model's own unverified peer trajectories. SRD compares how peers change the prediction distribution relative to a control, retaining adjustments that agree in direction and are conservative in magnitude. This yields supervision without treating any complete solution as correct, and the trained model requires no peers at inference time. Starting from the corresponding base models, SRD…
PDF arXiv
arXiv cs.CVOct 15
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Madhav Iyengar, Andreas Geiger, Hilde Kühne
Recent work shows that depth recurrence can be retrofitted into pretrained language models. In this work, we adapt this paradigm for visual understanding, using depth recurrence to reduce the parameter footprint of a pretrained vision-language model (VLM). To this end, we prune layers from the language backbone of the VLM and loop over a subset of the remaining layers. At each step of the loop, a linear adapter merges the block's output with the token representations from the preceding layers and projects the fused representation back into the block. As the pruning impacts the original model performance, we propose a three-stage recovery training: first, we leverage a language-only pretraining to recover the performance of the language backbone, followed by an alignment of the vision-language projector, and a full fine-tuning of the model via visual instruction tuning. In the first and the last stage, we distill the outputs of the pretrained non-recurrent model into the recurrent model, which further improves the recovery. We evaluate the proposed approach on the TinyLLaVA-1.4B backbone without introducing any new vision-language data during retrofitting. Our experiments…
PDF arXiv
arXiv cs.LGOct 15
ABNAT: Attention-Based Noise-Aware Training for Robust Transformers on Analog Compute-in-Memory Systems
Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Zhiyuan Ning, Yizhe Chen +3
Transformer-based models have become the backbone of a wide range of tasks, but their intensive computation and data movement incur substantial energy overhead. Analog compute-in-memory (CIM) offers an energy-efficient alternative, yet its inherent hardware non-idealities perturb stored weights and can significantly degrade model performance. Although existing noise-aware training (NAT) methods improve robustness by injecting noise during training, they largely overlook the critical role of attention mechanisms in Transformer inference.To bridge this gap, we analyze how CIM-induced noise reshapes attention in Transformers and identify two key observations: (1) the divergence between clean and noisy attention maps increases substantially in deeper layers, and (2) conventional NAT can remain overconfident even when its attention routes deviate markedly from the clean reference. Motivated by these findings, we propose ABNAT, an attention-based noise-aware training framework that teaches the noisy student to preserve the clean teacher's headwise attention routes rather than merely reproduce answers, without hardware changes or inference overhead; selective variants further reduce…
PDF arXiv
arXiv cs.DCOct 15
ETH: Data-Balanced Heterogeneous Pipeline Parallelism for Training Large Models
Mianjie Yu, Zhongmin Zhao, Zizhao Mo, Jianxiong Liao, Wenxuan Li +7
Current heterogeneity-aware training systems for large models balance workloads by assigning unequal numbers of layers across pipeline stages, a strategy aimed at equalizing execution time through a unique pipeline configuration. However, this approach fails to address critical memory constraints and suffers from inherently coarse balancing granularity. In this paper, we propose Heterogeneous Pipeline Parallelism (HPP), a novel approach enabling efficient and fine-grained data balance across heterogeneous pipelines. To address HPP's key challenges, prohibitive search overhead for optimal configurations and high communication overhead from asymmetric All-Reduce, we design the ETH training system. ETH employs a hierarchical workload balance framework that jointly optimizes balance across heterogeneous pipelines and within individual pipeline stages. To mitigate asymmetric communication costs, ETH implements block-wise gradient aggregation. Furthermore, it integrates specialized optimizations that dramatically reduce HPP configuration search overhead. Evaluations demonstrate ETH achieves up to 2x training speedup while reducing configuration generation overhead by up to 56x compared…
PDF arXiv
arXiv cs.LGOct 15▲ 3
Selection Is Not Learning: Choosing Model Futures
Haoyi Yang, Kshitij Mishra, Nils Lukas, Salem Lahlou
Self-evolving language models train on tasks they generate, so every round must decide which updated model the next round inherits. Current loops such as R-Zero, Agent0, and Skill Self-Play make this decision through the task, admitting tasks that look learnable and inheriting every resulting update. We identify a selection-to-update gap in these loops, where a task that looks learnable need not produce a useful update. Most updates on verified tasks fail to make correct answers more likely, yet these loops inherit them all. HeirCommit freezes the updated model and uses fresh evaluation questions to decide whether to inherit it or retain the current model. EvoHeir separates parameter inheritance from experience inheritance, using current and archived experience to repair failures and preserve successes before final confirmation. Our analysis characterizes selection-induced optimism and decomposes regret into selection, absorption, and confirmation errors. HeirCommit cuts the harmful share of accepted updates by 68-83% relative to random inheritance at the same acceptance rate. Across four backbones, EvoHeir achieves the best mathematics and general-reasoning averages, raising the…
PDF arXiv
arXiv cs.CLOct 15
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe
Soumye Singhal, Joseph Jennings, Ka Shun Shum, Alexander Bukharin, Khushi Bhardwaj +9
Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been published about how this stage is built. We present an open midtraining recipe that primes base models for agentic post-training, and we show its effectiveness on two Nemotron models of different sizes. Pretraining typically runs at short context and sees almost no agent interactions, while SFT data is mostly synthetic, and very long SFT stages can hurt the model's later RL trainability. Our midtraining stage sits between the two. It trains at 256K context for hundreds of billions of tokens on a diverse mixture of natural software-engineering data like packed repositories, pull requests, commit histories, and build logs together with a small share of agentic traces. On Nemotron 3.5 Lightning, 500B midtraining tokens improve SWE-bench Verified from 60.7 to 64.5, SWE-bench Multilingual from 55.8 to 62.8, and SWE-bench Pro from 32.0 to 35.5 after identical full SFT. On the larger Nemotron 3.5 Super, 280B midtraining tokens improve DeepSWE, a harder benchmark of long-horizon tasks, from 11.7 to 15.7 and SWE-bench Verified from 72.9 to…
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

Transformers3 releases this week
faster-whisper+39% downloads · 3.7M/wk+38.6%
Whisper+38% downloads · 1.7M/wk+38.5%
Tokenizers+14% downloads · 46.6M/wk+14.2%
DeepSpeed+23% downloads · 319.1k/wk+23.1%

Recent papers

All in Research →

arXiv cs.AIOct 15
Learning from Unverified Peers via Shared Residual Distillation
Feiyang Chen, Runze Ma, Zichen Wang, Jiong Lou, Zihan Wang +2
A language model can generate several solutions to the same problem, some leading to correct answers and others to incorrect ones. Many post-training methods turn these trajectories into learning signals using supervision from reference answers, verifiable feedback, or external teachers. Without such supervision, learning from the model's own generations becomes less straightforward, as their correctness cannot be reliably assessed. An incorrect solution may nevertheless contain useful intermediate reasoning, so its value as supervision need not be determined solely by its final answer. Can a model improve by learning from its unverified peer solutions without first determining which are correct? We propose Shared Residual Distillation (SRD), an on-policy self-distillation method that learns from the model's own unverified peer trajectories. SRD compares how peers change the prediction distribution relative to a control, retaining adjustments that agree in direction and are conservative in magnitude. This yields supervision without treating any complete solution as correct, and the trained model requires no peers at inference time. Starting from the corresponding base models, SRD…
PDF arXiv
arXiv cs.CVOct 15
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Madhav Iyengar, Andreas Geiger, Hilde Kühne
Recent work shows that depth recurrence can be retrofitted into pretrained language models. In this work, we adapt this paradigm for visual understanding, using depth recurrence to reduce the parameter footprint of a pretrained vision-language model (VLM). To this end, we prune layers from the language backbone of the VLM and loop over a subset of the remaining layers. At each step of the loop, a linear adapter merges the block's output with the token representations from the preceding layers and projects the fused representation back into the block. As the pruning impacts the original model performance, we propose a three-stage recovery training: first, we leverage a language-only pretraining to recover the performance of the language backbone, followed by an alignment of the vision-language projector, and a full fine-tuning of the model via visual instruction tuning. In the first and the last stage, we distill the outputs of the pretrained non-recurrent model into the recurrent model, which further improves the recovery. We evaluate the proposed approach on the TinyLLaVA-1.4B backbone without introducing any new vision-language data during retrofitting. Our experiments…
PDF arXiv
arXiv cs.LGOct 15
ABNAT: Attention-Based Noise-Aware Training for Robust Transformers on Analog Compute-in-Memory Systems
Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Zhiyuan Ning, Yizhe Chen +3
Transformer-based models have become the backbone of a wide range of tasks, but their intensive computation and data movement incur substantial energy overhead. Analog compute-in-memory (CIM) offers an energy-efficient alternative, yet its inherent hardware non-idealities perturb stored weights and can significantly degrade model performance. Although existing noise-aware training (NAT) methods improve robustness by injecting noise during training, they largely overlook the critical role of attention mechanisms in Transformer inference.To bridge this gap, we analyze how CIM-induced noise reshapes attention in Transformers and identify two key observations: (1) the divergence between clean and noisy attention maps increases substantially in deeper layers, and (2) conventional NAT can remain overconfident even when its attention routes deviate markedly from the clean reference. Motivated by these findings, we propose ABNAT, an attention-based noise-aware training framework that teaches the noisy student to preserve the clean teacher's headwise attention routes rather than merely reproduce answers, without hardware changes or inference overhead; selective variants further reduce…
PDF arXiv
arXiv cs.DCOct 15
ETH: Data-Balanced Heterogeneous Pipeline Parallelism for Training Large Models
Mianjie Yu, Zhongmin Zhao, Zizhao Mo, Jianxiong Liao, Wenxuan Li +7
Current heterogeneity-aware training systems for large models balance workloads by assigning unequal numbers of layers across pipeline stages, a strategy aimed at equalizing execution time through a unique pipeline configuration. However, this approach fails to address critical memory constraints and suffers from inherently coarse balancing granularity. In this paper, we propose Heterogeneous Pipeline Parallelism (HPP), a novel approach enabling efficient and fine-grained data balance across heterogeneous pipelines. To address HPP's key challenges, prohibitive search overhead for optimal configurations and high communication overhead from asymmetric All-Reduce, we design the ETH training system. ETH employs a hierarchical workload balance framework that jointly optimizes balance across heterogeneous pipelines and within individual pipeline stages. To mitigate asymmetric communication costs, ETH implements block-wise gradient aggregation. Furthermore, it integrates specialized optimizations that dramatically reduce HPP configuration search overhead. Evaluations demonstrate ETH achieves up to 2x training speedup while reducing configuration generation overhead by up to 56x compared…
PDF arXiv
arXiv cs.LGOct 15▲ 3
Selection Is Not Learning: Choosing Model Futures
Haoyi Yang, Kshitij Mishra, Nils Lukas, Salem Lahlou
Self-evolving language models train on tasks they generate, so every round must decide which updated model the next round inherits. Current loops such as R-Zero, Agent0, and Skill Self-Play make this decision through the task, admitting tasks that look learnable and inheriting every resulting update. We identify a selection-to-update gap in these loops, where a task that looks learnable need not produce a useful update. Most updates on verified tasks fail to make correct answers more likely, yet these loops inherit them all. HeirCommit freezes the updated model and uses fresh evaluation questions to decide whether to inherit it or retain the current model. EvoHeir separates parameter inheritance from experience inheritance, using current and archived experience to repair failures and preserve successes before final confirmation. Our analysis characterizes selection-induced optimism and decomposes regret into selection, absorption, and confirmation errors. HeirCommit cuts the harmful share of accepted updates by 68-83% relative to random inheritance at the same acceptance rate. Across four backbones, EvoHeir achieves the best mathematics and general-reasoning averages, raising the…
PDF arXiv
arXiv cs.CLOct 15
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe
Soumye Singhal, Joseph Jennings, Ka Shun Shum, Alexander Bukharin, Khushi Bhardwaj +9
Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been published about how this stage is built. We present an open midtraining recipe that primes base models for agentic post-training, and we show its effectiveness on two Nemotron models of different sizes. Pretraining typically runs at short context and sees almost no agent interactions, while SFT data is mostly synthetic, and very long SFT stages can hurt the model's later RL trainability. Our midtraining stage sits between the two. It trains at 256K context for hundreds of billions of tokens on a diverse mixture of natural software-engineering data like packed repositories, pull requests, commit histories, and build logs together with a small share of agentic traces. On Nemotron 3.5 Lightning, 500B midtraining tokens improve SWE-bench Verified from 60.7 to 64.5, SWE-bench Multilingual from 55.8 to 62.8, and SWE-bench Pro from 32.0 to 35.5 after identical full SFT. On the larger Nemotron 3.5 Super, 280B midtraining tokens improve DeepSWE, a harder benchmark of long-horizon tasks, from 11.7 to 15.7 and SWE-bench Verified from 72.9 to…
PDF arXiv

Active contributors · 90 days

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 51m ago

Training & fine-tuning

How models are trained and adapted: optimizers, fine-tuning, RLHF/RL, data curation, PEFT and scaling recipes.

64papers this week

All papers · 64 With code · 9

Newest Most relevant

64 papers this month

Open problems in Training & fine-tuning

Teacher recoverability in on-policy distillation1 paper raises it · 1 addresses it
Teachers often fail to correct student errors in rollouts, limiting the effectiveness of on-policy distillation.
- Recovering Reliable Supervision via Critique-Guided On-Policy Distillationraises · addresses
Policy shift consistency during RL transfer1 paper raises it · 1 addresses it
Endpoint policy shifts may hide inconsistent local action preferences, reducing the reliability of distillation.
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillationraises · addresses
Memory consolidation without performance loss1 paper raises it · 1 addresses it
Gains from context-based improvements are often lost after context removal, limiting long-term retention.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Automated data curation for supervised fine-tuning1 paper raises it · 1 addresses it
Existing systems lack the full range of data engineering actions, limiting their ability to optimize training data.
- AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuningraises · addresses
Cross-capability gradient conflicts in multimodal RL1 paper raises it · 1 addresses it
Joint optimization of heterogeneous objectives in VLMs leads to interference, degrading specialized skills.
- Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Modelsraises · addresses

Named by an AI from these papers' abstracts.