Papers

Field

Top tools here

All tools

NVIDIA Dynamo+81% downloads · 16.5k/wk+81.4%
BentoML+32% downloads · 42.8k/wk+31.7%
MLX+27% downloads · 357.1k/wk+27.3%
vLLM1 release this week
ort+5% downloads · 922.2k/wk+5.2%

Top tools here

Open problems in Inference & serving

Efficient long-horizon context management4 papers raise it · 4 address it
Long agent trajectories exceed context windows, degrading performance without compact, reusable representations.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cache Engineering for Agentsraises · addresses
- MemT-1.0: Towards Memory-Native Evolutionraises · addresses
Token-level routing system efficiency1 paper raises it · 1 addresses it
Fine-grained routing improves cost-quality trade-offs but causes step desynchronization and high implementation complexity.
- TokenRouter: Efficient Serving System for Token-Level LLM Routingraises · addresses
Length generalization in recurrent models2 papers raise it · 2 address it
Recurrent or looped models fail to extrapolate to much longer sequences than seen in training.
- PAR: Progress-Aligned Recurrence for Length Generalization in State Trackingraises · addresses
- LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocationraises · addresses
Sparse attention computation overhead1 paper raises it · 1 addresses it
Sparse attention still requires scoring all keys, limiting speedups despite reduced memory access.
- DoSA: Dots Sparse Attentionraises · addresses
Cache management for long trajectories1 paper raises it · 1 addresses it
KV cache I/O and storage dominate costs, and cache operations can invalidate states or require re-prefilling.
- Cache Engineering for Agentsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

Yizhu TechnologyStage not given · Oct 23$149.9M
Mach9Series A · Oct 23$34M
Youzhi ChuangxinSeed · Oct 23$15M
CygridPre-seed · Oct 23$3M
HeyVoliPre-seed · Oct 23$500k

Recently funded

Topics in this field

agent memory · memory construction · term memory270 papers42 this week
diffusion language models · language models diffusion · models diffusion language164 papers34 this week
gpus · gpu · throughput234 papers28 this week
continual learning · catastrophic forgetting · forgetting249 papers24 this week
speculative decoding accelerates · self speculative decoding · speculative decoding97 papers22 this week
looped language models · recurrent depth · loops98 papers21 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CVSep 30
MatLoom: Layered Text-to-Material Generation in a Compact Program Space
Anson Y. Lam, Shuqing Li, Michael R. Lyu
Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented language for text-to-material generation with pretrained language models. Each program composes alpha-masked layers whose shared spatial expressions define coverage and physically based rendering (PBR) channels, making dependencies between patterns, color, and relief explicit. A standalone interpreter evaluates the program into material maps, while the source retains named fields and layer parameters for subsequent authoring. Without task-specific fine-tuning, our pipeline uses parser-guided repair and preview-based critique to revise material designs, then searches noise seeds while keeping each candidate's remaining source fixed. On a curated benchmark of 141 prompts evaluated with six backbones, our best-performing configuration achieves higher mean scores than three diffusion baselines on all four flat-layout prompt-alignment metrics. Its initial programs already exceed all three baselines on mean BLIPScore, before critique or seed search. Retained programs have a median length of 21 lines when pooled across backbones. In a blind…
PDF arXiv
arXiv cs.LGSep 30
Scaling Laws for Looped Mixture of Experts
Yanbei Chen, Anirudh Goyal, Raghuraman Krishnamoorthi
Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed active compute. Yet existing scaling laws model recurrence or sparsity in isolation. In this work, we introduce Loop Scaling Laws, the first scaling law to jointly model recurrence and sparsity alongside model size and data. At its core is a bounded, sparsity-conditional recurrence mapping that characterizes the effective-parameter gain from looping and how sparsity raises this gain. The laws predict the held-out loss of looped models more accurately than prior alternatives, and recover the standard dense and MoE scaling laws as special cases. Beyond prediction, the fitted laws provide a principled foundation for designing looped MoE models under compute and memory constraints. Downstream evaluations further demonstrate the complementary benefits of the two axes: sparsity delivers ~3x active-parameter efficiency, recurrence yields ~2x total-parameter efficiency on reasoning, and joint scaling further advances the performance frontier. As a practical extension, we show…
PDF arXiv
arXiv cs.LGSep 30
Disentangling Computation in Multi-Task Neural Networks with the Green's Operator
James Hazelden
How is computation organized and reused across tasks and time in a trained recurrent network? Most analyses emphasize the geometry of neural activity, dynamical motifs, or local perturbation growth. We instead study the network's global first-order perturbation response. The finite-horizon Green's operator maps perturbations at each source along a trajectory to their downstream state-space responses and therefore directly represents perturbation routing. Simple reductions of this operator provide task-to-task and time-to-time views of the same computation, while matrix-free products make these views accessible without constructing the full operator. In a flexible multitask recurrent network, task reductions reveal structured reuse of known computational motifs, while temporal reductions reveal causal pathways and how they emerge during training. Our main point is simple: the Green's operator provides a global response geometry for mapping the organization of learned dynamical computation.
PDF arXiv
arXiv cs.CLSep 30
Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning
Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
Unlearning a fact in one language does not guarantee its removal in others as changing the query or even the requested answer language can reopen seemingly forgotten knowledge -- a cross-lingual loophole. The most straightforward solution to this challenge -- unlearning in all languages -- is neither scalable nor desirable as it amplifies damage to unrelated model capabilities. We introduce the task of language budgeted multilingual unlearning where the goal is to select a subset of languages that maximizes cross-lingual erasure. To study this task we introduce the Cross-Lingual Unlearning Tensor, an unlearning benchmark that spans 174 language--script pairs and 25 atomic paraphrase types to examine when forgetting generalizes across linguistic expressions of the same knowledge. We further propose COVER, which selects source languages to maximize predicted COVERage of languages receiving no forget supervision, enabling unlearning on a language budget. Surprisingly, we find naively selecting strong individual sources does not reliably compose into strong source sets motivating our development of COVER. At deployment COVER only requires benign calibration data and access to the…
PDF arXiv
arXiv cs.LGSep 30
Distribution Matching Distillation for Continuous Diffusion Language Models
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli, Alain Durmus
Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributional distillation can reduce this cost by exploiting the student's probabilistic token outputs. Our unified formulation connects the student's output parameterization to the resulting gradient estimators and yields two methods with the same student architecture and reverse-KL matching objective: Simplex-DMD uses continuous token relaxations and pathwise gradients, while Reinforce-DMD uses categorical sampling and REINFORCE with a learned density ratio. We develop both methods for multi-step generation and investigate the training and sampling choices associated with each parameterization. On OpenWebText, for sequences of 1,024 tokens, Simplex-DMD achieves a generative perplexity of 45.6 at a unigram entropy of 5.44 nats in just 4 NFEs, a 49% reduction relative to the strongest evaluated diffusion baseline at matched entropy and sampling budget. Reinforce-DMD improves the frontier at larger budgets, reaching a generative perplexity of 14.9 at an entropy of 5.00 nats with 256 NFEs, a 20% reduction under…
PDF arXiv
arXiv cs.CLSep 30
Provably Tractable NFA-Constrained Language Generation via HMMs
Jialiang Sun, Kuldeep Meel
Constrained generation aims to sample from language models (LMs) conditioned on hard constraints. Existing constrained-generation techniques for nondeterministic finite automaton (NFA) constraints either distort the distribution or sacrifice efficiency. Theoretically, this task reduces to counting the length-$n$ sequences accepted by an NFA (#NFA), and the exact #NFA problem is #P-complete. Recent work has shown that #NFA admits a fully polynomial randomized approximation scheme (FPRAS). Inspired by this result, we propose NFA-LM, a polynomial-time engine for NFA-constrained generation with theoretical guarantees under mild assumptions. Experiments show that NFA-LM efficiently generates high-quality outputs with theoretically bounded approximation error.
PDF arXiv

Papers

Top tools here

All tools

NVIDIA Dynamo+81% downloads · 16.5k/wk+81.4%
BentoML+32% downloads · 42.8k/wk+31.7%
MLX+27% downloads · 357.1k/wk+27.3%
vLLM1 release this week
ort+5% downloads · 922.2k/wk+5.2%

Recently funded

Market

Yizhu TechnologyStage not given · Oct 23$149.9M
Mach9Series A · Oct 23$34M
Youzhi ChuangxinSeed · Oct 23$15M
CygridPre-seed · Oct 23$3M
HeyVoliPre-seed · Oct 23$500k

Topics in this field

agent memory · memory construction · term memory270 papers42 this week
diffusion language models · language models diffusion · models diffusion language164 papers34 this week
gpus · gpu · throughput234 papers28 this week
continual learning · catastrophic forgetting · forgetting249 papers24 this week
speculative decoding accelerates · self speculative decoding · speculative decoding97 papers22 this week
looped language models · recurrent depth · loops98 papers21 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.CVSep 30
MatLoom: Layered Text-to-Material Generation in a Compact Program Space
Anson Y. Lam, Shuqing Li, Michael R. Lyu
Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented language for text-to-material generation with pretrained language models. Each program composes alpha-masked layers whose shared spatial expressions define coverage and physically based rendering (PBR) channels, making dependencies between patterns, color, and relief explicit. A standalone interpreter evaluates the program into material maps, while the source retains named fields and layer parameters for subsequent authoring. Without task-specific fine-tuning, our pipeline uses parser-guided repair and preview-based critique to revise material designs, then searches noise seeds while keeping each candidate's remaining source fixed. On a curated benchmark of 141 prompts evaluated with six backbones, our best-performing configuration achieves higher mean scores than three diffusion baselines on all four flat-layout prompt-alignment metrics. Its initial programs already exceed all three baselines on mean BLIPScore, before critique or seed search. Retained programs have a median length of 21 lines when pooled across backbones. In a blind…
PDF arXiv
arXiv cs.LGSep 30
Scaling Laws for Looped Mixture of Experts
Yanbei Chen, Anirudh Goyal, Raghuraman Krishnamoorthi
Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed active compute. Yet existing scaling laws model recurrence or sparsity in isolation. In this work, we introduce Loop Scaling Laws, the first scaling law to jointly model recurrence and sparsity alongside model size and data. At its core is a bounded, sparsity-conditional recurrence mapping that characterizes the effective-parameter gain from looping and how sparsity raises this gain. The laws predict the held-out loss of looped models more accurately than prior alternatives, and recover the standard dense and MoE scaling laws as special cases. Beyond prediction, the fitted laws provide a principled foundation for designing looped MoE models under compute and memory constraints. Downstream evaluations further demonstrate the complementary benefits of the two axes: sparsity delivers ~3x active-parameter efficiency, recurrence yields ~2x total-parameter efficiency on reasoning, and joint scaling further advances the performance frontier. As a practical extension, we show…
PDF arXiv
arXiv cs.LGSep 30
Disentangling Computation in Multi-Task Neural Networks with the Green's Operator
James Hazelden
How is computation organized and reused across tasks and time in a trained recurrent network? Most analyses emphasize the geometry of neural activity, dynamical motifs, or local perturbation growth. We instead study the network's global first-order perturbation response. The finite-horizon Green's operator maps perturbations at each source along a trajectory to their downstream state-space responses and therefore directly represents perturbation routing. Simple reductions of this operator provide task-to-task and time-to-time views of the same computation, while matrix-free products make these views accessible without constructing the full operator. In a flexible multitask recurrent network, task reductions reveal structured reuse of known computational motifs, while temporal reductions reveal causal pathways and how they emerge during training. Our main point is simple: the Green's operator provides a global response geometry for mapping the organization of learned dynamical computation.
PDF arXiv
arXiv cs.CLSep 30
Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning
Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
Unlearning a fact in one language does not guarantee its removal in others as changing the query or even the requested answer language can reopen seemingly forgotten knowledge -- a cross-lingual loophole. The most straightforward solution to this challenge -- unlearning in all languages -- is neither scalable nor desirable as it amplifies damage to unrelated model capabilities. We introduce the task of language budgeted multilingual unlearning where the goal is to select a subset of languages that maximizes cross-lingual erasure. To study this task we introduce the Cross-Lingual Unlearning Tensor, an unlearning benchmark that spans 174 language--script pairs and 25 atomic paraphrase types to examine when forgetting generalizes across linguistic expressions of the same knowledge. We further propose COVER, which selects source languages to maximize predicted COVERage of languages receiving no forget supervision, enabling unlearning on a language budget. Surprisingly, we find naively selecting strong individual sources does not reliably compose into strong source sets motivating our development of COVER. At deployment COVER only requires benign calibration data and access to the…
PDF arXiv
arXiv cs.LGSep 30
Distribution Matching Distillation for Continuous Diffusion Language Models
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli, Alain Durmus
Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributional distillation can reduce this cost by exploiting the student's probabilistic token outputs. Our unified formulation connects the student's output parameterization to the resulting gradient estimators and yields two methods with the same student architecture and reverse-KL matching objective: Simplex-DMD uses continuous token relaxations and pathwise gradients, while Reinforce-DMD uses categorical sampling and REINFORCE with a learned density ratio. We develop both methods for multi-step generation and investigate the training and sampling choices associated with each parameterization. On OpenWebText, for sequences of 1,024 tokens, Simplex-DMD achieves a generative perplexity of 45.6 at a unigram entropy of 5.44 nats in just 4 NFEs, a 49% reduction relative to the strongest evaluated diffusion baseline at matched entropy and sampling budget. Reinforce-DMD improves the frontier at larger budgets, reaching a generative perplexity of 14.9 at an entropy of 5.00 nats with 256 NFEs, a 20% reduction under…
PDF arXiv
arXiv cs.CLSep 30
Provably Tractable NFA-Constrained Language Generation via HMMs
Jialiang Sun, Kuldeep Meel
Constrained generation aims to sample from language models (LMs) conditioned on hard constraints. Existing constrained-generation techniques for nondeterministic finite automaton (NFA) constraints either distort the distribution or sacrifice efficiency. Theoretically, this task reduces to counting the length-$n$ sequences accepted by an NFA (#NFA), and the exact #NFA problem is #P-complete. Recent work has shown that #NFA admits a fully polynomial randomized approximation scheme (FPRAS). Inspired by this result, we propose NFA-LM, a polynomial-time engine for NFA-constrained generation with theoretical guarantees under mild assumptions. Experiments show that NFA-LM efficiently generates high-quality outputs with theoretically bounded approximation error.
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 20m ago

Inference & serving

Making models fast and cheap to run: serving systems, kernels, KV-cache, speculative decoding, hardware.

242papers this week−19.1% vs last week

All papers · 1.6k With code · 69

Newest Most relevant

1.5k papers this month

Open problems in Inference & serving

Efficient long-horizon context management4 papers raise it · 4 address it
Long agent trajectories exceed context windows, degrading performance without compact, reusable representations.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- Cache Engineering for Agentsraises · addresses
- MemT-1.0: Towards Memory-Native Evolutionraises · addresses
Token-level routing system efficiency1 paper raises it · 1 addresses it
Fine-grained routing improves cost-quality trade-offs but causes step desynchronization and high implementation complexity.
- TokenRouter: Efficient Serving System for Token-Level LLM Routingraises · addresses
Length generalization in recurrent models2 papers raise it · 2 address it
Recurrent or looped models fail to extrapolate to much longer sequences than seen in training.
- PAR: Progress-Aligned Recurrence for Length Generalization in State Trackingraises · addresses
- LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocationraises · addresses
Sparse attention computation overhead1 paper raises it · 1 addresses it
Sparse attention still requires scoring all keys, limiting speedups despite reduced memory access.
- DoSA: Dots Sparse Attentionraises · addresses
Cache management for long trajectories1 paper raises it · 1 addresses it
KV cache I/O and storage dominate costs, and cache operations can invalidate states or require re-prefilling.
- Cache Engineering for Agentsraises · addresses

Named by an AI from these papers' abstracts.