Papers

Field

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
BentoML+30% downloads · 41k/wk+29.7%
MLX+26% downloads · 342.6k/wk+25.5%
mlx-lm+8% downloads · 140.9k/wk+7.5%
vLLM1 release this week

Top tools here

Open problems in Inference & serving

Efficient long-horizon context management3 papers raise it · 3 address it
Long-horizon agents need to retain and reuse information without exceeding finite context windows or degrading performance.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- MemT-1.0: Towards Memory-Native Evolutionraises · addresses
Token-level LLM routing efficiency1 paper raises it · 1 addresses it
Fine-grained token-level routing improves cost-quality trade-offs but strains existing serving systems with desynchronization and complexity.
- TokenRouter: Efficient Serving System for Token-Level LLM Routingraises · addresses
Memory-induced sycophancy in agents1 paper raises it · 1 addresses it
Persistent memories can cause agents to over-align with user beliefs, even when those beliefs are incorrect or outdated.
- Memadapter: Counterfactual Adaptation Against Memory-induced Sycophancyraises · addresses
Quality gap in parallel token generation2 papers raise it · 2 address it
Diffusion language models generate tokens in parallel but lag behind autoregressive models in quality due to uniform computation allocation.
- ALoDLM: Adaptively Looped Diffusion Language Modelsraises · addresses
- Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Modelsraises · addresses
Sparse attention computation overhead1 paper raises it · 1 addresses it
Sparse attention reduces computation but still requires scoring all keys, limiting efficiency gains.
- DoSA: Dots Sparse Attentionraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

EMode PhotonixPre-seed · Oct 16$1M
MueonSeries A · Oct 15$65M
TrimagnetixSeed · Oct 15$1M
ModalSeries B+ · Oct 14$750M
Salience LabsSeries B+ · Oct 14$125M

Recently funded

Topics in this field

agent memory · memory construction · term memory257 papers52 this week
gpus · gpu · throughput220 papers32 this week
diffusion language models · language models diffusion · models diffusion language150 papers29 this week
continual learning · catastrophic forgetting · forgetting237 papers25 this week
speculative decoding accelerates · self speculative decoding · speculative decoding89 papers24 this week
sparse attention · linear attention · attention layers134 papers23 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.LGOct 15
Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing
Adam Deng, Shicheng Liu, Yada Zhu
Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that must communicate needs its own transfer bridge, and every agent added to a system needs a new bridge fitted against each previous agent. We reframe this problem as sharing: an agent publishes its internal state once into a pool, and every peer reads that pool through a single mechanism fitted to no sender in particular, so adding an agent adds a publisher and no new bridge. Whereas the carrier of the internal state and the fusion mechanism are usually chosen independently in the literature, our main finding is that these two choices are coupled; the deciding factor is where the carrier enters the receiver's computation. Consumability is a property of the receiving computation rather than of the representation. We find that the key-value (KV) cache must be augmented rather than swapped in: the receiver cross-attends over a peer's cache while keeping its own, and…
PDF arXiv
arXiv cs.LGOct 15
Close Enough is Nowhere Near: Fidelity-Consumability Decoupling in KV Caches
Adam Deng, Shicheng Liu, Yada Zhu
Multi-agent LLM systems communicate in text, which is lossy and wasteful: the sender throws away the reading it already did. A growing literature instead transmits the KV cache, the natural carrier of that reading. We study the most appealing form of the idea: both models stay frozen and the sender's cache is substituted for the reader's with nothing trained. It does not work, and the obstruction belongs to the reader. A frozen transformer reads only caches close to the one it would have written itself, a region we call its tolerable set. We show that geometric similarity to that cache does not predict whether the reader can use it, and the failure is not gradual, so a more faithful substitute is not the remedy it appears to be. We explain this phenomenon with competitive attention. Experimentally, we test eight diverse substitution methods against two measured anchors: an uninformative cache baseline and the reader's own accuracy from having read the passage. All sit at the baseline. That turns a long tail of separate negative results into one outcome with a stated condition for overturning it. A different design does work: if the reader keeps its own cache and attends over the…
PDF arXiv
arXiv cs.LGOct 15
Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting
Kamil Faber, Katarzyna Filus, Łukasz Pawela
Catastrophic forgetting remains a challenge in supervised continual learning, as adapting to new tasks causes representation drift and loss of previously learned information. This raises the question of which representational changes should be constrained while preserving plasticity. We therefore introduce Sparse Autoencoder Geometric Anchoring (SAGA), which converts a Sparse Autoencoder (SAE) trained at each task boundary into local, low-rank constraints on representation drift. Replay examples specify where the constraints apply, while leading right singular directions of the encoder Jacobian specify which changes are penalized. The resulting geometry is input dependent and can be retained without keeping the SAE for subsequent tasks' training. We establish its local relation to latent code matching, define its spectral truncation error, and derive a conditional bound on frozen-readout drift. With replay and snapshot history, the method achieves the highest mean Class-IL accuracy and lowest mean forgetting among the evaluated ER, DER++, ER-ACE, and SGD baselines on three pretrained ViT-B/16 benchmarks, with accuracy gains of 12.15, 4.79, and 3.22 percentage points over the…
PDF arXiv
arXiv cs.LGOct 15
Time to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Kajaree Das, Oliver Böse, Subarnaduti Paul, Gabriel Storch, Martin Mundt
Deepfake detectors safeguard against misinformation, rendering their failures as much a societal as a technical concern. As generators constantly evolve, detectors trained on static datasets miss newly arising threats, while progressive training interferes catastrophically. Yet, mitigating forgetfulness in terms of accuracy, the prevalent focus of continual learning (CL), is insufficient to reflect real-world impact. We therefore scrutinize the effects of CL methods with respect to evolving fairness. To this end, we analyze four dimensions of fairness across diverse demographics for detectors trained progressively on a chronological sequence of 11 real-world deepfake generation methods. Tracking fairness metrics over time, we expose non-trivial interplays between performance and fairness that depend on both the choice of CL algorithm and demographic intersections. We find accuracy degradation to typically exacerbate unfairness and, conversely, fairness improvements to remain scarce over time. Dissecting the CL method families (rehearsal, prompts, regularization), we observe that inequitable forgetting is not explained by underrepresentation alone, but likely entangled with…
PDF arXiv
arXiv cs.CLOct 15
TEM: Temporal State Tracking Memory for Large Language Model Agents
Pengfeng Song, Yunkai Zhuang, Liye Shi
An agent's memory must remain useful as information changes across conversations. A new fact may replace an earlier value without making that value irrelevant to a later question. We present Temporal State Tracking Memory (TEM), a training-free framework for tracking these changes. TEM stores facts as versioned records linked to their sources. It retrieves relevant versions, places them in source order and labels historical records before passing the evidence to the answer model. Event memory and original dialogue turns provide details missing from structured facts. We evaluated TEM on 420 questions drawn from 600 sessions involving 60 personas. Across four open-weight models, key-entity hit rates reached 75.71-97.62% overall and 96.67-100.00% on the historical-residence subset. Adding historical labels to otherwise identical, source-ordered facts increased overall hit rates by 17.14-55.95 percentage points. The evidence was compact: 94-118 tokens per answer, or 48.75-56.73% less than session-aware retrieval-augmented generation (RAG). A further evaluation on 1,540 LoCoMo questions tested hybrid memory under a fixed evidence budget. TEM exceeded session-aware RAG with both…
PDF arXiv
arXiv cs.LGOct 15
Rank Measures of Fine-Tuning Updates Need a Noise Baseline
Muhammad Usman Altaf
Studies of what fine-tuning does to a language model often summarise the weight update $ΔW$ by its rank. We compare the rank measures used for this against the same measures on a random matrix of the same shape, and against two harder references: the pretrained weights, and matched fine-tunes without alignment content. Measures that count directions put our fine-tuning updates only a little below noise; stable rank, computed from the same singular values, puts them far below. Both are right: an update has a few strong directions on top of a long tail. The pattern holds for thirteen released pairs up to 8B parameters, and it is not a mark of alignment: with training held fixed, instruction data does not stand out from Wikipedia or code, DPO gives the same update whether its preference labels are real or half swapped at random (one model, modest preference strength), and scrambled text gives the most concentrated update of all. Applied to a recent claim that post-training is superficial because the effective rank of the weights barely changes, the baselines show that a random update of the same size leaves that rank almost unchanged while wrecking the model, and that an edit built…
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
BentoML+30% downloads · 41k/wk+29.7%
MLX+26% downloads · 342.6k/wk+25.5%
mlx-lm+8% downloads · 140.9k/wk+7.5%
vLLM1 release this week

Topics in this field

agent memory · memory construction · term memory257 papers52 this week
gpus · gpu · throughput220 papers32 this week
diffusion language models · language models diffusion · models diffusion language150 papers29 this week
continual learning · catastrophic forgetting · forgetting237 papers25 this week
speculative decoding accelerates · self speculative decoding · speculative decoding89 papers24 this week
sparse attention · linear attention · attention layers134 papers23 this week

Recently funded

Market

EMode PhotonixPre-seed · Oct 16$1M
MueonSeries A · Oct 15$65M
TrimagnetixSeed · Oct 15$1M
ModalSeries B+ · Oct 14$750M
Salience LabsSeries B+ · Oct 14$125M

Active contributors · 90 days

Recent papers

All in Research →

arXiv cs.LGOct 15
Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing
Adam Deng, Shicheng Liu, Yada Zhu
Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that must communicate needs its own transfer bridge, and every agent added to a system needs a new bridge fitted against each previous agent. We reframe this problem as sharing: an agent publishes its internal state once into a pool, and every peer reads that pool through a single mechanism fitted to no sender in particular, so adding an agent adds a publisher and no new bridge. Whereas the carrier of the internal state and the fusion mechanism are usually chosen independently in the literature, our main finding is that these two choices are coupled; the deciding factor is where the carrier enters the receiver's computation. Consumability is a property of the receiving computation rather than of the representation. We find that the key-value (KV) cache must be augmented rather than swapped in: the receiver cross-attends over a peer's cache while keeping its own, and…
PDF arXiv
arXiv cs.LGOct 15
Close Enough is Nowhere Near: Fidelity-Consumability Decoupling in KV Caches
Adam Deng, Shicheng Liu, Yada Zhu
Multi-agent LLM systems communicate in text, which is lossy and wasteful: the sender throws away the reading it already did. A growing literature instead transmits the KV cache, the natural carrier of that reading. We study the most appealing form of the idea: both models stay frozen and the sender's cache is substituted for the reader's with nothing trained. It does not work, and the obstruction belongs to the reader. A frozen transformer reads only caches close to the one it would have written itself, a region we call its tolerable set. We show that geometric similarity to that cache does not predict whether the reader can use it, and the failure is not gradual, so a more faithful substitute is not the remedy it appears to be. We explain this phenomenon with competitive attention. Experimentally, we test eight diverse substitution methods against two measured anchors: an uninformative cache baseline and the reader's own accuracy from having read the passage. All sit at the baseline. That turns a long tail of separate negative results into one outcome with a stated condition for overturning it. A different design does work: if the reader keeps its own cache and attends over the…
PDF arXiv
arXiv cs.LGOct 15
Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting
Kamil Faber, Katarzyna Filus, Łukasz Pawela
Catastrophic forgetting remains a challenge in supervised continual learning, as adapting to new tasks causes representation drift and loss of previously learned information. This raises the question of which representational changes should be constrained while preserving plasticity. We therefore introduce Sparse Autoencoder Geometric Anchoring (SAGA), which converts a Sparse Autoencoder (SAE) trained at each task boundary into local, low-rank constraints on representation drift. Replay examples specify where the constraints apply, while leading right singular directions of the encoder Jacobian specify which changes are penalized. The resulting geometry is input dependent and can be retained without keeping the SAE for subsequent tasks' training. We establish its local relation to latent code matching, define its spectral truncation error, and derive a conditional bound on frozen-readout drift. With replay and snapshot history, the method achieves the highest mean Class-IL accuracy and lowest mean forgetting among the evaluated ER, DER++, ER-ACE, and SGD baselines on three pretrained ViT-B/16 benchmarks, with accuracy gains of 12.15, 4.79, and 3.22 percentage points over the…
PDF arXiv
arXiv cs.LGOct 15
Time to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Kajaree Das, Oliver Böse, Subarnaduti Paul, Gabriel Storch, Martin Mundt
Deepfake detectors safeguard against misinformation, rendering their failures as much a societal as a technical concern. As generators constantly evolve, detectors trained on static datasets miss newly arising threats, while progressive training interferes catastrophically. Yet, mitigating forgetfulness in terms of accuracy, the prevalent focus of continual learning (CL), is insufficient to reflect real-world impact. We therefore scrutinize the effects of CL methods with respect to evolving fairness. To this end, we analyze four dimensions of fairness across diverse demographics for detectors trained progressively on a chronological sequence of 11 real-world deepfake generation methods. Tracking fairness metrics over time, we expose non-trivial interplays between performance and fairness that depend on both the choice of CL algorithm and demographic intersections. We find accuracy degradation to typically exacerbate unfairness and, conversely, fairness improvements to remain scarce over time. Dissecting the CL method families (rehearsal, prompts, regularization), we observe that inequitable forgetting is not explained by underrepresentation alone, but likely entangled with…
PDF arXiv
arXiv cs.CLOct 15
TEM: Temporal State Tracking Memory for Large Language Model Agents
Pengfeng Song, Yunkai Zhuang, Liye Shi
An agent's memory must remain useful as information changes across conversations. A new fact may replace an earlier value without making that value irrelevant to a later question. We present Temporal State Tracking Memory (TEM), a training-free framework for tracking these changes. TEM stores facts as versioned records linked to their sources. It retrieves relevant versions, places them in source order and labels historical records before passing the evidence to the answer model. Event memory and original dialogue turns provide details missing from structured facts. We evaluated TEM on 420 questions drawn from 600 sessions involving 60 personas. Across four open-weight models, key-entity hit rates reached 75.71-97.62% overall and 96.67-100.00% on the historical-residence subset. Adding historical labels to otherwise identical, source-ordered facts increased overall hit rates by 17.14-55.95 percentage points. The evidence was compact: 94-118 tokens per answer, or 48.75-56.73% less than session-aware retrieval-augmented generation (RAG). A further evaluation on 1,540 LoCoMo questions tested hybrid memory under a fixed evidence budget. TEM exceeded session-aware RAG with both…
PDF arXiv
arXiv cs.LGOct 15
Rank Measures of Fine-Tuning Updates Need a Noise Baseline
Muhammad Usman Altaf
Studies of what fine-tuning does to a language model often summarise the weight update $ΔW$ by its rank. We compare the rank measures used for this against the same measures on a random matrix of the same shape, and against two harder references: the pretrained weights, and matched fine-tunes without alignment content. Measures that count directions put our fine-tuning updates only a little below noise; stable rank, computed from the same singular values, puts them far below. Both are right: an update has a few strong directions on top of a long tail. The pattern holds for thirteen released pairs up to 8B parameters, and it is not a mark of alignment: with training held fixed, instruction data does not stand out from Wikipedia or code, DPO gives the same update whether its preference labels are real or half swapped at random (one model, modest preference strength), and scrambled text gives the most concentrated update of all. Applied to a recent claim that post-training is superficial because the effective rank of the weights barely changes, the baselines show that a random update of the same size leaves that rank almost unchanged while wrecking the model, and that an edit built…
PDF arXiv

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 24m ago

Inference & serving

Making models fast and cheap to run: serving systems, kernels, KV-cache, speculative decoding, hardware.

268papers this week−13.3% vs last week

All papers · 1.5k With code · 62

Newest Most relevant

1.5k papers this month

Open problems in Inference & serving

Efficient long-horizon context management3 papers raise it · 3 address it
Long-horizon agents need to retain and reuse information without exceeding finite context windows or degrading performance.
- REMORY: Learning Residual Memory for Context Compactionraises · addresses
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentsraises · addresses
- MemT-1.0: Towards Memory-Native Evolutionraises · addresses
Token-level LLM routing efficiency1 paper raises it · 1 addresses it
Fine-grained token-level routing improves cost-quality trade-offs but strains existing serving systems with desynchronization and complexity.
- TokenRouter: Efficient Serving System for Token-Level LLM Routingraises · addresses
Memory-induced sycophancy in agents1 paper raises it · 1 addresses it
Persistent memories can cause agents to over-align with user beliefs, even when those beliefs are incorrect or outdated.
- Memadapter: Counterfactual Adaptation Against Memory-induced Sycophancyraises · addresses
Quality gap in parallel token generation2 papers raise it · 2 address it
Diffusion language models generate tokens in parallel but lag behind autoregressive models in quality due to uniform computation allocation.
- ALoDLM: Adaptively Looped Diffusion Language Modelsraises · addresses
- Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Modelsraises · addresses
Sparse attention computation overhead1 paper raises it · 1 addresses it
Sparse attention reduces computation but still requires scoring all keys, limiting efficiency gains.
- DoSA: Dots Sparse Attentionraises · addresses

Named by an AI from these papers' abstracts.