Papers

Field

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
BentoML+30% downloads · 41k/wk+29.7%
MLX+26% downloads · 342.6k/wk+25.5%
mlx-lm+8% downloads · 140.9k/wk+7.5%
vLLM1 release this week

Top tools here

Open problems in Small LLMs

Efficient quantization for small LLMs without accuracy loss3 papers raise it · 2 address it
Enables deployment of small models on resource-constrained devices while preserving performance.
- N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Modelsraises · addresses
- RoleQ: Role-Aligned Scales for Converting Language Models to Ternaryraises · addresses
- Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensorsraises
Secure on-device model deployment against reverse engineering1 paper raises it · 1 addresses it
Protects proprietary small models from IP theft and tampering in edge environments.
- BiShield-TEE: On the (In-)Security of Unilateral Weight Obfuscation in On-Device TEE-Shielded LLM Partitionraises · addresses
Effective distillation for small LLMs with limited teacher access1 paper raises it · 1 addresses it
Improves small model performance when full teacher access or internal representations are unavailable.
- Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixingraises · addresses
Task-aware tokenization for morphologically rich languages1 paper raises it · 1 addresses it
Improves model performance in languages with complex morphology by preserving linguistic structure.
- Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalograises · addresses
Dynamic generation order for sequence labeling in low-resource settings1 paper raises it · 1 addresses it
Reduces error propagation in autoregressive models for tasks with non-linear dependencies.
- Dynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenariosraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Recently funded

Topics in this field

Active contributors · 90 days

Giacomo BaisDynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenarios1
Tejaswini DeoskarDynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenarios1
Noman KhanPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Remus TeodorescuPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Shahariya Rasheed Rayamarakkarveettil PuthanpurayilPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Vlad Alexandru SănduleacPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.LGOct 14
Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixing
Andrey Goncharov, Daniil Vyazhev, Simon Epanov, Alexey Zaytsev
Distillation of reasoning chains from Large Language Models (LLMs) into their smaller siblings is a standard training approach. Frequently, researchers do not have full access to a teacher model and its internal representations, which limits the resulting performance. We propose a recursive offline distillation method combining two components: entropy-based sampling for selective distillation and answer mixing. Specifically, for three open models (3-4B), at each epoch, we sample reasoning chains weighted by entropy difference with an auxiliary medium model (70-72B) and, next, supplement thinking traces with single-token ground truth answers. Our approach consistently outperforms naive distillation with comparable data budgets ($57.15 \pm 1.01$ vs 51.61 on average) and offers a much steeper learning curve. Additionally, we present a comparative analysis of methods to generate and normalize distillation traces.
PDF arXiv
arXiv cs.CLOct 14
Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalog
Abdul Rehman Antall, Baidya Nath Saha
Typology-aware tokenizers are motivated by the claim that segmentation which respects morpheme boundaries preserves information that frequency-driven segmentation destroys. Studies routinely establish that it helps while assuming the boundaries must be linguistically correct. We separate them. Tagalog marks voice with infixes (s-um-ulat 'wrote', s-in-ulat 'was written'), so the root is not contiguous and no sequence of BPE merges can recover it. We build MorphAustro-Tok, a tokenizer that emits the root as a contiguous unit followed by a positional infix token, and compare it against a corpus-matched BPE baseline and, at two of three fertility levels, against a scrambled-morpheme control that preserves morph count, invertibility and per-type consistency while randomising the pseudo-infix. Across 33 runs, MorphAustro-Tok improves Tagalog machine translation over matched BPE by 0.84, 1.49 and 3.40 mean sentence-level chrF++ points. Token count does not explain the gain, since it grows as segmentation coarsens, and neither does capacity: at the coarsest level the scrambled arm carries more parameters and scores lower. Scrambling the analysis moves the gain by less than between-seed…
PDF arXiv
arXiv cs.LGOct 14
Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors
Yifan Liu
We investigate whether per-layer, per-task-phase (prompt/reasoning/response) expert-routing statistics in a Mixture-of-Experts (MoE) language model can guide layer-wise mixed-precision quantization of routed-expert FFN tensors (down/gate and up). To attain layerwise routing statistics, we build a routing-capture and profiling pipeline for Qwen-3.6-35B-A3B in ik_llama.cpp, derive a composite sensitivity score from routing entropy/decisiveness, coefficient of variation (CV), Jaccard/phase-fragility, and top-1 expert strength, and use coordinate descent to tune a layer allocation that respects a fixed average bits-per-weight budget. Notably, the profiling pipeline is applicable to standard top-k MoE architectures. Across independent experimental conditions (about 4.0 bpw flat, about 2.7 bpw steep, and about 2.7 bpw random allocation as a control), reallocation fails to outperform the uniform baseline within each quality tier. In the flat regime, reallocation is statistically neutral (z about 0.9). For the steep tier, the random reallocation performs within 1 SE of the routing-guided reallocation at the same tier, indicating a structural loss at this quality rung caused by mandatory…
PDF arXiv
arXiv cs.CLOct 14
Wolof-Adapted Industry-Ready Open-Weight Language Models
Gwénolé Lecorvé, Delphine Charlet, Pierre Adam, Floriane Behanzin, Lionel Delphin-Poulat +13
Wolof is spoken by 23M people, mostly in Senegal, but remains weakly supported by open-weight language models, while API models raise cost, governance, and sovereignty constraints for local deployments. In this paper, we present 0.6B-14B open-weight models adapted to Wolof through instruction tuning, preference alignment, and model merging. After presenting the pipeline and data, extensive experiments on 63~public and in-house tasks show good performance on Wolof understanding and generation while remaining effective on other languages and cross-lingual skills, with the 8B and 14B models being particularly competitive with substantially larger commercial systems. The models are currently being evaluated in internal customer-care pilots and are publicly available at https://huggingface.co/collections/Orange/wolof-lm .
PDF arXiv
arXiv cs.CLOct 14
Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado Porto
Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples. We document three DPO attempts, all negative, including a case where the training reward margin reaches 1.0 while free generation does not change at all, and show that safety came from a dedicated safety-SFT (LLM-jp AnswerCarefully style), refined over three iterations until closing the safety-helpfulness frontier. A controlled scan of the fraction of general data in the safety-SFT mix (10% to 60%) reveals a non-monotonic over-refusal curve with a knee point at 40-50%. The final model is compared against four external PT-BR models on the same items, with Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction: the lower bound of its safety interval (94.1%) lies above the upper bound of all four baselines. We also report a methodological finding for LLM-judge evaluation: on two…
PDF arXiv
arXiv cs.LGOct 14
SpectrumQuant: Spectrum Alignment for LLM Quantization
Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula
Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an…
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
BentoML+30% downloads · 41k/wk+29.7%
MLX+26% downloads · 342.6k/wk+25.5%
mlx-lm+8% downloads · 140.9k/wk+7.5%
vLLM1 release this week

Recent papers

All in Research →

arXiv cs.LGOct 14
Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixing
Andrey Goncharov, Daniil Vyazhev, Simon Epanov, Alexey Zaytsev
Distillation of reasoning chains from Large Language Models (LLMs) into their smaller siblings is a standard training approach. Frequently, researchers do not have full access to a teacher model and its internal representations, which limits the resulting performance. We propose a recursive offline distillation method combining two components: entropy-based sampling for selective distillation and answer mixing. Specifically, for three open models (3-4B), at each epoch, we sample reasoning chains weighted by entropy difference with an auxiliary medium model (70-72B) and, next, supplement thinking traces with single-token ground truth answers. Our approach consistently outperforms naive distillation with comparable data budgets ($57.15 \pm 1.01$ vs 51.61 on average) and offers a much steeper learning curve. Additionally, we present a comparative analysis of methods to generate and normalize distillation traces.
PDF arXiv
arXiv cs.CLOct 14
Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalog
Abdul Rehman Antall, Baidya Nath Saha
Typology-aware tokenizers are motivated by the claim that segmentation which respects morpheme boundaries preserves information that frequency-driven segmentation destroys. Studies routinely establish that it helps while assuming the boundaries must be linguistically correct. We separate them. Tagalog marks voice with infixes (s-um-ulat 'wrote', s-in-ulat 'was written'), so the root is not contiguous and no sequence of BPE merges can recover it. We build MorphAustro-Tok, a tokenizer that emits the root as a contiguous unit followed by a positional infix token, and compare it against a corpus-matched BPE baseline and, at two of three fertility levels, against a scrambled-morpheme control that preserves morph count, invertibility and per-type consistency while randomising the pseudo-infix. Across 33 runs, MorphAustro-Tok improves Tagalog machine translation over matched BPE by 0.84, 1.49 and 3.40 mean sentence-level chrF++ points. Token count does not explain the gain, since it grows as segmentation coarsens, and neither does capacity: at the coarsest level the scrambled arm carries more parameters and scores lower. Scrambling the analysis moves the gain by less than between-seed…
PDF arXiv
arXiv cs.LGOct 14
Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors
Yifan Liu
We investigate whether per-layer, per-task-phase (prompt/reasoning/response) expert-routing statistics in a Mixture-of-Experts (MoE) language model can guide layer-wise mixed-precision quantization of routed-expert FFN tensors (down/gate and up). To attain layerwise routing statistics, we build a routing-capture and profiling pipeline for Qwen-3.6-35B-A3B in ik_llama.cpp, derive a composite sensitivity score from routing entropy/decisiveness, coefficient of variation (CV), Jaccard/phase-fragility, and top-1 expert strength, and use coordinate descent to tune a layer allocation that respects a fixed average bits-per-weight budget. Notably, the profiling pipeline is applicable to standard top-k MoE architectures. Across independent experimental conditions (about 4.0 bpw flat, about 2.7 bpw steep, and about 2.7 bpw random allocation as a control), reallocation fails to outperform the uniform baseline within each quality tier. In the flat regime, reallocation is statistically neutral (z about 0.9). For the steep tier, the random reallocation performs within 1 SE of the routing-guided reallocation at the same tier, indicating a structural loss at this quality rung caused by mandatory…
PDF arXiv
arXiv cs.CLOct 14
Wolof-Adapted Industry-Ready Open-Weight Language Models
Gwénolé Lecorvé, Delphine Charlet, Pierre Adam, Floriane Behanzin, Lionel Delphin-Poulat +13
Wolof is spoken by 23M people, mostly in Senegal, but remains weakly supported by open-weight language models, while API models raise cost, governance, and sovereignty constraints for local deployments. In this paper, we present 0.6B-14B open-weight models adapted to Wolof through instruction tuning, preference alignment, and model merging. After presenting the pipeline and data, extensive experiments on 63~public and in-house tasks show good performance on Wolof understanding and generation while remaining effective on other languages and cross-lingual skills, with the 8B and 14B models being particularly competitive with substantially larger commercial systems. The models are currently being evaluated in internal customer-care pilots and are publicly available at https://huggingface.co/collections/Orange/wolof-lm .
PDF arXiv
arXiv cs.CLOct 14
Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado Porto
Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples. We document three DPO attempts, all negative, including a case where the training reward margin reaches 1.0 while free generation does not change at all, and show that safety came from a dedicated safety-SFT (LLM-jp AnswerCarefully style), refined over three iterations until closing the safety-helpfulness frontier. A controlled scan of the fraction of general data in the safety-SFT mix (10% to 60%) reveals a non-monotonic over-refusal curve with a knee point at 40-50%. The final model is compared against four external PT-BR models on the same items, with Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction: the lower bound of its safety interval (94.1%) lies above the upper bound of all four baselines. We also report a methodological finding for LLM-judge evaluation: on two…
PDF arXiv
arXiv cs.LGOct 14
SpectrumQuant: Spectrum Alignment for LLM Quantization
Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula
Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an…
PDF arXiv

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 31m ago

Small LLMs

Compact or efficient language models (roughly under 15B parameters), distillation, quantization and on-device LLMs.

12papers this week

All papers · 12 With code · 0

Newest Most relevant

12 papers this month

Active contributors · 90 days

Giacomo BaisDynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenarios1
Tejaswini DeoskarDynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenarios1
Noman KhanPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Remus TeodorescuPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Shahariya Rasheed Rayamarakkarveettil PuthanpurayilPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1
Vlad Alexandru SănduleacPhysics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference1

Open problems in Small LLMs

Efficient quantization for small LLMs without accuracy loss3 papers raise it · 2 address it
Enables deployment of small models on resource-constrained devices while preserving performance.
- N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Modelsraises · addresses
- RoleQ: Role-Aligned Scales for Converting Language Models to Ternaryraises · addresses
- Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensorsraises
Secure on-device model deployment against reverse engineering1 paper raises it · 1 addresses it
Protects proprietary small models from IP theft and tampering in edge environments.
- BiShield-TEE: On the (In-)Security of Unilateral Weight Obfuscation in On-Device TEE-Shielded LLM Partitionraises · addresses
Effective distillation for small LLMs with limited teacher access1 paper raises it · 1 addresses it
Improves small model performance when full teacher access or internal representations are unavailable.
- Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixingraises · addresses
Task-aware tokenization for morphologically rich languages1 paper raises it · 1 addresses it
Improves model performance in languages with complex morphology by preserving linguistic structure.
- Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalograises · addresses
Dynamic generation order for sequence labeling in low-resource settings1 paper raises it · 1 addresses it
Reduces error propagation in autoregressive models for tasks with non-linear dependencies.
- Dynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenariosraises · addresses

Named by an AI from these papers' abstracts.