Papers

Field

Top tools here

All tools

QwenPaw+1266% downloads · 39k/wk+1266.0%
billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
NVIDIA Dynamo+81% downloads · 16.5k/wk+81.4%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%

Top tools here

Open problems in Frontier LLMs

Efficient and accurate fact-checking of long, multi-claim texts1 paper raises it · 1 addresses it
Frontier LLMs are costly and slow for verifying complex review comments, while cheaper models fail on implicit or multi-part assertions.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Adaptive depth for decision models without token generation2 papers raise it · 3 address it
Fixed-depth decision models waste compute on easy questions, while generative reasoning is slow and token-costly.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces or filler text.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions, collapsing performance.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation for context-induced gains1 paper raises it · 1 addresses it
LLMs lose context-provided knowledge after context removal, limiting long-term retention of learned information.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

DeepSeekStage not given · Oct 22$15B
Flapping AirplanesSeries A · Oct 21$1B
Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M
Drylab AISeed · Oct 13$3M

Recently funded

Topics in this field

reasoning traces · reasoning models · latent reasoning405 papers56 this week
dense retrieval · retrieval augmented generation · retrieval278 papers56 this week
question answering · answering · large language models154 papers42 this week
attention heads · large language models · language models241 papers40 this week
large language models · attribution · language models llms219 papers37 this week
large language models · large language · language models196 papers33 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.AIOct 11
CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets
Anahita Sharma, Apaar Shanker, Rishav Chakravarti, Sneha Srinivasan, Vijay Kalmath +4
Enterprise agents need retrieval evaluation sets that expose domain-specific failures and guide developers towards efficient deployment tradeoffs. Neither expert-human-authored sets nor current synthetic generation methods provide a methodology for achieving the necessary coverage, reliability, efficiency, and discriminability required for such decision-making. Simultaneously, the field does not have an assessment framework for determining when an evaluation set has satisfied these properties. We introduce an evaluation set assessment framework for these criteria and CoverSynth -- a generator specifically designed to satisfy these criteria without incurring the high cost of human expert labeling. Starting from an unannotated corpus, CoverSynth targets coverage across retrieval demands and system capabilities. Deterministic checks locate evidence quotes in their sources, while model-based checks assess semantic validity to guarantee reliable ground truth. Across five public enterprise-style corpora, our method provides informative variation in difficulty, revealing unseen failure modes such as an inability of agents to handle underspecified or ambiguous requests. On the RepLiQA…
PDF arXiv
arXiv cs.CLOct 11
In-Place Indic Tokenizer Expansion and Continual Pretraining of Nemotron-3-Nano-30B-A3B
Avinash Benki, Prasanjith Pasunuti, Sutanshu Saksena Raj, Parijat Chatterjee, Nayanjit Sarkar +1
We adapted Nemotron-3-Nano-30B-A3B for eight Indian languages without changing its backbone architecture. We expanded the vocabulary in place from 131,072 to 163,072 entries using continued byte-pair encoding and AdaptMix rebalancing, reducing tokens per word by 38.6%. We initialized new input and output rows by averaging their original constituent rows, then trained them on 90% Indic and 10% English text before updating the backbone. Continual pretraining then progressed through broad training, high-quality annealing, and long-context training over approximately 1.27 trillion tokens. The annealed checkpoint gained 12.9 points on the Indic MILU benchmark but lost 3.4 on English MMLU. We also report practical negative results. Direct token addition underperformed continued byte-pair encoding. AdamW weight decay changed rows intended to stay fixed. Perplexity on an Indic-heavy holdout did not select the learning rate that best preserved English performance.
PDF arXiv
arXiv cs.CLOct 11
A Sovereign, Indic-First, Reasoning Model
Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik +2
Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA's Nemotron-3-Nano-30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved…
PDF arXiv
arXiv cs.LGOct 11
Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
Heng-Sheng Chang
A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations can catch deceptive responses that look honest. Yet current detectors depend on which layers they read, weaken on broader tests, and do not state how certain their alarms are. This white paper proposes the Machine Deception Detector, which targets both detection accuracy and the uncertainty of detector scores. The detector reads every layer of the language model, learns which tokens carry evidence of deception using only response labels, and calibrates its alarm threshold so that its false--positive rate stays below a prescribed level. An ensemble of such detectors measures the uncertainty of each score and raises an alarm only when its members agree, with a confidence level that sets how certain each alarm must be. The proposed evaluation trains on instructed honest and deceptive statements, tests on five unseen deceptive tasks of Liars' Bench read by six Qwen3 models from 0.6B to 32B parameters, and is designed to also show which layers carry the deception feature and how this changes with model size.
PDF arXiv
arXiv cs.CLOct 11
North or South? Diagnosing LLMs' Understanding of North Korean Language, Society, and Knowledge
Kiwoong Park, Seyoung Song, Donghyun Woo, Alice Oh
Large Language Models (LLMs) have made substantial progress in multilingual question answering, yet their ability to understand low-resource sociolinguistic contexts remains under-explored. We study North Korea as a unique testbed where decades of division have produced distinct differences from South Korea in vocabulary, orthography, norms, and ideological discourse. We introduce LENS (Language and sociocultural Evaluation for North Korean Society), a human-validated benchmark of 1,765 questions evaluating LLMs across three dimensions: linguistic robustness, sociocultural perspective alignment, and domain-specific knowledge. Evaluating diverse recent LLMs, we find a modest performance drop on North Korean language inputs, a larger gap in sociocultural perspective alignment, and severe limitations on domain-specific knowledge. Qualitative analysis reveals that models often conflate formal institutions with lived realities, project liberal-democratic assumptions, or rely on coarse stereotypes. Our results demonstrate that linguistic similarity alone does not guarantee sociocultural or domain-specific understanding, highlighting the critical need for grounded evaluation resources…
PDF arXiv
arXiv cs.CLOct 11
Do correct answers have stable latent knowledge support?
Chuancheng Shi, Shangze Li, Wenhua Wu, Kun Hu, Fei Shen +2
Large language models are typically evaluated in terms of the correctness of their output. However, output correctness alone does not confirm whether the latent knowledge supporting an answer has stable internal representations. We therefore propose latent state stability (LSS) and knowledge topology stability (KTS) to investigate the stability of individual knowledge states and their relational structure, respectively. Specifically, we first separately average the model's internal representations of the same knowledge under each elicitation condition. Using the mean representation under the baseline condition as a reference, we remove shared shifts to obtain corrected representations. We then formulate LSS as the distance between each corrected representation and its reference representation, normalized against the typical distance between knowledge items of the same type. We formulate KTS as the cross-condition consistency of the ordered pairwise representation distances among knowledge items of the same type. Next, we construct EpiStableBench, which comprises 4,411 items spanning factual knowledge and mathematical reasoning, with 76,766 controlled queries. Across 16 LLMs from…
PDF arXiv

Papers

Recently funded

Market

DeepSeekStage not given · Oct 22$15B
Flapping AirplanesSeries A · Oct 21$1B
Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M
Drylab AISeed · Oct 13$3M

Top tools here

All tools

QwenPaw+1266% downloads · 39k/wk+1266.0%
billion-context+259% downloads · 1.1M/wk+258.8%
jcode+233% downloads · 25.5k/wk+233.1%
NVIDIA Dynamo+81% downloads · 16.5k/wk+81.4%
oh-my-claudecode+47% downloads · 7.7k/wk+47.4%

Topics in this field

reasoning traces · reasoning models · latent reasoning405 papers56 this week
dense retrieval · retrieval augmented generation · retrieval278 papers56 this week
question answering · answering · large language models154 papers42 this week
attention heads · large language models · language models241 papers40 this week
large language models · attribution · language models llms219 papers37 this week
large language models · large language · language models196 papers33 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.AIOct 11
CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets
Anahita Sharma, Apaar Shanker, Rishav Chakravarti, Sneha Srinivasan, Vijay Kalmath +4
Enterprise agents need retrieval evaluation sets that expose domain-specific failures and guide developers towards efficient deployment tradeoffs. Neither expert-human-authored sets nor current synthetic generation methods provide a methodology for achieving the necessary coverage, reliability, efficiency, and discriminability required for such decision-making. Simultaneously, the field does not have an assessment framework for determining when an evaluation set has satisfied these properties. We introduce an evaluation set assessment framework for these criteria and CoverSynth -- a generator specifically designed to satisfy these criteria without incurring the high cost of human expert labeling. Starting from an unannotated corpus, CoverSynth targets coverage across retrieval demands and system capabilities. Deterministic checks locate evidence quotes in their sources, while model-based checks assess semantic validity to guarantee reliable ground truth. Across five public enterprise-style corpora, our method provides informative variation in difficulty, revealing unseen failure modes such as an inability of agents to handle underspecified or ambiguous requests. On the RepLiQA…
PDF arXiv
arXiv cs.CLOct 11
In-Place Indic Tokenizer Expansion and Continual Pretraining of Nemotron-3-Nano-30B-A3B
Avinash Benki, Prasanjith Pasunuti, Sutanshu Saksena Raj, Parijat Chatterjee, Nayanjit Sarkar +1
We adapted Nemotron-3-Nano-30B-A3B for eight Indian languages without changing its backbone architecture. We expanded the vocabulary in place from 131,072 to 163,072 entries using continued byte-pair encoding and AdaptMix rebalancing, reducing tokens per word by 38.6%. We initialized new input and output rows by averaging their original constituent rows, then trained them on 90% Indic and 10% English text before updating the backbone. Continual pretraining then progressed through broad training, high-quality annealing, and long-context training over approximately 1.27 trillion tokens. The annealed checkpoint gained 12.9 points on the Indic MILU benchmark but lost 3.4 on English MMLU. We also report practical negative results. Direct token addition underperformed continued byte-pair encoding. AdamW weight decay changed rows intended to stay fixed. Perplexity on an Indic-heavy holdout did not select the learning rate that best preserved English performance.
PDF arXiv
arXiv cs.CLOct 11
A Sovereign, Indic-First, Reasoning Model
Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik +2
Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA's Nemotron-3-Nano-30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved…
PDF arXiv
arXiv cs.LGOct 11
Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
Heng-Sheng Chang
A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations can catch deceptive responses that look honest. Yet current detectors depend on which layers they read, weaken on broader tests, and do not state how certain their alarms are. This white paper proposes the Machine Deception Detector, which targets both detection accuracy and the uncertainty of detector scores. The detector reads every layer of the language model, learns which tokens carry evidence of deception using only response labels, and calibrates its alarm threshold so that its false--positive rate stays below a prescribed level. An ensemble of such detectors measures the uncertainty of each score and raises an alarm only when its members agree, with a confidence level that sets how certain each alarm must be. The proposed evaluation trains on instructed honest and deceptive statements, tests on five unseen deceptive tasks of Liars' Bench read by six Qwen3 models from 0.6B to 32B parameters, and is designed to also show which layers carry the deception feature and how this changes with model size.
PDF arXiv
arXiv cs.CLOct 11
North or South? Diagnosing LLMs' Understanding of North Korean Language, Society, and Knowledge
Kiwoong Park, Seyoung Song, Donghyun Woo, Alice Oh
Large Language Models (LLMs) have made substantial progress in multilingual question answering, yet their ability to understand low-resource sociolinguistic contexts remains under-explored. We study North Korea as a unique testbed where decades of division have produced distinct differences from South Korea in vocabulary, orthography, norms, and ideological discourse. We introduce LENS (Language and sociocultural Evaluation for North Korean Society), a human-validated benchmark of 1,765 questions evaluating LLMs across three dimensions: linguistic robustness, sociocultural perspective alignment, and domain-specific knowledge. Evaluating diverse recent LLMs, we find a modest performance drop on North Korean language inputs, a larger gap in sociocultural perspective alignment, and severe limitations on domain-specific knowledge. Qualitative analysis reveals that models often conflate formal institutions with lived realities, project liberal-democratic assumptions, or rely on coarse stereotypes. Our results demonstrate that linguistic similarity alone does not guarantee sociocultural or domain-specific understanding, highlighting the critical need for grounded evaluation resources…
PDF arXiv
arXiv cs.CLOct 11
Do correct answers have stable latent knowledge support?
Chuancheng Shi, Shangze Li, Wenhua Wu, Kun Hu, Fei Shen +2
Large language models are typically evaluated in terms of the correctness of their output. However, output correctness alone does not confirm whether the latent knowledge supporting an answer has stable internal representations. We therefore propose latent state stability (LSS) and knowledge topology stability (KTS) to investigate the stability of individual knowledge states and their relational structure, respectively. Specifically, we first separately average the model's internal representations of the same knowledge under each elicitation condition. Using the mean representation under the baseline condition as a reference, we remove shared shifts to obtain corrected representations. We then formulate LSS as the distance between each corrected representation and its reference representation, normalized against the typical distance between knowledge items of the same type. We formulate KTS as the cross-condition consistency of the ordered pairwise representation distances among knowledge items of the same type. Next, we construct EpiStableBench, which comprises 4,411 items spanning factual knowledge and mathematical reasoning, with 76,766 controlled queries. Across 16 LLMs from…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 35m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

429papers this week−16.2% vs last week

All papers · 2.4k With code · 62

Newest Most relevant

2.2k papers this month

Open problems in Frontier LLMs

Efficient and accurate fact-checking of long, multi-claim texts1 paper raises it · 1 addresses it
Frontier LLMs are costly and slow for verifying complex review comments, while cheaper models fail on implicit or multi-part assertions.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Adaptive depth for decision models without token generation2 papers raise it · 3 address it
Fixed-depth decision models waste compute on easy questions, while generative reasoning is slow and token-costly.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces or filler text.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions, collapsing performance.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation for context-induced gains1 paper raises it · 1 addresses it
LLMs lose context-provided knowledge after context removal, limiting long-term retention of learned information.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses

Named by an AI from these papers' abstracts.