Papers

Field

Top tools here

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno4 releases this week
FastMCP6 releases this week
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%

Top tools here

Open problems in Frontier LLMs

Structural requirements for decision-native models1 paper raises it · 1 addresses it
Current decision models often ignore key properties like null-awareness, permutation invariance, and option scalability, limiting their reliability and efficiency.
- Jeva: System One Models from First Principlesraises · addresses
Efficient and calibrated decision-making for search agents1 paper raises it · 1 addresses it
Generative models introduce latency and unreliable confidence in short, repeated decisions, hindering real-time search agent performance.
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-training on generated questions can collapse performance due to invalid or repetitive questions, limiting long-term improvement.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation in context-conditioned models1 paper raises it · 1 addresses it
Gains from context are often lost after its removal, reducing the long-term utility of contextual learning.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Detecting and mitigating scientific slop in AI-generated papers1 paper raises it · 1 addresses it
AI-generated scientific papers can appear plausible but contain flawed reasoning, misleading readers and degrading research quality.
- Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papersraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Recently funded

Topics in this field

reasoning traces · reasoning models · latent reasoning381 papers57 this week
dense retrieval · retrieval augmented generation · retrieval256 papers50 this week
attention heads · large language models · language models231 papers48 this week
question answering · answering · large language models140 papers40 this week
large language models · attribution · language models llms203 papers38 this week
large language models · large language · language models184 papers35 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CLOct 14
OpenTWBench (Open Taiwan Benchmark): Taiwan-Domain Knowledge Evaluation for Large Language Models, and How the Measurement Apparatus Shapes the Score
Liang-Hsun Huang, Min-Yi Chen, Chao-Chun Chuang, Yu-Wei Liu, Che-Yu Wu +1
We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench contains 255,854 four-option questions across 148 academic subjects, from fifteen years of Taiwan's national examinations published as open data by the Ministry of Examination; answer keys agree across 31,312 duplicated question groups at 99.91%, with all 27 disagreements accounted for. A reviewed second pass withdrew 2,180 items whose stems depend on a figure the text does not carry; all scores use the remaining 253,674. tw-formosa-bench contains 1,696 difficulty-curated questions about everyday and contemporary Taiwan - the Taiwan that licensing examinations never ask about - generated from a community-curated corpus under a discipline in which no model writes a ground truth, and curated against a frozen seven-model pool so that the pool scored at the passing margin (46.9-74.6%, median ~54%); a leave-one-out experiment shows the selection preserves model ranking (near-zero bias). Within six weeks the frontier had cleared that band: of 108 models now on the public board, 22 score above its ceiling, the best 91.6%. We also document how…
PDF arXiv
arXiv cs.LGOct 14
MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation
Minsuh Joo, Youna Kim, Choonghyun Park, Soo Kyung Kim, Hyunsoo Cho
LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate accuracy, which obscures diverse effects by collapsing recovered failures and disrupted successes into a single score. We introduce MAPLE, a unified framework for characterizing how KEE techniques affect parametric knowledge extraction. Its diagnostic metrics disentangle these effects at the surface, probability, and beyond-sampling levels by measuring answer recovery and disruption, gold-token rank shifts on unrecovered failures, and recovery of answers missed by fixed-budget sampling. We evaluate six KEE techniques spanning decoding, activation steering, and prompting across three model families: OLMo2, Qwen2.5, and Llama3. Our analysis reveals distinct effects and trade-offs across techniques, highlighting the value of MAPLE for understanding…
PDF arXiv
arXiv stat.MLOct 14
Calibration and Risk-Budgeted Decisions with System One Models
Soundouss Messoudi, Youness Mansar
LLM workflows and agents rely on a set of binary decisions like allow/block an input, or discard/retain a retrieved document. When human deferral is unavailable, uncertain cases must still receive an action. We study System One decision scores in this setting by defaulting to a conservative fallback when uncertain, and controlling the false omission risk within the subset of inputs assigned the utility-maximizing action. Across six datasets covering moderation, safety, and reranking, we compare general System One models with task baselines and a local open-weights fine-tuned System One model. We show that naive baselines for threshold selection exceed the target risk in roughly half of settings with defined test risk, compared with 7.8\% for Learn Then Test (LTT), which leverages finite-sample statistical guarantees. Our results show why relying on a probability-like interface is insufficient for deployment decisions and how risk-budgeted selection trades useful action against observed error.
PDF arXiv
arXiv cs.AIOct 14
Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
Soyoung Oh, Nursulu Sagimbayeva, Nicholas Jennings, Paulina Hussein, Mehwish Alam +1
In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting or judging the agent's subsequent actions. In this work, we investigate whether explicitly connecting perception, belief, and action with rules improves ToM reasoning. We generate concise \textit{if--then} rules that express relations between perception to action and use Group Relative Policy Optimization (GRPO) to train Qwen2.5 and Llama3 models across four ToM benchmarks. GRPO with rule generation outperforms prompting, supervised fine-tuning, and think reasoning. Moreover, rules generated by larger models ($\geq$3B) provide transferable knowledge that improves downstream performance of GPT-4o-mini. These findings suggest that explicit rules help models bridge the reasoning gap in ToM and support knowledge transfer across models.
PDF arXiv
arXiv cs.CLOct 14
Likelihood Is Not Authorship: Can We Reliably Measure Human Contribution in AI-Assisted Writing?
Hasan Amin, Zhuoran Lu, Ming Yin
AI-text detection generally asks whether a machine wrote a text. As AI becomes an increasingly integral and accepted part of writing, the more relevant question is no longer whether AI was used, but how much of the text reflects human contribution. A prominent line of work estimates this by measuring how predictive a person's prompt is of the final text. We show that this measure can be easily manipulated. We introduce SPECIFICITY PADDING that simply asks the model to copy a few random IDs from the prompt, and that alone makes a title-only request look more human than one that supplies the article's key ideas. A context-swap audit then holds each document fixed, and only shows or hides the IDs to the evaluator. The score of the unchanged text swings by a third to half of its scale, and it rises for every document we test, on news and scientific abstracts, across varied LLMs from 3.8B to 753B parameters. The same articles can average 85\% human or 15\% human depending only on which prompt is on record, and even with no prompt at all, the output-only variant rewards the random strings themselves. An exact identity, the credit ledger, explains why. The score is a surprisal-weighted…
PDF arXiv
arXiv cs.CLOct 14
Qwen-MT-Lumen at WMT2026: Instruction-Specialized Translation via Quality-Aware Training and On-Policy Distillation
Hao Wang, Yangyang Liu, Xiaohu Zhao, Heng Liu, Zifu Shang +9
We present Lumen, The open-source model is available at, our submission to the WMT 2026 General Machine Translation Shared Task. We focus on instruction-specialized translation, where systems must produce accurate translations while satisfying fine-grained requirements on style, terminology, and structured output. Lumen follows a four-stage pipeline. First, continued pre-training and supervised fine-tuning jointly strengthen multilingual translation competence and the ability to follow translation-specific instructions, such as terminology, style, and format constraints. Second, we extend the originally offline M^{2}PO preference-optimization framework into an online preference-optimization stage: the current policy samples candidate translations, an LLM judge assigns continuous direct-assessment scores, and the resulting multi-pair preferences are used to update the model. Third, on-policy distillation transfers stronger translation-instruction-following behavior from a larger teacher model to the Qwen3-14B student, improving translation-instruction adherence while retaining a 14B primary translation backbone. Finally, structure-aware inference and postprocessing preserve…
PDF arXiv

Papers

Top tools here

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno4 releases this week
FastMCP6 releases this week
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Topics in this field

reasoning traces · reasoning models · latent reasoning381 papers57 this week
dense retrieval · retrieval augmented generation · retrieval256 papers50 this week
attention heads · large language models · language models231 papers48 this week
question answering · answering · large language models140 papers40 this week
large language models · attribution · language models llms203 papers38 this week
large language models · large language · language models184 papers35 this week

Active contributors · 90 days

Activity · 12 weeks

Open problems in Frontier LLMs

Structural requirements for decision-native models1 paper raises it · 1 addresses it
Current decision models often ignore key properties like null-awareness, permutation invariance, and option scalability, limiting their reliability and efficiency.
- Jeva: System One Models from First Principlesraises · addresses
Efficient and calibrated decision-making for search agents1 paper raises it · 1 addresses it
Generative models introduce latency and unreliable confidence in short, repeated decisions, hindering real-time search agent performance.
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-training on generated questions can collapse performance due to invalid or repetitive questions, limiting long-term improvement.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation in context-conditioned models1 paper raises it · 1 addresses it
Gains from context are often lost after its removal, reducing the long-term utility of contextual learning.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Detecting and mitigating scientific slop in AI-generated papers1 paper raises it · 1 addresses it
AI-generated scientific papers can appear plausible but contain flawed reasoning, misleading readers and degrading research quality.
- Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papersraises · addresses

Named by an AI from these papers' abstracts.

Recent papers

All papers in Research

arXiv cs.CLOct 14
OpenTWBench (Open Taiwan Benchmark): Taiwan-Domain Knowledge Evaluation for Large Language Models, and How the Measurement Apparatus Shapes the Score
Liang-Hsun Huang, Min-Yi Chen, Chao-Chun Chuang, Yu-Wei Liu, Che-Yu Wu +1
We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench contains 255,854 four-option questions across 148 academic subjects, from fifteen years of Taiwan's national examinations published as open data by the Ministry of Examination; answer keys agree across 31,312 duplicated question groups at 99.91%, with all 27 disagreements accounted for. A reviewed second pass withdrew 2,180 items whose stems depend on a figure the text does not carry; all scores use the remaining 253,674. tw-formosa-bench contains 1,696 difficulty-curated questions about everyday and contemporary Taiwan - the Taiwan that licensing examinations never ask about - generated from a community-curated corpus under a discipline in which no model writes a ground truth, and curated against a frozen seven-model pool so that the pool scored at the passing margin (46.9-74.6%, median ~54%); a leave-one-out experiment shows the selection preserves model ranking (near-zero bias). Within six weeks the frontier had cleared that band: of 108 models now on the public board, 22 score above its ceiling, the best 91.6%. We also document how…
PDF arXiv
arXiv cs.LGOct 14
MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation
Minsuh Joo, Youna Kim, Choonghyun Park, Soo Kyung Kim, Hyunsoo Cho
LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate accuracy, which obscures diverse effects by collapsing recovered failures and disrupted successes into a single score. We introduce MAPLE, a unified framework for characterizing how KEE techniques affect parametric knowledge extraction. Its diagnostic metrics disentangle these effects at the surface, probability, and beyond-sampling levels by measuring answer recovery and disruption, gold-token rank shifts on unrecovered failures, and recovery of answers missed by fixed-budget sampling. We evaluate six KEE techniques spanning decoding, activation steering, and prompting across three model families: OLMo2, Qwen2.5, and Llama3. Our analysis reveals distinct effects and trade-offs across techniques, highlighting the value of MAPLE for understanding…
PDF arXiv
arXiv stat.MLOct 14
Calibration and Risk-Budgeted Decisions with System One Models
Soundouss Messoudi, Youness Mansar
LLM workflows and agents rely on a set of binary decisions like allow/block an input, or discard/retain a retrieved document. When human deferral is unavailable, uncertain cases must still receive an action. We study System One decision scores in this setting by defaulting to a conservative fallback when uncertain, and controlling the false omission risk within the subset of inputs assigned the utility-maximizing action. Across six datasets covering moderation, safety, and reranking, we compare general System One models with task baselines and a local open-weights fine-tuned System One model. We show that naive baselines for threshold selection exceed the target risk in roughly half of settings with defined test risk, compared with 7.8\% for Learn Then Test (LTT), which leverages finite-sample statistical guarantees. Our results show why relying on a probability-like interface is insufficient for deployment decisions and how risk-budgeted selection trades useful action against observed error.
PDF arXiv
arXiv cs.AIOct 14
Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
Soyoung Oh, Nursulu Sagimbayeva, Nicholas Jennings, Paulina Hussein, Mehwish Alam +1
In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting or judging the agent's subsequent actions. In this work, we investigate whether explicitly connecting perception, belief, and action with rules improves ToM reasoning. We generate concise \textit{if--then} rules that express relations between perception to action and use Group Relative Policy Optimization (GRPO) to train Qwen2.5 and Llama3 models across four ToM benchmarks. GRPO with rule generation outperforms prompting, supervised fine-tuning, and think reasoning. Moreover, rules generated by larger models ($\geq$3B) provide transferable knowledge that improves downstream performance of GPT-4o-mini. These findings suggest that explicit rules help models bridge the reasoning gap in ToM and support knowledge transfer across models.
PDF arXiv
arXiv cs.CLOct 14
Likelihood Is Not Authorship: Can We Reliably Measure Human Contribution in AI-Assisted Writing?
Hasan Amin, Zhuoran Lu, Ming Yin
AI-text detection generally asks whether a machine wrote a text. As AI becomes an increasingly integral and accepted part of writing, the more relevant question is no longer whether AI was used, but how much of the text reflects human contribution. A prominent line of work estimates this by measuring how predictive a person's prompt is of the final text. We show that this measure can be easily manipulated. We introduce SPECIFICITY PADDING that simply asks the model to copy a few random IDs from the prompt, and that alone makes a title-only request look more human than one that supplies the article's key ideas. A context-swap audit then holds each document fixed, and only shows or hides the IDs to the evaluator. The score of the unchanged text swings by a third to half of its scale, and it rises for every document we test, on news and scientific abstracts, across varied LLMs from 3.8B to 753B parameters. The same articles can average 85\% human or 15\% human depending only on which prompt is on record, and even with no prompt at all, the output-only variant rewards the random strings themselves. An exact identity, the credit ledger, explains why. The score is a surprisal-weighted…
PDF arXiv
arXiv cs.CLOct 14
Qwen-MT-Lumen at WMT2026: Instruction-Specialized Translation via Quality-Aware Training and On-Policy Distillation
Hao Wang, Yangyang Liu, Xiaohu Zhao, Heng Liu, Zifu Shang +9
We present Lumen, The open-source model is available at, our submission to the WMT 2026 General Machine Translation Shared Task. We focus on instruction-specialized translation, where systems must produce accurate translations while satisfying fine-grained requirements on style, terminology, and structured output. Lumen follows a four-stage pipeline. First, continued pre-training and supervised fine-tuning jointly strengthen multilingual translation competence and the ability to follow translation-specific instructions, such as terminology, style, and format constraints. Second, we extend the originally offline M^{2}PO preference-optimization framework into an online preference-optimization stage: the current policy samples candidate translations, an LLM judge assigns continuous direct-assessment scores, and the resulting multi-pair preferences are used to update the model. Third, on-policy distillation transfers stronger translation-instruction-following behavior from a larger teacher model to the Qwen3-14B student, improving translation-instruction adherence while retaining a 14B primary translation backbone. Finally, structure-aware inference and postprocessing preserve…
PDF arXiv

Fields · AI · updated 18m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

418papers this week−14.0% vs last week

All papers · 2.3k With code · 54

Newest Most relevant

2.1k papers this month