Papers

Field

Top tools here

All tools

NVIDIA Dynamo+62% downloads · 16k/wk+62.3%
Agno5 releases this week
FastMCP6 releases this week
BentoML+40% downloads · 44.6k/wk+39.6%
Pydantic AI+17% downloads · 1.5M/wk+17.0%

Top tools here

Open problems in Frontier LLMs

Efficient and accurate fact-checking of long, multi-claim texts1 paper raises it · 1 addresses it
Frontier LLMs are slow and expensive for verifying complex, multi-premise statements in reviews or papers.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Adaptive reasoning depth for decision models2 papers raise it · 2 address it
Fixed-depth decision models waste computation on easy questions and underperform on hard ones.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M
InfinigenceSeries B+ · Oct 12$343.6M
Phinity LabsSeed · Oct 12$5.2M
Lean SuperIntelligencePre-seed · Oct 9undisclosed

Recently funded

Topics in this field

reasoning traces · reasoning models · latent reasoning395 papers68 this week
dense retrieval · retrieval augmented generation · retrieval267 papers58 this week
attention heads · large language models · language models234 papers46 this week
question answering · answering · large language models147 papers46 this week
large language models · attribution · language models llms210 papers44 this week
large language models · large language · language models189 papers40 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.CLOct 12
Where Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference
Yuchen Li, Xintong Li, Zongqi Fan, Ken-tye Yong, Nguyen H. Tran
Does a large language model have a self, and if so, of what kind? We separate two functional levels familiar from biology: a self-index, which distinguishes one's own material from external material and shapes processing, as corollary discharge does in nearly all animals, and self-reference, the higher ability to recognize what is one's own, as in mirror self-recognition. We probe both in Qwen3.8-27B by placing the same text in an Assistant, a User, or a System message rendered by the model's own chat template, and report three levels of evidence. Persistence: as predicted in advance, the representation of the text separates the model's own message from both external sources at all 64 layers and for every test text, along a shared direction, even when the answer does not depend on who supplied the text. Coupling: asked who wrote the preceding message, the model claims its own messages (0.77) and assigns User and System messages alike to the user, and how strongly it claims each message is predicted by the size of its self-index (Spearman correlation up to 0.68). Intervention: exchanging the internal states that follow a message, rather than those of its text, reverses the…
PDF arXiv
arXiv cs.LGOct 12
Attention-Graph Curvature for the Verification of Mathematical Reasoning
Kanghyeon Kim, Johannes F. Lutzeyer, Yanzhu Guo
Language models have recently solved open Erdős problems and achieved gold-medal level performance on International Mathematical Olympiad problems. But their solutions, written in natural language, cannot yet be verified by proof assistants. An erroneous reasoning step can invalidate every step that comes after, so it is advantageous to detect errors early. Several methods have been proposed to detect erroneous reasoning quickly based on the features of the forward pass of the language model that performs reasoning. In this work, we study whether the directed Forman curvature of that model's attention graph, a local quantity computed at an edge from the attention weights around it, carries information about a step's correctness. To do so, we evaluate four per-step verifiers on ProcessBench with and without the curvature appended to their input. Appending curvature values improves the overall mean performance of all four verifiers. ReProbe, the verifier that takes every token of a step as input, shows the largest improvement. Adding the curvature increases the time of the language model's forward pass by 18% on average.
PDF arXiv
arXiv cs.CLOct 12
EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI
Ruijia Yang, Bing Xu, Zulong Chen, Zhipeng Xu, Jia Xu +3
Recruitment applications span diverse operations, from drafting job descriptions and interpreting queries to organizing profiles, assessing fit, and recommending candidates and jobs. Systematic LLM evaluation therefore requires task coverage that reflects this range of business needs. We introduce EHR-Bench, an open Chinese benchmark comprising twelve tasks across six capability families and five recruitment scenarios. The tasks cover generation, extraction, classification, retrieval, semantic similarity, and reciprocal ranking within one evaluation framework. An internal recruitment dataset guides task definitions, the occupation taxonomy and sampling targets, output requirements, and scoring protocols. Following these specifications, we construct 8,336 task instances from public materials and controlled augmentation, with source-aware expert review and documented provenance. We evaluate twelve LLMs through task-level, capability-level, and aggregate comparisons. Closely ranked models exhibit distinct task strengths, and performance varies across input sources, reference evidence, and candidate pools. Five shared models retain the same observed Overall ordering on a private…
PDF arXiv
arXiv cs.AIOct 12
Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev
Dushyant Rajput, Nirdesh Chauhan, Siddharth Kosaraju
Decision APIs that return a probability for each option are routinely used as if those probabilities were properties of the options themselves. That reading is valid only if the model satisfies independence of irrelevant alternatives (IIA): adding an option that is neither the right answer nor its strongest rival should leave the odds between those two unchanged. We test this in a pre-registered study of jev-1.13.0, a hosted decision model, using 299,463 calls on intent classification (BANKING77, CLINC150) and topic classification (AG News). Every added option moves the log-odds between the gold label and the strongest wrong label: the root-mean-square systematic shift is 0.20-0.52 nats, 1.2 to 3 times the run-to-run variation, even when the added option takes at most 0.1% of the probability mass. The shifts are directional. An option similar to the wrong answer draws probability from it (+0.10 nats toward the gold label). A paraphrase of the wrong answer produces an intermediate pattern: the gold label's probability is almost conserved, but the paraphrase receives only about a third of the wrong answer's share rather than the half that strict substitution implies. The distortion…
PDF arXiv
arXiv cs.CLOct 12
SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation
Maria Carmen Staiano, Lifeng Han, Johanna Monti, Francesca Chiusaroli
Machine translation (MT) is increasingly utilized in multilingual crisis communication, yet conventional evaluation does not directly assess whether translated emergency messages preserve their actionability. We introduce SIREN (Situation-aware Index for Response and Emergency ActioNability), a human evaluation framework assessing whether translation choices compromise protective action, directive force, and urgency of the source text. We evaluate SIREN in two complementary settings. At the error level, we apply it to HOPE-annotated Italian--English translations produced by ChatGPT-4o and Google Translate: 31.63\% and 48.73\% of errors, respectively, compromise actionability, predominantly by affecting the protective action. At the message level, we evaluate English--French translations produced by TranslateGemma-4B and Google Translate, with actionability preserved in 85.34\% and 92.67\% of translations, respectively. TranslateGemma-4B failures predominantly weaken directive force, whereas Google Translate failures more often compromise protective actions and receive Major impact ratings. These findings show that translation systems can differ not only in how often actionability…
PDF arXiv
arXiv cs.HCOct 12
Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations
Nicola Rossberg, Bennett Kleinberg, Barry O'Sullivan, Luca Longo, Andrea Visentin
Existing human-centred XAI research predominantly evaluates explanations after they have been generated. However, this fails to provide actionable insights into improving accessibility. This study incorporates human-based feature understandability into feature selection to improve the generation of human-centred explanations. Natural language explanations for machine learning models trained on tabular data often cite influential features in the input data to explain a given decision. As a consequence, users' ability to understand the produced explanation hinges on their understanding of the cited features. To quantify this natural-language feature understandability, Rossberg et al. proposed the Feature Understandability Scale, which assigns numerical `understandability scores' to features based on user feedback. To evaluate the degree to which these scores can improve machine learning explanation quality, this study collects understandability scores across two datasets, proposes an understandability-aware feature selection algorithm to select features that are both understandable and informative, and trains a random forest based on these feature subsets. The SHAP-based…
PDF arXiv

Papers

Top tools here

All tools

NVIDIA Dynamo+62% downloads · 16k/wk+62.3%
Agno5 releases this week
FastMCP6 releases this week
BentoML+40% downloads · 44.6k/wk+39.6%
Pydantic AI+17% downloads · 1.5M/wk+17.0%

Topics in this field

reasoning traces · reasoning models · latent reasoning395 papers68 this week
dense retrieval · retrieval augmented generation · retrieval267 papers58 this week
attention heads · large language models · language models234 papers46 this week
question answering · answering · large language models147 papers46 this week
large language models · attribution · language models llms210 papers44 this week
large language models · large language · language models189 papers40 this week

Recently funded

Market

Electric PlantSeed · Oct 20$11.2M
PennantSeed · Oct 20$5M
InfinigenceSeries B+ · Oct 12$343.6M
Phinity LabsSeed · Oct 12$5.2M
Lean SuperIntelligencePre-seed · Oct 9undisclosed

Active contributors · 90 days

Activity · 12 weeks

Recent papers

All papers in Research

arXiv cs.CLOct 12
Where Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference
Yuchen Li, Xintong Li, Zongqi Fan, Ken-tye Yong, Nguyen H. Tran
Does a large language model have a self, and if so, of what kind? We separate two functional levels familiar from biology: a self-index, which distinguishes one's own material from external material and shapes processing, as corollary discharge does in nearly all animals, and self-reference, the higher ability to recognize what is one's own, as in mirror self-recognition. We probe both in Qwen3.8-27B by placing the same text in an Assistant, a User, or a System message rendered by the model's own chat template, and report three levels of evidence. Persistence: as predicted in advance, the representation of the text separates the model's own message from both external sources at all 64 layers and for every test text, along a shared direction, even when the answer does not depend on who supplied the text. Coupling: asked who wrote the preceding message, the model claims its own messages (0.77) and assigns User and System messages alike to the user, and how strongly it claims each message is predicted by the size of its self-index (Spearman correlation up to 0.68). Intervention: exchanging the internal states that follow a message, rather than those of its text, reverses the…
PDF arXiv
arXiv cs.LGOct 12
Attention-Graph Curvature for the Verification of Mathematical Reasoning
Kanghyeon Kim, Johannes F. Lutzeyer, Yanzhu Guo
Language models have recently solved open Erdős problems and achieved gold-medal level performance on International Mathematical Olympiad problems. But their solutions, written in natural language, cannot yet be verified by proof assistants. An erroneous reasoning step can invalidate every step that comes after, so it is advantageous to detect errors early. Several methods have been proposed to detect erroneous reasoning quickly based on the features of the forward pass of the language model that performs reasoning. In this work, we study whether the directed Forman curvature of that model's attention graph, a local quantity computed at an edge from the attention weights around it, carries information about a step's correctness. To do so, we evaluate four per-step verifiers on ProcessBench with and without the curvature appended to their input. Appending curvature values improves the overall mean performance of all four verifiers. ReProbe, the verifier that takes every token of a step as input, shows the largest improvement. Adding the curvature increases the time of the language model's forward pass by 18% on average.
PDF arXiv
arXiv cs.CLOct 12
EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI
Ruijia Yang, Bing Xu, Zulong Chen, Zhipeng Xu, Jia Xu +3
Recruitment applications span diverse operations, from drafting job descriptions and interpreting queries to organizing profiles, assessing fit, and recommending candidates and jobs. Systematic LLM evaluation therefore requires task coverage that reflects this range of business needs. We introduce EHR-Bench, an open Chinese benchmark comprising twelve tasks across six capability families and five recruitment scenarios. The tasks cover generation, extraction, classification, retrieval, semantic similarity, and reciprocal ranking within one evaluation framework. An internal recruitment dataset guides task definitions, the occupation taxonomy and sampling targets, output requirements, and scoring protocols. Following these specifications, we construct 8,336 task instances from public materials and controlled augmentation, with source-aware expert review and documented provenance. We evaluate twelve LLMs through task-level, capability-level, and aggregate comparisons. Closely ranked models exhibit distinct task strengths, and performance varies across input sources, reference evidence, and candidate pools. Five shared models retain the same observed Overall ordering on a private…
PDF arXiv
arXiv cs.AIOct 12
Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev
Dushyant Rajput, Nirdesh Chauhan, Siddharth Kosaraju
Decision APIs that return a probability for each option are routinely used as if those probabilities were properties of the options themselves. That reading is valid only if the model satisfies independence of irrelevant alternatives (IIA): adding an option that is neither the right answer nor its strongest rival should leave the odds between those two unchanged. We test this in a pre-registered study of jev-1.13.0, a hosted decision model, using 299,463 calls on intent classification (BANKING77, CLINC150) and topic classification (AG News). Every added option moves the log-odds between the gold label and the strongest wrong label: the root-mean-square systematic shift is 0.20-0.52 nats, 1.2 to 3 times the run-to-run variation, even when the added option takes at most 0.1% of the probability mass. The shifts are directional. An option similar to the wrong answer draws probability from it (+0.10 nats toward the gold label). A paraphrase of the wrong answer produces an intermediate pattern: the gold label's probability is almost conserved, but the paraphrase receives only about a third of the wrong answer's share rather than the half that strict substitution implies. The distortion…
PDF arXiv
arXiv cs.CLOct 12
SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation
Maria Carmen Staiano, Lifeng Han, Johanna Monti, Francesca Chiusaroli
Machine translation (MT) is increasingly utilized in multilingual crisis communication, yet conventional evaluation does not directly assess whether translated emergency messages preserve their actionability. We introduce SIREN (Situation-aware Index for Response and Emergency ActioNability), a human evaluation framework assessing whether translation choices compromise protective action, directive force, and urgency of the source text. We evaluate SIREN in two complementary settings. At the error level, we apply it to HOPE-annotated Italian--English translations produced by ChatGPT-4o and Google Translate: 31.63\% and 48.73\% of errors, respectively, compromise actionability, predominantly by affecting the protective action. At the message level, we evaluate English--French translations produced by TranslateGemma-4B and Google Translate, with actionability preserved in 85.34\% and 92.67\% of translations, respectively. TranslateGemma-4B failures predominantly weaken directive force, whereas Google Translate failures more often compromise protective actions and receive Major impact ratings. These findings show that translation systems can differ not only in how often actionability…
PDF arXiv
arXiv cs.HCOct 12
Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations
Nicola Rossberg, Bennett Kleinberg, Barry O'Sullivan, Luca Longo, Andrea Visentin
Existing human-centred XAI research predominantly evaluates explanations after they have been generated. However, this fails to provide actionable insights into improving accessibility. This study incorporates human-based feature understandability into feature selection to improve the generation of human-centred explanations. Natural language explanations for machine learning models trained on tabular data often cite influential features in the input data to explain a given decision. As a consequence, users' ability to understand the produced explanation hinges on their understanding of the cited features. To quantify this natural-language feature understandability, Rossberg et al. proposed the Feature Understandability Scale, which assigns numerical `understandability scores' to features based on user feedback. To evaluate the degree to which these scores can improve machine learning explanation quality, this study collects understandability scores across two datasets, proposes an understandability-aware feature selection algorithm to select features that are both understandable and informative, and trains a random forest based on these feature subsets. The SHAP-based…
PDF arXiv

Open problems in Frontier LLMs

Efficient and accurate fact-checking of long, multi-claim texts1 paper raises it · 1 addresses it
Frontier LLMs are slow and expensive for verifying complex, multi-premise statements in reviews or papers.
- PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviewsraises · addresses
Adaptive reasoning depth for decision models2 papers raise it · 2 address it
Fixed-depth decision models waste computation on easy questions and underperform on hard ones.
- LEV: Thinking Fast and Deep with Looped Decision Modelsraises · addresses
- Jeva: System One Models from First Principlesraises · addresses
Structural requirements for decision-native models1 paper raises it · 3 address it
Current decision models lack null-awareness, permutation invariance, option scalability, and cacheability.
- Jeva: System One Models from First Principlesraises · addresses
- LEV: Thinking Fast and Deep with Looped Decision Modelsaddresses
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsaddresses
Reliable no-thinking compliance in LLMs1 paper raises it · 1 addresses it
LLMs often ignore or misinterpret no-thinking instructions, leading to unwanted reasoning traces.
- Thinking Inertia: LLMs Keep Thinking When Told Not Toraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-generated training data degrades over rounds due to invalid or repetitive questions.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 26m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

475papers this week−4.0% vs last week

All papers · 2.4k With code · 59

Newest Most relevant

2.2k papers this month