Papers

Field

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno4 releases this week
FastMCP6 releases this week
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%

Top tools here

Open problems in Frontier LLMs

Structural requirements for decision-native models1 paper raises it · 1 addresses it
Current decision models often ignore key properties like null-awareness, permutation invariance, and option scalability, limiting their reliability and efficiency.
- Jeva: System One Models from First Principlesraises · addresses
Efficient and calibrated decision-making for search agents1 paper raises it · 1 addresses it
Generative models introduce latency and unreliable confidence in short, repeated decisions, hindering real-time search agent performance.
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-training on generated questions can collapse performance due to invalid or repetitive questions, limiting long-term improvement.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation in context-conditioned models1 paper raises it · 1 addresses it
Gains from context are often lost after its removal, reducing the long-term utility of contextual learning.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Detecting and mitigating scientific slop in AI-generated papers1 paper raises it · 1 addresses it
AI-generated scientific papers can appear plausible but contain flawed reasoning, misleading readers and degrading research quality.
- Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papersraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Recently funded

Topics in this field

reasoning traces · reasoning models · latent reasoning381 papers62 this week
dense retrieval · retrieval augmented generation · retrieval256 papers50 this week
attention heads · large language models · language models231 papers49 this week
large language models · attribution · language models llms203 papers40 this week
question answering · answering · large language models140 papers40 this week
large language models · large language · language models184 papers36 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.AIOct 16
Sep-Latent: Separator Tokens as Natural Compression Anchors for Implicit Reasoning
Yinlong Xu, Jianfeng Huang, Zhengdong Zhou, Mingjie Lin, Fuyu Lv +3
Large Language Models (LLMs) have demonstrated remarkable success in complex reasoning through explicit Chain-of-Thought (CoT), but the associated computational overhead and token verbosity remain significant barriers to efficiency. While implicit reasoning in continuous latent spaces offers a promising alternative, existing methods like Coconut and CODI often rely on fixed-length latent sequences and suffer from representational collapse. In this paper, we propose Sep-Latent, an adaptive implicit reasoning framework that utilizes separator tokens as structural semantic anchors. By introducing an Attention-Decay mechanism, we force the model to aggregate and compress reasoning logic into these anchors during training. We then align explicit and implicit pathways through multi-level distillation across all layers to ensure logical consistency. Crucially, Sep-Latent enables dynamic reasoning depth by adaptively terminating the latent thinking phase based on query complexity. Experimental results on the GSM8k-Aug benchmark show that Sep-Latent achieves an accuracy of 56.8\% with an average of only 5.4 tokens, significantly outperforming prior fixed-length implicit models in…
PDF arXiv
arXiv cs.CLOct 16
Where's the Indirect Evidence? Investigating Grammatical Generalization with Influence Functions
William Dai, Zhenghao Herbert Zhou, Maya Viswanathan, Robert Frank
Language models generalize beyond their training corpus to learn linguistic phenomena (Patil et al., 2024; Misra and Mahowald, 2024), showing that there exists indirect evidence driving learning. Prior work has inferred that indirect evidence exists without identifying its character or considered specific hypotheses about what it might be. We explore an alternative bottom-up method for identifying indirect evidence. We use influence functions (Koh and Liang, 2017) to measure how individual training sentences affect model predictions in 15 grammatical phenomena from BLiMP (Warstadt et al., 2020). We filter out a phenomenon's direct evidence using hand-constructed rules and take the 60,000 most influential remaining sentences as indirect evidence. This indirect evidence substantially affects learning: retraining without it lowers held-out accuracy more than removing a matched random group for every phenomenon across five models (by a median of 12 points). We characterize the indirect evidence by its similarity to our direct evidence filters: it (partially) meets the filters' structural conditions at a median of 4.7 times the rate of a matched random group. Finally, clustering…
PDF arXiv
arXiv cs.CLOct 16
FederatedBytes: Tokenizer-Free, Byte-Level Small Language Models for Efficient Multimodal Federated Learning
Ali Sarabadani, Massimo Piccardi, Mehran Abolhasan, Farzad Tofigh, Ian Zhou +1
Federated Learning (FL) is a powerful machine learning framework that allows building a global model from the contributions of multiple, distributed clients, while ensuring the privacy of their local data. However, its heavy computational and communication overheads challenge its use for contemporary AI workloads such as natural language processing (NLP) and multimodal analysis. For this reason, this paper presents a novel small language model that leverages byte-level text and multimodal data processing, enhanced by a linear attention mechanism, to enable manageable Federated Learning of NLP and multimodal analysis tasks. The proposed model, nicknamed FederatedBytes, manages to process text and any other data types, including images, by treating its input as a mere sequence of bytes, while its lean linear attention mechanism allows abating the overall memory and computational footprint. To prove the effectiveness of the proposed model we have carried out an extensive performance evaluation on sentiment analysis tasks in multilingual and multimodal settings over the Multilingual Amazon Reviews and ScienceQA datasets. The experimental results show that FederatedBytes has been able…
PDF arXiv
arXiv cs.CLOct 16▲ 0
Competence-Based Evaluation of Logical Reasoning in Large Language Models
Jize Jiang, Chengxiang Zhai
Recent advances in large language models (LLMs) have brought impressive performance across diverse tasks; however, shortcomings remain in logical reasoning, including inconsistencies under linguistic variation. To improve the reasoning capacity of LLMs, we must be able to accurately measure their reasoning capacity. Current benchmarks primarily report single accuracy numbers on selected reasoning tasks. Such a performance-based evaluation methodology says little about a model's generalization capacity in reasoning. To address this limitation, we argue for a different evaluation paradigm, called competence-based evaluation (CBE), which tests whether models reliably apply the same competence across a family of problems. A parameterized generator scales complexity while keeping the required competence fixed, and matched answer-preserving presentations test correctness beyond one formulation. We instantiate CBE on elementary ordering tasks, scaling input size while varying presentations of the same ordering rule. Across six GPT and Qwen models, performance is reasonably good over bounded complexity ranges, but even strong models fail the competence condition across the tested family.…
PDF arXiv
arXiv cs.CLOct 16
Bengali Hallucination Detection with Retrieval and Programs: A System Study
Shamim Hasan, Samiya Ahmed Chowdhury
Bengali hallucination detection must distinguish unsupported factual claims from incorrect reasoning, often without a supplied reference passage. We present and characterize an inference-only system that combines response-conditioned query rewriting, hybrid retrieval, and task-specific language-model judging. Six retrieval routes search question-answer and passage collections; a router directs each example to a 27B textual judge or a 14B agent that can compare retrieved answers, reason directly, or generate and execute Python. We characterize the system using version-specific code, saved predictions, execution traces, and corpus metadata from the Olikbochon challenge. Kaggle associates a displayed private macro-F1 of 0.956 with Version 2; its saved run processes 2,516 examples in 220.99 minutes on two T4 GPUs. Of 253 math-routed examples, 88 terminate through candidate matching, while execution traces expose 49 stderr-bearing run events across 35 examples. A later run changes six predictions despite identical core worker code. A bounded audit of 1,090 benchmark-related corpus records finds exact question or context overlap for 38 processed evaluation inputs, precluding an…
PDF arXiv
arXiv cs.CLOct 16
NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data
Lijia Liu, Kyohei Atarashi, Jiyi Li, Koh Takeuchi, Shunji Umetani +1
Formulating optimization problems from raw inputs of problem descriptions and data is time-consuming and costly. While Large Language Models (LLMs) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete. This diverges from reality, where data-centric reasoning is required to extract, compute, and predict parameters from noisy sources. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi-stage reasoning inherent in practical optimization workflows. NLData2Opt decomposes problems into problem descriptions and data, and requires models to (1) extract relevant parameters from data containing misleading attributes, (2) compute missing parameters derived by transforming existing data attributes, and (3) predict unavailable future values from historical observations. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows. NLData2Opt is available at https://github.com/sumarivyn/NLData2Opt.
PDF arXiv

Papers

Top tools here

Coming with more sources: stars history

All tools

NVIDIA Dynamo+60% downloads · 15.8k/wk+60.0%
Agno4 releases this week
FastMCP6 releases this week
BentoML+30% downloads · 41k/wk+29.7%
n8n+25% downloads · 148.3k/wk+25.0%

Topics in this field

reasoning traces · reasoning models · latent reasoning381 papers62 this week
dense retrieval · retrieval augmented generation · retrieval256 papers50 this week
attention heads · large language models · language models231 papers49 this week
large language models · attribution · language models llms203 papers40 this week
question answering · answering · large language models140 papers40 this week
large language models · large language · language models184 papers36 this week

Recently funded

Market

InfinigenceSeries B+ · Oct 12$343.6M
Lean SuperIntelligencePre-seed · Oct 9undisclosed
VinciSeries B+ · Oct 6$250M
Siena AISeries A · Oct 6$17M
Ocular AIPre-seed · Oct 5$2M

Active contributors · 90 days

Activity · 12 weeks

Coming with more sources: repos

Recent papers

All in Research →

arXiv cs.AIOct 16
Sep-Latent: Separator Tokens as Natural Compression Anchors for Implicit Reasoning
Yinlong Xu, Jianfeng Huang, Zhengdong Zhou, Mingjie Lin, Fuyu Lv +3
Large Language Models (LLMs) have demonstrated remarkable success in complex reasoning through explicit Chain-of-Thought (CoT), but the associated computational overhead and token verbosity remain significant barriers to efficiency. While implicit reasoning in continuous latent spaces offers a promising alternative, existing methods like Coconut and CODI often rely on fixed-length latent sequences and suffer from representational collapse. In this paper, we propose Sep-Latent, an adaptive implicit reasoning framework that utilizes separator tokens as structural semantic anchors. By introducing an Attention-Decay mechanism, we force the model to aggregate and compress reasoning logic into these anchors during training. We then align explicit and implicit pathways through multi-level distillation across all layers to ensure logical consistency. Crucially, Sep-Latent enables dynamic reasoning depth by adaptively terminating the latent thinking phase based on query complexity. Experimental results on the GSM8k-Aug benchmark show that Sep-Latent achieves an accuracy of 56.8\% with an average of only 5.4 tokens, significantly outperforming prior fixed-length implicit models in…
PDF arXiv
arXiv cs.CLOct 16
Where's the Indirect Evidence? Investigating Grammatical Generalization with Influence Functions
William Dai, Zhenghao Herbert Zhou, Maya Viswanathan, Robert Frank
Language models generalize beyond their training corpus to learn linguistic phenomena (Patil et al., 2024; Misra and Mahowald, 2024), showing that there exists indirect evidence driving learning. Prior work has inferred that indirect evidence exists without identifying its character or considered specific hypotheses about what it might be. We explore an alternative bottom-up method for identifying indirect evidence. We use influence functions (Koh and Liang, 2017) to measure how individual training sentences affect model predictions in 15 grammatical phenomena from BLiMP (Warstadt et al., 2020). We filter out a phenomenon's direct evidence using hand-constructed rules and take the 60,000 most influential remaining sentences as indirect evidence. This indirect evidence substantially affects learning: retraining without it lowers held-out accuracy more than removing a matched random group for every phenomenon across five models (by a median of 12 points). We characterize the indirect evidence by its similarity to our direct evidence filters: it (partially) meets the filters' structural conditions at a median of 4.7 times the rate of a matched random group. Finally, clustering…
PDF arXiv
arXiv cs.CLOct 16
FederatedBytes: Tokenizer-Free, Byte-Level Small Language Models for Efficient Multimodal Federated Learning
Ali Sarabadani, Massimo Piccardi, Mehran Abolhasan, Farzad Tofigh, Ian Zhou +1
Federated Learning (FL) is a powerful machine learning framework that allows building a global model from the contributions of multiple, distributed clients, while ensuring the privacy of their local data. However, its heavy computational and communication overheads challenge its use for contemporary AI workloads such as natural language processing (NLP) and multimodal analysis. For this reason, this paper presents a novel small language model that leverages byte-level text and multimodal data processing, enhanced by a linear attention mechanism, to enable manageable Federated Learning of NLP and multimodal analysis tasks. The proposed model, nicknamed FederatedBytes, manages to process text and any other data types, including images, by treating its input as a mere sequence of bytes, while its lean linear attention mechanism allows abating the overall memory and computational footprint. To prove the effectiveness of the proposed model we have carried out an extensive performance evaluation on sentiment analysis tasks in multilingual and multimodal settings over the Multilingual Amazon Reviews and ScienceQA datasets. The experimental results show that FederatedBytes has been able…
PDF arXiv
arXiv cs.CLOct 16▲ 0
Competence-Based Evaluation of Logical Reasoning in Large Language Models
Jize Jiang, Chengxiang Zhai
Recent advances in large language models (LLMs) have brought impressive performance across diverse tasks; however, shortcomings remain in logical reasoning, including inconsistencies under linguistic variation. To improve the reasoning capacity of LLMs, we must be able to accurately measure their reasoning capacity. Current benchmarks primarily report single accuracy numbers on selected reasoning tasks. Such a performance-based evaluation methodology says little about a model's generalization capacity in reasoning. To address this limitation, we argue for a different evaluation paradigm, called competence-based evaluation (CBE), which tests whether models reliably apply the same competence across a family of problems. A parameterized generator scales complexity while keeping the required competence fixed, and matched answer-preserving presentations test correctness beyond one formulation. We instantiate CBE on elementary ordering tasks, scaling input size while varying presentations of the same ordering rule. Across six GPT and Qwen models, performance is reasonably good over bounded complexity ranges, but even strong models fail the competence condition across the tested family.…
PDF arXiv
arXiv cs.CLOct 16
Bengali Hallucination Detection with Retrieval and Programs: A System Study
Shamim Hasan, Samiya Ahmed Chowdhury
Bengali hallucination detection must distinguish unsupported factual claims from incorrect reasoning, often without a supplied reference passage. We present and characterize an inference-only system that combines response-conditioned query rewriting, hybrid retrieval, and task-specific language-model judging. Six retrieval routes search question-answer and passage collections; a router directs each example to a 27B textual judge or a 14B agent that can compare retrieved answers, reason directly, or generate and execute Python. We characterize the system using version-specific code, saved predictions, execution traces, and corpus metadata from the Olikbochon challenge. Kaggle associates a displayed private macro-F1 of 0.956 with Version 2; its saved run processes 2,516 examples in 220.99 minutes on two T4 GPUs. Of 253 math-routed examples, 88 terminate through candidate matching, while execution traces expose 49 stderr-bearing run events across 35 examples. A later run changes six predictions despite identical core worker code. A bounded audit of 1,090 benchmark-related corpus records finds exact question or context overlap for 38 processed evaluation inputs, precluding an…
PDF arXiv
arXiv cs.CLOct 16
NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data
Lijia Liu, Kyohei Atarashi, Jiyi Li, Koh Takeuchi, Shunji Umetani +1
Formulating optimization problems from raw inputs of problem descriptions and data is time-consuming and costly. While Large Language Models (LLMs) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete. This diverges from reality, where data-centric reasoning is required to extract, compute, and predict parameters from noisy sources. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi-stage reasoning inherent in practical optimization workflows. NLData2Opt decomposes problems into problem descriptions and data, and requires models to (1) extract relevant parameters from data containing misleading attributes, (2) compute missing parameters derived by transforming existing data attributes, and (3) predict unavailable future values from historical observations. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows. NLData2Opt is available at https://github.com/sumarivyn/NLData2Opt.
PDF arXiv

Open problems in Frontier LLMs

Structural requirements for decision-native models1 paper raises it · 1 addresses it
Current decision models often ignore key properties like null-awareness, permutation invariance, and option scalability, limiting their reliability and efficiency.
- Jeva: System One Models from First Principlesraises · addresses
Efficient and calibrated decision-making for search agents1 paper raises it · 1 addresses it
Generative models introduce latency and unreliable confidence in short, repeated decisions, hindering real-time search agent performance.
- SearchJev: A Fast and Calibrated System-1 Model for Search Agentsraises · addresses
Sustaining self-evolution in reasoning models1 paper raises it · 1 addresses it
Self-training on generated questions can collapse performance due to invalid or repetitive questions, limiting long-term improvement.
- Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelsraises · addresses
Memory consolidation in context-conditioned models1 paper raises it · 1 addresses it
Gains from context are often lost after its removal, reducing the long-term utility of contextual learning.
- Cortex: Consolidating Context into Parametric Memoryraises · addresses
Detecting and mitigating scientific slop in AI-generated papers1 paper raises it · 1 addresses it
AI-generated scientific papers can appear plausible but contain flawed reasoning, misleading readers and degrading research quality.
- Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papersraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 58m ago

Frontier LLMs

The largest general-purpose language models, their releases, capabilities, scaling and training at frontier scale.

431papers this week−11.9% vs last week

All papers · 2.3k With code · 54

Newest Most relevant

2.1k papers this month