TodayNewsResearchFieldsStacksMarketAnalytics
⌘K

Research

New papers ranked by attention, with their code, models and discussion linked. Pick a field on the left.

WeekMonthQuarterAll
NewestMost relevant

Papers

Open problems

Rising research topics

Ask the research

Sample
ByteTrail · AI and developer-tools intelligence
NewsResearchStacksAPI
Fields · this week
All fields3179Multimodal510Agents492Frontier LLMs421Inference & serving255Image segmentation1703D scene153Learning verifiable rewards149Graph learning142Voice AI95Classifier & causal discovery85Forecasting time76Flow matching68Evals66Teacher policy distillation48Federated learning48All 32 fields

Open problems across fields

  • General in-context learning for embodied agents1 paper raises it · 0 address it

    Enables models to adapt to new tasks and environments without retraining, improving robustness in real-world deployment.

    • Has GPT-6-Astra Achieved Embodied Intelligence?raises
  • Cost and latency in embodied control loops1 paper raises it · 0 address it

    High computational and time costs limit real-time applicability of agentic systems in dynamic environments.

    • Has GPT-6-Astra Achieved Embodied Intelligence?raises
  • Physical-state verification and failure recovery1 paper raises it · 0 address it

    Agents must reliably detect and correct errors in physical interactions to avoid irreversible failures in real-world tasks.

    • Has GPT-6-Astra Achieved Embodied Intelligence?raises
  • Long-horizon task preservation in embodied agents1 paper raises it · 0 address it

    Maintaining progress over extended sequences is critical for complex, multi-step tasks in robotics and navigation.

    • Has GPT-6-Astra Achieved Embodied Intelligence?raises
  • Open-source stacks for generalist computer-use agents1 paper raises it · 1 addresses it

    Closed systems hinder reproducibility and systematic study of agent design for real-world computer tasks.

    • OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses

Named by an AI from these papers' abstracts.

Rising research topics

  • Training & fine-tuning46+155.6%
  • AI safety32+113.3%
  • Computer vision34+100.0%
  • Robotics21+90.9%
  • Evals66+57.1%
  • Code models14+40.0%
  • Small LLMs7+40.0%
  • Federated learning48+29.7%

Papers

  • arXiv cs.CLOct 16

    CoDA: A Human-LM Collaborative Codebook Development Assistant for Domain-Specific Text Annotation

    Jaspreet Ranjit, Swaminathan Chellappa, Augusto Rivas Costante, Chandan Manjunath, Laura Onasch-Vera +3

    We present CoDA, an open-source human-LM collaborative codebook development assistant that helps domain experts develop annotation guidelines for annotating new variables in unstructured text with language models. Existing language model annotation systems primarily focus on downstream classification performance, while assuming the availability of high-quality annotation guidelines, or codebooks. In practice, developing these codebooks is time-consuming and challenging, particularly for concepts that require expert knowledge. CoDA addresses this gap by collaborating with domain experts to iteratively develop codebooks through targeted review of incorrect language model predictions and incorporating expert feedback into the evolving codebook. We demonstrate CoDA on two real-world case studies: extracting health indicators from homelessness case management notes and classifying sentiment of wildlife trafficking posts on social media. Across both domains, CoDA produces codebooks that achieve downstream classification performance comparable to expert codebooks while reducing codebook development time from several weeks to two hours and requiring 5X fewer manually annotated examples.…

    PDF
  • arXiv cs.LGOct 16

    Budgeted Bayesian Bandits for Future Utilization

    Sudipto Guha

    The multi-armed bandit problem models the trade-off between exploration and exploitation. In this paper, we study the variant where exploration has a hard budget and precedes exploitation. Prior guarantees include a 1/3 approximation by Guha and Munagala using Lagrangian duality, and a 4/27 approximation by Ma against a primal linear programming relaxation for a more general model. We show that the analyses of both algorithms are tight, even when their relaxations are asymptotically exact. We provide a different efficient algorithm that achieves a 0.43 approximation. We then provide an efficient algorithm whose guarantee approaches 1-e^{-1} approx 0.63 for bounded exchangeable reward observations in a non-vanishing regime, which is asymptotically tight against natural linear programming relaxations. The technique also gives certificates for stopping sequential model evaluation.

    PDF
  • arXiv cs.CVOct 16

    NV-Generate-Body: 3D Latent Diffusion Model forControllable Anatomy, Tumor, and Demographic Body Mask Synthesis

    Can Zhao, Pengfei Guo, Dong Yang, Yufan He, Yucheng Tang +6

    Curating large labeled CT datasets is expensive and constrained by patient privacy. We present NV-Generate-BodyMask, a 3D latent diffusion model that synthesizes whole-body CT segmentation masks with precise, joint control over organ anatomy, tumor burden, and patient demographics. The model operates in the compressed latent space of a frozen mask autoencoder, generating $256^3$ volumetric label maps conditioned on a 19-dimensional vector via Adaptive Group Normalization and cross-attention. A size-adaptive ROI loss with spatial halo weighting and a paired tumor injection pipeline enable accurate tumor size conditioning despite the scarcity of annotated tumor cases. Trained on 8,571 masks across 26 cohorts spanning 132 labels, the model achieves Spearman $ρ= 0.70$--$0.92$ for organ size, 85--100\% tumor recall for lung, pancreatic, colon, and bone lesions, and strong demographic alignment (BMI $ρ= 0.78$, sex-conditioned perfect prostate control).

    PDF
  • arXiv cs.CLOct 16

    Preservation and loss of algorithmic structure in experimental diffusion of joint-action norms

    Dhara Yu, Bill D. Thompson

    Everyday norms of social interaction - e.g. norms of sharing space, taking turns, or queuing - are essential to large societies, offering structured procedures for common interactions. However, it remains unclear how the underlying algorithms of these implicit joint plans evolve as they diffuse through populations. Here we show that this process of diffusion can select against complex algorithms in the earliest stages. We conducted a large-scale study of experimental diffusion of novel social norms among human participants, leveraging advances in program synthesis to characterize the procedures people developed in independent populations. Inherited procedures became structurally simplified and lost during interactions between participants who inherited different variants of joint-action norms; however, opportunities for interactive repair offset this pressure, preserving complex algorithms despite misalignment. These results demonstrate that interactive reasoning mediates selective pressures on structural complexity during cultural evolution of procedural norms.

    PDF
  • arXiv stat.MEOct 16

    lepidoptera: A Domain-Agnostic Engine for Causal Simulation with Mixed Variable Types, Vine Copula Dependence, and Individual-Level Counterfactual Propagation

    Danilo Augusto Sarti

    lepidoptera is an open-source R and Python package for simulation from structural causal models with heterogeneous variable types. The package is built around a typed mechanism architecture, which separates the causal graph from the generating rule at each node and exposes each rule as an inspectable, serialisable object. A pre-built mechanism library covers all nine combinations of continuous, categorical, and ordinal parent and child types, removing the need for users to implement type-specific sampling from scratch. A vine copula mechanism estimates the full joint dependence structure from observed data using pair-copula constructions and the Rosenblatt conditional-simulation algorithm, supporting over twenty copula families. Individual-level counterfactual queries are supported through partial propagation, which freezes the random draws of unaffected nodes and replays only the downstream consequences of an intervention, thereby separating the individual treatment effect (ITE) estimand from the population-level average treatment effect (ATE) without additional model assumptions. An extension layer allows any pre-fitted predictive model to serve as a generative mechanism via a…

    PDF
  • arXiv eess.SPOct 16

    Sequence-to-Sequence Neural Surrogate Modeling of Transient Responses in High-Speed Links with Variable Design Parameters

    Julian Withöft, Shayan Ghanbari, Werner John, Emre Ecik, Ralf Brüning +1

    This paper presents sequence-to-sequence (seq2seq) neural surrogates for high-speed transient response modeling and signal integrity analysis in fixed and variable design parameter spaces. To identify the structural requirements for prediction accuracy, multiple neural architectures were evaluated with respect to different training strategies such as sample-based and unit interval-based, as well as input configurations such as TX-only and TX+RX. For fixed design parameters, feedforward neural networks achieve accuracy comparable to recurrent models while requiring lower computational cost, as the static mapping can be approximated by fully connected and non-sequential representations. Variable design parameters alter the electromagnetic characteristics of the interconnect, which creates parameter-dependent temporal dynamics that require sequence-aware architectures to resolve the evolving interactions within the transient response. The results based on a 14-dimensional variable design parameter scenario show that while autoregressive TX+RX feedback suffers from severe phase drift and error propagation under variable conditions, non-autoregressive TX-only inference leads to…

    PDF
  • arXiv cs.SDOct 16

    From Spectrograms to Shallow Speech SSL: Audio-Frontend Design for Generator-Disjoint and Low-Latency Audio--Visual Deepfake Detection

    Qiwu Wen, Jahangir Alam, Eric Charton

    Video-mediated conversational systems rely on synchronized speech and facial video, making audio--visual input integrity important for reliable perception. We study audio-frontend design in a fixed MRDF (Modality-Regularization-based DeepFake)-based pipeline with AV-HuBERT visual processing and an MSCT (multi-scale cross-modal transformer) fusion encoder. We compare spectrogram-style log-filterbank features with frozen wav2vec~2.0 and WavLM representations, dual-SSL fusion, executed depth, and shallow-layer aggregation. Across three training runs, full wav2vec~2.0 improves mean macro AUROC from $84.19\%$ to $87.19\%$ on generator-disjoint MMDF. Six-layer execution further improves detection, and learned non-uniform aggregation adds $2.53$ percentage points, reaching $90.61\%$. Shallow execution reduces measured inference latency relative to full-depth wav2vec~2.0 across the tested input durations; first-second evaluation reaches $86.85\%$ macro AUROC. In a single-run comparison, dual-SSL concatenation outperforms either single stream, while one-way cross-attention yields lower detection performance and higher latency than concatenation. Layer-weight analysis shows greater…

    PDF
  • arXiv cs.LGOct 16

    Continual Test-Time Learning with Evolution Strategies

    Guangya Hao, Yiting Duan, Longyan Tan, Yunbo Long, Hao Wu +3

    Large language models increasingly operate in open-ended environments, where they are expected to continually adapt to evolving tasks and distributions, often without access to external supervision. Existing continual learning approaches rely on external supervision, such as labeled data, demonstrations, or reward signals, which can be costly or unavailable in practice. Test-time adaptation removes this dependence by learning from self-generated supervision, but existing methods derive that supervision from the single policy being updated, so once the policy drifts, its own learning signal drifts with it, leading to policy collapse, degraded reasoning, and catastrophic forgetting. In this paper, we propose \textbf{Evolution Strategy Reinforcement Learning (ESRL)}, a continual test-time learning framework that draws its self-generated supervision from a population of policies instead of the single policy being trained. ESRL perturbs an anchor policy in parameter space to form a population of nearby policies, samples one trajectory from each, and aggregates their answers into label-free group-relative learning signals for the central student. The anchor is held fixed within each…

    PDF
  • arXiv cs.LGOct 16

    Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks

    Magnus Sesodia, Juil Sock, Philip Torr, Christian Schroeder de Witt

    Solving algorithmic tasks with many sequential steps typically requires scaling up transformer depth or chain-of-thought length, incurring unfavorable parameter and inference-compute costs, respectively. Looped transformers, which reuse one block of layers across many iterations, are poised to address this, since they scale effective depth without adding parameters or tokens. Yet, it remains unclear what each iteration is actually computing, or whether the loop merely simulates a deeper untied stack. Here, we study directed permutation traversal, in which a model follows $H$ hops from a starting node and reports the node it reaches, so every intermediate hop has a known ground truth. We find that a minimum effective depth is necessary to solve the task, beyond which the degree of weight tying becomes the determining factor. For instance, a looped transformer with a four-layer block iterated eight times solves the task in every seed, while an identically-trained untied transformer of the same effective depth never does, succeeding only with a lower learning rate or supervision of its intermediate layers. Probing and intervening on the residual stream, we find that the looped model…

    PDF
  • arXiv cs.AIOct 16▲ 6

    A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt

    Shuyang Xiang

    We ask how much an autoregressive transformer retains of its prompt after generating $t$ tokens. Consider two different prompts followed by the same continuation text, so that any difference in their hidden states comes from the prompt alone, and we find that the distance between their hidden states decays approximately as a power law $t^{-α}$, where $α=1$ is the rate at which a fixed prompt's share of uniform attention shrinks and smaller values mean longer memory. Trained models forget fast ($α\approx0.55$--$0.7$), untrained ones slowly ($α\approx0.2$--$0.5$). We prove that this offset is $O\big(P(\log t)^{L-1}/t\big)$ for a prompt of length $P$ and depth $L$. The bound holds for finite-depth pre-LN softmax transformers with bounded attention logits, under one additional condition: the pre-norm activations stay away from zero. The mechanism is geometric: the prompt's direct influence is diluted as the context grows, and \emph{relay} --- continuation tokens passing on what earlier tokens read from the prompt --- can only add logarithmic factors. This is what we call the \emph{conditional no-hair theorem}: given the text generated so far, the prompt's influence on the next token…

    Code
  • arXiv cs.MAOct 16

    The Weight of Teamwork: Does Communication Structure Help LLM Scientists Search for Scientific Laws?

    Jianing Shi, Zaibin Zhang, Shijun Lei, Zhenfei Yin

    Team performance is often reported as one delivered score, even though that score is produced in two stages: a system generates candidate answers and a selection rule returns one of them. We use symbolic regression as an observable scientific-search setting and compare fixed-budget LLM teams that differ in peer visibility and update timing, evaluating both their frozen candidate archives and the expressions selected from those archives. Distributing the call budget across more independent trajectories produced the largest measured gain, improving mean held-out fitting score Q by 0.70 at the widest tested allocation. Under a common archive-wide selector, the implemented Ring policy reduced mean Q by 0.33 relative to Independent, while Star and Complete remained unresolved. Holding the generated archives fixed, changing the delivery rule changed the measured architecture gaps, including the sign of the Complete point estimate. These results support reporting candidate generation and final selection as separate endpoints. They do not identify a communication mechanism or a universally preferred topology; rather, they show that an architecture ranking is not identified by…

    PDF
  • arXiv q-bio.GNOct 16

    Reducing double-dipping in single-cell topic models with three-way count splitting

    Zheng Qu

    Topic models learn gene programs from single-cell RNA sequencing (scRNA-seq) data, and the same counts are then used to test which genes and donors are associated with those programs. Count splitting is the standard remedy for this reuse, but it has not been tested when an encoder reads each cell's counts. On a 104,400-cell lupus cohort with 300 permuted null genes per run, naive tests rejected 36% of null gene-topic pairs at a nominal 5% level, and the usual two-way split rejected 38%, because the encoder sees the tested gene's own held-out count. Encoding topic usage on a separate third of the counts, with beta-binomial thinning, lowered this to 7.7% (6.7% for genes outside the model); for two other models, LDVAE and scETM, the rate was 7.0-7.3% (naive 51-59%). A label-free interferon program shifts in held-out counts in the lupus cohort and in a pre-specified COVID-19 cohort, and donor-level topic usage predicts disease (AUROC 0.96) beyond cell-type proportions (0.86 and 0.58 in the two cohorts). Calibration and prediction did not depend on the model. We do not claim better clustering, prediction or topic stability. Code, figures and result tables are public (MIT license; DOI…

    PDF
  • arXiv cs.CROct 16

    Recording-Aware Evaluation of Multimodal OT Intrusion Detection

    Mustafa Umut Ozbek, Khalil El-Khatib, Li Yang

    Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording-associated differences as well as attack behavior. We introduce a recording-aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden. On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack. Same-recording evaluation reverses the apparent modality ranking across all studied window sizes: the physical view becomes the strongest individual modality, while cyber-only fusion shows the largest difference between cross-recording and same-recording AUC, reaching 0.54. Operational analysis further shows that permissive event-overlap metrics can overstate detection when a small number of long alarm episodes cover multiple actions. An external…

    PDF
  • arXiv cs.LGOct 16

    Data-Driven Resource Aggregation for Scalable Digital Twin and Simulation Model Generation

    Yucheng Wu, Alp Akcay

    As industrial digitization accelerates, the use of digital twins has become widespread. However, modern manufacturing systems are highly dynamic and frequently reconfigured, making manual model development and updates increasingly difficult. Recent research on automated simulation model generation shows that historical data such as event logs can yield simulation models automatically. However, modeling every resource at full granularity leads to unnecessary complexity and computational cost. We propose a method for automatically selecting and aggregating resources to simplify simulation modeling. Our method uses ideas from inverse reinforcement learning to learn behavioral value representations of resources from production event logs. This process identifies and aggregates resources with similar functional contributions. Experiments on manufacturing systems, using throughput and system time as performance measures, demonstrate that value-based resource aggregation performs well when the goal is to substantially reduce complexity while preserving performance accuracy.

    PDF
  • arXiv cs.LGOct 16

    On the Selectivity of Generative Models in Structure-Based Drug Design

    Ella Miray Rajaonson, Jungyoon Lee, William Chau, Alan Aspuru-Guzik, Benjamin Sanchez-Lengeling +3

    Designing small-molecule drugs that selectively bind to a target protein while avoiding unintended interactions with other proteins is essential for reducing side effects that contribute substantially to clinical attrition. Although structure-based drug design (SBDD) has advanced rapidly with deep learning, most generative models still optimize binding to a single target and disregard selectivity due to limited training data and unknown off-target interactions. Motivated by this gap, we introduce SELECTBENCH, a comprehensive benchmark assessing the selectivity of generative SBDD models. SELECTBENCH provides three curated evaluation tasks reflecting different real-world drug discovery scenarios: literature-based case studies, safety screening panels used in pharmaceutical development, and large-scale structure-based evaluation sets. We further introduce a suite of metrics to quantify selectivity and conduct a systematic evaluation of nine representative SBDD models, including methods explicitly trained for selectivity and widely-used inference-time guidance techniques. Our results show that even selectivity-trained models achieve only modest off-target discrimination and…

    PDF
  • arXiv cs.CLOct 16

    Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers

    Peter Cho-Ho Lam, Ziyi Wang, Zirui Zhou

    Prefix-tuning is a parameter-efficient fine-tuning method for large language models (LLMs) that steers model outputs by prepending task-specific prefix representations to the input sequence. Despite its practical success, the theoretical understanding of the expressiveness of prefixes remains incomplete. In particular, the following question remains open: Given a pretrained autoregressive Transformer, for any input-output sequence pair $(\mathtt{x},\mathtt{y})$, does there always exist a prefix $\mathcal{S}$ such that prepending $\mathcal{S}$ to $\mathtt{x}$ induces the model to generate the target output $\mathtt{y}$? In this work, we prove that under the assumption that $\mathtt{y}$ is in the range of the pretrained Transformer, such a prefix $\mathcal{S}$ exists and can be explicitly computed by solving multiple linear systems. We further show that this assumption is necessary, in the sense that such a prefix may not exist when the assumption fails, thereby providing a complete answer to the above question. Our analysis is constructive and the obtained theoretical results hold under fairly general assumptions on model architecture. Extensive experiments on GPT-2 and Qwen3-4B…

    PDF
  • arXiv cs.CVOct 16

    HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation

    Dongzhihan Wang, Yi Du, Chen Wang, Xiuxian Li, Liang Xu

    Vision-Language Navigation (VLN) requires long-horizon instruction-consistent planning and short-horizon local planning in current geometry. Video-action models (VAMs) offer a promising way to learn predictive representations for navigation. However, existing VAMs jointly predict visual futures and local trajectories in a single process, even though they operate at different paces: future prediction remains stable across multiple updates while local trajectories must react to fresh observations. To bridge this gap, we introduce HAVANA, a dual-system VAM that decouples predictive reasoning from reactive local planning. A slow system produces a compact language-conditioned latent plan from observation history, while a fast system combines the latest plan with the current observation to generate local trajectories. Their asynchronous execution enables rapid adaptation without repeated future prediction, and no future video is generated during inference. Experiments on OmniNavBench demonstrate state-of-the-art navigation performance over prior baselines, while real-world deployments further show robust generalization to unseen environments across different robot embodiments. Our code…

    PDF
  • arXiv cs.CLOct 16

    Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles

    Célian Vasson, Daniil Ognev, Kentaro Inui, Benjamin Heinzerling

    Variable roles describe recurring behaviors such as accumulation, traversal, and retaining the best candidate encountered. We use Sajaniemi's independently defined taxonomy to study which role distinctions can be recovered from large language models, how sparse autoencoders (SAEs) retain those distinctions, and whether associated directions support control of generation. We introduce static annotation rules for all eleven roles and a corpus of 126,926 natural-code files across five languages. Across five model-SAE systems, residual probes outperform individual SAE features and tested lexical baselines, with modest losses after neutral target renaming. Combining SAE features closes much of the single-feature gap, but probes on SAE reconstructions are less sensitive to role-changing edits reviewed by LLMs. High paired ranking accuracy coexists with frequent false positives at fixed thresholds. During generation, interventions often change role annotations at the cost of correctness; changes accompanied by passing tests show no clear aggregate advantage over random controls. The benchmark separates role-related recoverability, SAE fidelity, and correctness-preserving control through…

    PDF
  • arXiv cs.LGOct 16

    Graph-Regularized Synthetic Control for Stable Counterfactual Estimation under Donor Redundancy

    Yanlin Zhang, Sungyong Chung, Alireza Talebpour

    Synthetic control methods estimate counterfactual outcomes by weighting untreated donor units to match a treated unit before treatment. In recent applications, however, the donor pool can be large, highly correlated, and short in pre-treatment history, yielding many near-equivalent solutions with unstable and difficult-to-interpret donor weights. We propose Graph-Regularized Synthetic Control (Graph-SC), a convex estimator that augments the classical pre-treatment matching objective with a graph Laplacian penalty. The graph encodes similarity among donor trajectories, encouraging coherent weights over structurally related donors while preserving the transparency of synthetic controls. We provide theoretical results characterizing the bias-variance tradeoff induced by graph regularization and establish efficient convergence guarantees with an accelerated proximal gradient method. Across a controlled simulation under donor redundancy, the California Proposition 99 benchmark, and observational counterfactual analyses on the Waymo Open Motion Dataset, Graph-SC improves weight stability, reduces sensitivity to redundant donor pools, and maintains competitive pre- and post-treatment…

    PDF
  • arXiv cs.LGOct 16

    Looped GNNs Beyond Fixed Depth: Graph Size Generalization and Parameter-Efficient Adaptation

    Zhiyang Wang, Zhishang Luo, Yusu Wang

    Generalizing from small graphs to larger ones can require computation beyond the depth of a neural model used during training. We study target tasks that admit a common local iterative computation and whose effective dependency radius grows with graph size. The central idea is to learn a local update rule through GNN pretraining, then repeatedly apply the same learned processor beyond the training depth to extend information reach on larger graphs without retraining. Calling such parameter-shared architecture a \emph{looped GNN}, we categorize six schemes for looped GNNs to separate the effects of depth extension, fine-tuning, and freeze-and-steer adaptation. Our analysis establishes fixed-depth information limits and sufficient conditions under which accurate iterative reuse overcomes them. To improve beyond unadapted reuse, we introduce \emph{Loop-Freeze-Steer} (LFS), which freezes the pretrained processor and learns a small controller to steer its repeated application, using much fewer trainable parameters than full fine-tuning. Our finite-sample analysis gives sufficient conditions for adaptation to improve performance on larger graphs. Experiments on synthetic tasks and chip…

    PDF
Load more
PDF