Research

• General in-context learning for embodied agents1 paper raises it · 0 address it

  Enables models to adapt to new tasks and environments without retraining, improving robustness in real-world deployment.

  • Has GPT-6-Astra Achieved Embodied Intelligence?raises
• Cost and latency in embodied control loops1 paper raises it · 0 address it

  High computational and time costs limit real-time applicability of agentic systems in dynamic environments.

  • Has GPT-6-Astra Achieved Embodied Intelligence?raises
• Physical-state verification and failure recovery1 paper raises it · 0 address it

  Agents must reliably detect and correct errors in physical interactions to avoid irreversible failures in real-world tasks.

  • Has GPT-6-Astra Achieved Embodied Intelligence?raises
• Long-horizon task preservation in embodied agents1 paper raises it · 0 address it

  Maintaining progress over extended sequences is critical for complex, multi-step tasks in robotics and navigation.

  • Has GPT-6-Astra Achieved Embodied Intelligence?raises
• Open-source stacks for generalist computer-use agents1 paper raises it · 1 addresses it

  Closed systems hinder reproducibility and systematic study of agent design for real-world computer tasks.

  • OpenGCUA: Toward a Fully Open Stack for Generalist Computer Useraises · addresses

Named by an AI from these papers' abstracts.

• arXiv cs.LGOct 18

  Finite Interaction Lens: Structured Precision Sharing for Vision-Language Models

  Nozomu Fujisawa

  Identifying which visual interventions interact in a vision-language model requires comparing their joint and separate effects. The resulting four-output difference can be small enough for rounding error to obscure it. We introduce the Finite Interaction Lens (FIL), a mixed-precision method for evaluating finite intervention families. At each selected linear projection, FIL decomposes each candidate's input into a native state, two single-intervention components, and a full-dimensional mixed residual. It projects the single-intervention components at higher precision and reuses the same numerical contributions in the joint candidates, while retaining every mixed residual and nonlinear trajectory. We derive a local cancellation identity and a conditional bound on error propagation through a smooth nonlinearity. In one MLP of Qwen3-VL-4B-Instruct, FIL reduces final interaction MSE by 94.4% on 16 synthetic families and by 99.85% and 99.35% in two separate four-image COCO studies, relative to a control that has the same nominal BF16-product budget and higher-precision single interventions but projects each joint candidate independently. One image accounts for most of the absolute…

  PDF
• arXiv cs.LGOct 18

  Benchmark for Structured Retrieval-Augmented Generation in Perovskite Photovoltaics

  Qiwei Han, Chi Zhou, Gaoxiang Yuan

  While retrieval-augmented language models increasingly serve as literature assistants in the physical sciences, existing benchmarks evaluate them primarily on qualitative text overlap or through model-based scoring rather than against verified experimental ground truth. In experimental materials disciplines such as perovskite photovoltaics, however, the most critical inquiries are inherently quantitative and aggregate measurements across dozens or hundreds of papers. Here, we present a quantitative question-answering benchmark where all 755 ground-truth answers are computed directly from 45,441 measured device records (40,889 human-curated records through 2021 and 4,552 records extracted from 2022-2026 literature). Benchmarking six retrieval architectures over 158,274 passages from 25,662 papers using an open 14B model on a single 24 GB GPU, standard passage retrieval tops out at 16-23% accuracy (compared to 1.6-6.5% zero-shot). In contrast, equipping the language model with read-only SQL execution over the structured device table elevates accuracy to 51.5% on historical and 50.8% on prospective literature -- a 33.01.8 percentage-point advantage over hybrid text retrieval.…

  PDF
• arXiv cs.CLOct 18

  Representations in Motion: Layer-wise Rotation and Trajectory Divergence in LLMs

  Zhuonan Yang, Yik Siu Chan, Wannan Yang, Ellie Pavlick

  Recent work suggests that activation dynamics--how representations change during inference--might be more revealing about language model behavior than static readouts from individual layers. In this work, we investigate the layer-wise evolution of representations of factuality, using true and false statements when the model is prompted to either lie or to be honest. We report a geometric phenomenon in which the factuality directions extracted from the lying and honest conditions appear to rotate away from each other across layers. We find that such layer-wise rotation also occurs in a task that requires the model to take different visual perspectives in the 3D world, suggesting that the rotation phenomenon might be indicative of manipulating conceptual spaces in general, rather than unique to factuality. We then investigate the "real time" transformation of conceptual representations on a trial-to-trial basis. We show that it is possible to make strong predictions about the model's target task and behavior from trajectories alone. In addition, diverging points in single-trial trajectories correspond to the onset of rotation in aggregated-trial analysis. Finally, we show that both…

  PDF
• arXiv cs.CVOct 18

  DiffGround: Semantic Difference-Guided Zero-Shot Video Question Grounding

  Cheng-Han Lee, Amir Salimi, Hsiang-Wei Huang, Che-Chun Su, Min Sun +1

  Video Question Grounding (VideoQG) requires both accurate question answering and precise temporal localization. Existing supervised methods rely on costly temporal annotations, while existing zero-shot approaches are primarily designed for Video Question Answering (VQA) and are not optimized for temporal grounding. We propose DiffGround, an action-driven zero-shot framework built upon frozen Multimodal Large Language Models (MLLMs). Our framework first predicts an answer-relevant action cue that transforms the input question into an explicit semantic target for grounding. Action Cue converts a reasoningoriented question into an action-centric retrieval query. Guided by the action cue, a Vision-Language Model (VLM) efficiently selects informative keyframes for reasoning. Finally, we introduce Semantic Diff Refinement, a training-free temporal refinement method that exploits semantic differences between neighboring SigLIP2 embeddings to recover complete action segments from sparse predictions. Extensive experiments on NExT-GQA and STAR show that DiffGround consistently outperforms existing zero-shot and supervised methods, establishing a new state of the art for Video Question…

  PDF
• arXiv cs.AIOct 18

  Design and Cross-Domain Evaluation of an Agent Execution Harness

  Saurav Joshi, Manas Gupta, Ashwin Venkatesh Raman, Akshay Budhkar

  Completing complex tasks with language-model agents requires coordinating planning, execution, and review so the final result addresses the user's original objective. We present an agent execution harness that keeps the original objective separate from a revisable plan and an execution record. A planner assigns scoped tasks, workers produce outputs and artifacts, and a reviewer examines results against the original request before submitting or requesting additional work. Reusable domain-specific instructions guide these roles and are refined during construction using task inputs and execution trajectories, without official grading feedback. We evaluate complete harness configurations across terminal tasks, clinical responses, application building, and scientific questions. On Terminal-Bench 2.1, the harness passes 83 of 89 instances (93.26%); on HealthBench Professional, a Fable-configured system achieves a length-adjusted rubric score of 69.63 across 525 tasks. In a 53-task paired HealthBench comparison, the Fable-configured harness achieves an aggregate score 19.89 points higher than direct completion, despite a $-1.46$-point median task-level change; mean inference cost rises…

  PDF
• arXiv cs.CLOct 18

  Flattening Fantasies: Analyzing Stereotypes in Pornographic Discourse

  Sarmad Chandio, Osama Khalid, Sharaf Zia, Ethan Kutlu

  This study presents a large-scale quantitative analysis of 5.4 million video titles from xHamster, the world's fourth most visited pornography platform, to examine linguistic patterns used to describe different dimensions of identities (e.g., race and gender). Specifically, we investigate how nationalities, ethnic groups, and gender identities are portrayed in pornographic discourse, focusing on stereotypes and deviations from implicit norms of pornography. The results show the dominance of American pornography over other groups, which acts as the de facto norm for the adult industry. We identify persistent ethnic stereotypes reflecting colonial power dynamics, religious fetishization, and the sexualization of cultural taboos. The analysis reveals gendered representational differences across cultures, with a prevalent Madonna-Whore dichotomy in the portrayal of women and stereotypical depictions of men in physically dominant roles.

  PDF
• arXiv physics.comp-phOct 18

  Concurrent two-scale topology optimization of printable bone scaffolds via rectified-flow generation and exact latent-space refinement

  Reza T Batley, Sourav Saha

  Bone scaffolds must reconcile a macroscale mechanical response with microscale geometric feasibility, yet conventional approaches to their design restrict attainable constitutive properties. This work presents a concurrent two-scale topology optimization framework; a conditional rectified flow generates periodic lattice microstructures, which are then refined in latent space by exact finite-element homogenization sensitivities. The resulting unit-cell geometries and their spatial assignments are jointly optimized under mechanical and feature-size constraints. In a distal rabbit femur benchmark, all eight flow-started refinements produce printable cells with maximum relative errors under 1\% across six prescribed engineering moduli. Constructing the implant from a single tiled cell reproduces intact whole-femur compliance within 0.1\% and regional strain energy within 0.35\% under four loading conditions. In human femoral reconstruction, multiscale concurrent optimization produces a cortical mid-shaft scaffold with porosity 63\% while satisfying the compliance constraint within 2\% under walking and stair-climbing loads. It achieves an 84\% porosity on a proximal cancellous…

  PDF
• arXiv cs.AIOct 18

  Learning Which Device to Call: Online Tool Grounding for LLM Agents in IoT

  Yichen Liu, Xiaochong Jiang, Haoran Yu, Lifei Liu

  The Internet of Things (IoT) gives large language model (LLM) agents access to physical observations, but one requested capability is usually offered by several devices, and the agent's plan does not say which one to call. Existing work retrieves relevant tools and checks plan feasibility, without learning which eligible device returns a useful observation. Unlike previous works, we study online tool grounding: binding an already specified tool call to one physical endpoint under service cost and device participation constraints. We propose a framework that learns which physical device should execute each agent-issued call. It formulates selection as a contextual bandit problem and combines adaptive context partitions with hierarchical endpoint trees, so that execution feedback is reused across related requests and devices. Selection balances task success against service cost while respecting device capability, authorization, battery reserves, and availability. Gaussian reward aggregation provides an optional mechanism for protecting individual feedback values. Theoretical results establish a finite-horizon regret bound and differential privacy for released reward statistics…

  PDF
• arXiv cs.LGOct 18

  FTFM: Factorized Tabular Foundation Models

  Josif Grabocka

  Tabular foundation models (TFMs) are becoming a powerful alternative to conventional tabular learning, adapting to new datasets in context without task-specific training. Current architectures, however, face a trade-off: models that preserve cell-level representations are expensive because they repeatedly attend across rows and columns, while faster models collapse each row into a fixed embedding and lose this fine-grained state. We introduce Factorized Tabular Foundation Models (FTFM), which preserve cell representations while factorizing their computation through one contextual latent per row and one per column. Cells no longer attend to one another directly; instead, each cell is updated through the interaction of its row and column latents, reducing repeated attention while retaining feature-specific information. FTFM achieves competitive predictive performance in tabular classification on major benchmarks such as TabArena and BeyondArena.

  PDF
• arXiv cs.CVOct 18

  Tricking Synthetic Image Detectors with Amortized Generative Attacks

  Mohamed Awad, Khaled Elhady, Yasser Rohaim, Samuele Poppi, Nils Lukas

  Rapid advances in image synthesis have increased the need for reliable synthetic-image detectors, yet their robustness to transferable attacks remains incomplete and understudied. Existing evaluations generally examine whether an attack constructed against one detector transfers to another, but rarely isolate how its effectiveness changes when the attacked images originate from different generators. We conduct a systematic evaluation of three representative attacks: two iterative perturbation-based methods, MIG and MuMoDIG, and the amortized generative attack Tricker. Across four detectors and five generators spanning diverse architectures, we introduce a two-axis evaluation protocol that independently varies the surrogate detector used to construct or train the attack and the generator supplying its input images. We assess attack effectiveness, cross-detector transferability, perceptual quality, runtime, and interaction with common post-processing operations. We revisit Tricker and show that its data and training requirements can be substantially reduced, enabling broader and more efficient evaluation. Our results demonstrate that transferability is not a detector-only property:…

  PDF
• arXiv cs.AIOct 18

  Narrow Decision Models: A System-1 Architecture for Calibrated, Typed Decisions

  Carlos André Antunes, Aldry Albuquerque, Gabriel de Melo Schmith, Natã Batista, Pedro Oziel Escobar da Silva +5

  System One models, introduced with TypeSafe's Jev, answer typed questions about a state with calibrated probabilities instead of generated text: choosing among options described in natural language (Choice), placing the state on an ordered scale (Score), or judging a yes/no statement (Noul). Jev is general-purpose: one set of weights, served as an API and adapted to a domain only through the request. We study the other end of the design space. A Narrow Decision Model (NDM) offers the same interface but is trained for the decisions of one organization, is small enough to run on its own infrastructure, and has an open architecture whose properties can be stated and tested. The state can be any text or structured record about which decisions are made; our running case is the routing of court communications. The architecture serializes a request as a prefix tree (the state, then each question, then each option) and runs a bidirectional encoder under a block attention mask with logical positions, so that each answer is, by construction, invariant to the order of its options and independent of the other questions asked. Routing through an organization chart is a composition of Choice…

  PDF
• arXiv cs.AIOct 18

  Symbolic Constraint Agents: Mining Formal Tool-Call Constraints to Prevent Agent Errors

  Jinu Lee, Shraddha Barke

  Symbolic constraints (programmatic, deterministic checks for tool calls) enable deterministic verification of LLM agents' tool calls, but existing approaches primarily enforce predefined safety policies rather than tool dependencies in complex tool environments. We explore whether symbolic constraints can be mined from environments and observations beyond predefined policies, and whether they can improve end-to-end task performance. We present Symbolic Constraint Agents (SCA), a framework that mines symbolic constraints from API documentation, refines them against execution traces, and checks the agent's tool calls with an SMT solver at runtime. A violation triggers a natural-language warning or, when the valid replacement is uniquely determined, an automatic repair. On AppWorld, refinement raises error-detection F1 from 37.0 to 58.3 on GPT-5.5 traces and from 39.3 to 61.9 on Qwen3.5-27B traces. At runtime, repair raises GPT-5.4-mini's task success by 5.4 points under a 40-call budget and 9.8 points when environment errors are hidden, while removing 59--87\% of failed calls across models, and transfers to $τ^2$-bench with gains of up to 5.6 points. These findings show that…

  PDF
• arXiv cs.DCOct 18

  StreamAhead: Real-Time Streaming Video Question Answering via Adaptive Precomputation

  Haoyang Fan, Dhruv Parikh, Rajgopal Kannan, Viktor Prasanna

  Real-time streaming video question answering with vision-language models must answer queries that arrive at any time with low latency while sustaining the video-processing rate required by the application. Existing systems construct key-value (KV) caches for the entire video as it arrives, although a query typically uses only a few of the preceding frames as evidence. This highlights a fundamental trade-off: performing more computation as video arrives reduces query-time computation but can increase processing lag, whereas deferring computation can increase the video-processing rate but requires additional query-time computation for retrieved content. We present StreamAhead, an algorithm-system co-design that exploits this trade-off through adaptive precomputation: dynamically controlling how much computation to complete for each video segment before a query arrives. StreamAhead encodes video segments independently and lets each remain at a different representation level, from inexpensive visual features to costly KV caches, completing KV caches only for the retrieved segments when a query arrives. StreamAhead profiles offline the cost of advancing a segment to each…

  PDF
• arXiv cs.AIOct 18

  Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing

  Yunlong Deng, Yan Li, Xing Hu, Yida Zhu, Sean Du +2

  A synopsis in novels folds a story into a few lines; a function in code gathers many steps with a single execution. Context editing is native to language itself. In this paper, we propose the Native Context Editing (NCE) framework, which gives language model agents a grammar for natively editing their own context. By encoding context-editing operations in a grammar of dedicated special tokens, we teach the agents to invoke edits through the language itself. We train agents on iteratively constructed trajectory segments, with each teacher edit conditioned on the evolving context. This establishes context editing as an atomic and inherent capability of LLM agents, enabling them to dynamically modify their context as they generate responses over long-horizon reasoning and interaction. Compared with context-editing methods that learn to generate tool-use code within specific external harness environments, our approach integrates context editing as an internal mechanism of the model's native generation process, enabling better generalization across diverse real-world workflows. On four long-horizon agentic benchmarks (Terminal-Bench 2.0, SWE-bench Verified, SWE-bench Multilingual, and…

  PDF
• arXiv cs.CLOct 18

  EmotiQ: A Quantum Residual Specialist Framework for Text Emotion Classification

  Carlton Kigonya, Tanaka Chirara, Yunting Yin

  Emotion classification from text is challenging because many affective states are distinguished by subtle linguistic signals and semantically overlapping boundaries. We investigate whether compact quantum modules can serve as targeted residual specialists that assist traditional classifiers on difficult emotion decisions. We introduce EmotiQ, a residual specialist framework for text emotion classification using a frozen transformer classifier as the primary decision maker, identifies challenging emotion boundaries from validation behavior, and trains boundary specialists, including quantum encodings, data re-uploading specialists, prototype and fidelity specialists, and kernel-based specialists. Specialists are permitted to intervene only when they satisfy validation-based admission criteria. We evaluate the framework across six text emotion classification datasets and establish traditional model baselines. Experimental results show that the admitted EmotiQ policy improves mean macro-F1 metrics over the baseline, and achieves positive gains on five of six datasets. Among the quantum components, ZZ interaction feature maps are the most consistently effective, yielding a mean…

  PDF
• arXiv cs.LGOct 18

  Beyond the record: evaluating probabilistic AI weather forecasts deep in the tail

  Bastiaan Van Velthoven, Cas Decancq, Diego G. Miralles, Stijn Luca, Thomas Mortier

  Can probabilistic artificial intelligence (AI) weather forecasts retain skill beyond their training domain? We compare four state-of-the-art AI models (ArchesWeatherGen, U-Cast, WeatherNext2 and GenCast-mini) with the physics-based IFS-ENS over land in 2022, at 1.5° resolution and lead times of 2--10 days. Monthly ERA5 records from 1979--2021 define common historical boundaries, and we characterize new records set in 2022 by their depth, i.e., the exceedance margin relative to local historical variability. Two proper scores, record-threshold weighted CRPS and Brier score, assess the magnitude and probability of record exceedances across all eligible cases, retaining both missed extremes and false alarms. Verified against ERA5, AI models outperform IFS-ENS for heat and cold records at almost every lead and depth. For heat, the advantage strengthens with depth, partly because IFS-ENS over-forecasts deep records, and largely persists for deeper records under HRES-fc0 verification. For wind, gains erode for deeper records and longer leads. Better tail scores can nevertheless coexist with aggregate probability deficits or surpluses, and some improvements arise chiefly on…

  PDF
• arXiv cs.LGOct 18

  Reinforcement learning on the EPRI AI-ccelerating Unit Commitment benchmark: diagnosing over-commitment and guiding MILP with learned policies

  Maharani Devira Pramita, Aidan O'Sullivan

  Reinforcement learning (RL) is increasingly proposed as a fast solver for unit commitment (UC), but standalone agents remain far more expensive than mixed-integer linear programming (MILP). In the first study of RL on the 51-generator, 72-hour benchmark of the Electric Power Research Institute (EPRI) AI-ccelerating Unit Commitment competition, we trace this gap to the agents' own exploration, close most of it, and show that the resulting policy can accelerate an exact solver. In every configuration tested, Proximal Policy Optimisation (PPO) and an action-branching Deep Q-Network (DQN) trained from scratch committed about twice the units of a thermal-only MILP optimum, at 39-64% higher cost, whereas a merit-order rule came within 1.6-7% (10-24% when unserved energy is priced). When each unit's decision is sampled independently, spare units insure against random de-commitments. Consistent with this, injecting exploration noise during fine-tuning enlarged a lean policy's fleet, a reward without shaping terms did not remove the over-commitment, and under their training noise the agents outperformed the rule subjected to comparable noise. Imitating the MILP solutions and fine-tuning…

  PDF
• arXiv cs.AIOct 18

  Auditing Tool Traces in Agentic LLMs: Source Localization and Counterfactual Dependence Testing

  Junhao Liu, Tianxiao Li, Yongxian Wu, Ziheng Duan, Ren Li +2

  Agentic LLMs increasingly support scientific prediction by grounding decisions in external evidence. In biology, they promise accurate, interpretable predictions for important problems by integrating complex experimental and computational evidence. However, serialized tool results can contain hidden label shortcuts, such as dataset identifiers or template artifacts, that create apparent success without biological reasoning. To address this risk, we introduce a two-stage audit. Stage 1, Per-Tool TF-IDF Decomposition (PT2D), localizes label-predictive patterns to individual tool results while avoiding signal dilution in the concatenated tool trace. Stage 2, Token-Class Redaction, counterfactually tests whether the frozen LLM depends on the localized shortcut. We apply this audit to TCR-pMHC binding, where tool augmentation substantially outperforms a no-evidence baseline. At face value, this suggests that the LLM has learned to use scientific evidence. The audit overturns this interpretation: PT2D localizes the strongest signal to identifiers in one tool result, and redacting them reduces performance to chance. These findings show how hidden label shortcuts can masquerade as…

  PDF
• arXiv cs.CVOct 18

  Residential Construction Image Dataset Using CSI Divisions for Deep Learning-based Object Detection

  Fahad Iqbal, Shayan Mirzabeigi

  Accurate recognition of construction site elements is essential for the application of deep learning (DL) in the construction industry. Despite rapid advances in computer vision, the adoption of DL in construction has been limited by the lack of structured, publicly available datasets aligned with established industry standards. To address this gap, we introduce the Residential Construction Dataset Using CSI Divisions (ResCD-CSI) image dataset, a manually labeled image benchmark organized according to the divisional structure defined by the Construction Specifications Institute (CSI). The dataset comprises high-resolution images collected from active residential construction sites across the United States, capturing diverse building components, materials, and equipment under real-world conditions. In total, 16,365 objects spanning 14 major CSI divisions were precisely annotated using bounding boxes, ensuring consistency and reliability for object detection tasks. By aligning with the CSI divisional framework, ResCD-CSI establishes a standardized resource that bridges construction practice with computer vision research and supports the development of Artificial Intelligence-driven…

  PDF
• arXiv cs.CVOct 18

  Modaliff: A Unified Token Diffusion Model for Any-to-Any Generation

  Duc Kieu, Kien Do, Minh Khoa Le, Tung Kieu, Dang Nguyen +1

  Any-to-any generation seeks a single model that translates flexibly among text, image, and audio, yet existing approaches either require fully aligned multimodal tuples or a shared representation that suppresses modality-specific structure. We introduce Modaliff, a novel token-based diffusion model that learns from widely available bimodal pairs while preserving each modality in its native latent sequence. Modaliff assembles conditions and targets into a unified token sequence, with per-token attributes to disambiguate token identities and role-based positional embedding to preserve each modality's internal structure across sequence layouts. This design enables a single denoiser to support diverse generation tasks without task-specific architectural changes or a shared cross-modal latent space. Beyond one-to-one generation, Modaliff supports multiple conditions either by combining single-condition predictions at inference or by lightweight finetuning on captions that jointly describe multiple inputs. Experiments across text, image, and audio show that Modaliff achieves the best fidelity among any-to-any models on most generation tasks while using substantially fewer parameters…

  PDF

• Federated learning50+35.1%
• Agents527+1.2%
• Quantum machine learning240.0%
• Character recognition OCR170.0%
• Sensing communication150.0%
• Frontier LLMs447−7.8%
• Forecasting time78−10.3%
• Image segmentation183−12.4%