Research

• Reliable request execution in streaming video generation1 paper raises it · 0 address it

  Models often fail to follow incremental user requests while maintaining state consistency in real-time video generation.

  • StreamEval: Evaluating Interactive Streaming Video Generation Modelsraises
• Calibration and contrast-group accuracy in decision models1 paper raises it · 0 address it

  Aggregate accuracy hides failures on controlled variations and unreliable confidence estimates in decision-making systems.

  • DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises
• Length generalization in state tracking1 paper raises it · 1 addresses it

  Models struggle to maintain accuracy on problems much longer than their training data without explicit progress alignment.

  • PAR: Progress-Aligned Recurrence for Length Generalization in State Trackingraises · addresses
• Token collapse in graph soft prompting1 paper raises it · 0 address it

  Graph-specific tokens lose discriminative power, reducing the method to a generic prompt with no graph information.

  • On the Token Collapse Phenomenon in Textual Graph Soft Promptingraises
• Coupled evaluation of multimodal agents and world models1 paper raises it · 0 address it

  Separate evaluation of agents and world models fails to predict success when they are combined in real-world tasks.

  • Dual Exam: Controlled Evaluation of Multimodal Agents and World Modelsraises

Named by an AI from these papers' abstracts.

• arXiv cs.LGOct 9

  Agreement versus Softmax under Spatial Disambiguation

  Michał Zakrzewski, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv

  ScanRefer Multiple queries require spatial disambiguation: several instances of the target class are present, and the referencer must pick one. When per-query logits were never logged, a common substitute is cross-epoch agreement. On the full ScanRefer validation set (9,508 queries; GeoClassifier with ground-truth proposals), object-level softmax is similarly miscalibrated on Unique and Multiple (ECE 0.068 vs. 0.102; the scene-bootstrap gap interval includes zero). Fixed-final agreement on the same checkpoints reports ECE 0.041 vs. 0.272 (gap +0.231). Modal-vote agreement is 0.045 vs. 0.318. Most Multiple errors are same-class wrong-instance (45.6%); geometric miss (H2) is 59.6%, and wrong-category (H1) is 16.7%. A non-monotone H1 pattern does not appear under the default fixed-final protocol, and we retract it. Logit-free thresholds rank spatially hard queries poorly. If only agreement was logged, calibrate abstention on Multiple and report wrong-instance rates with H1.

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs

  Boris Radovič, Marco Canini, Veljko Pejović

  The most accurate agents executing user-provided tasks are driven by large, cloud-hosted models, which solve them by reading the private data of the user. Agents powered by models small enough to run on edge devices keep that data local, but yield far worse results. A third option is to have cloud models write a program that devices execute: the data stays local, but as the program is written before any data is seen, it fails whenever a step depends on interpreting content that appears only at run time. We explore an intermediate architecture, in which the cloud-written program defers bounded questions over user data to a small local model through an ordinary function call. On AppWorld this improves the success rate of a code-only agent in all six combinations of backbone model and test partition we evaluate, by up to 14.9pp, recovering a substantial part of the gap to a cloud-hosted ReAct loop while keeping user data on the device.

  PDF

  Summarize this
• arXiv cs.CVOct 9

  Token-Budgeted Candidate Retrieval for 3D Scene-Grounded VLMs: Category Names vs. Pairwise Geometry

  Michał Zakrzewski, Mariia Shpir, Taras Rumezhak

  Under a fixed context budget, which objects and relations should a 3D scene-grounded VLM see first? On ScanNet indoor referring benchmarks (Dai et al., 2017), matching the query to object category names outperforms pairwise geometry scoring at every reported K. The gaps are largest at high recall. Analytic pairwise cues beat per-object cue scoring and query-agnostic kNN graphs, including on paraphrases. MiniLM and BGE-small still score higher on those splits. We evaluate token-budgeted candidate retrieval with axis-aligned boxes, a token-ratio metric, training-free baselines, two small encoders that match q to each object's category name, and QCRR (Query-Conditioned Relation Retrieval), a deterministic pairwise-geometry scorer. Across ScanRefer gt_val plus a custom 20% scan-id split of Nr3D and Sr3D (35,504 queries; not official ReferIt3D val), additive noun matching reaches R@10 0.864 on ScanRefer against QCRR's 0.626; MiniLM and BGE-small reach 0.911 to 0.913 on ScanRefer and 0.941 to 0.947 on Nr3D. A corrected 3DGraphLLM harness (native Acc@0.25 0.67 on the full graph, n=300 ScanRefer queries) shows that context selection changes downstream accuracy under a matched object…

  PDF

  Summarize this
• arXiv cs.LGOct 9

  Refined Grid Sampling for Efficient Implicit Neural Representations of Large-Scale Scientific Data

  Guang Zhao, Xihaier Luo, Sean McCue, Luke P. Van Roekel, Seungjun Lee +3

  Implicit neural representations (INRs) offer a powerful framework for modeling high-dimensional physical fields, but their training remains inefficient because evaluating dense coordinate grids is computationally expensive. Existing coordinate-pruning strategies rely on local error functions that can oversample correlated points and incur substantial overhead from full-grid error evaluation. We introduce Refined Grid Sampling (RGS), which combines stratified grid sampling, error-based refinement, and dynamic grid-resolution adjustment. This design reduces spatial redundancy without evaluating errors over the full grid. On high-resolution Navier--Stokes fields, RGS achieves higher reconstruction accuracy within a fixed wall-clock budget and reaches the accuracy of full-grid training up to $6\times$ faster than the evaluated baselines.

  PDF

  Summarize this
• arXiv cs.ROOct 9

  ClothCode: Generative Modeling for Garment State Estimation and Dynamics Prediction

  Yingdong Ru, Lipeng Zhuang, Shiyu Fan, Zhuo He, Florent P. Audonnet +1

  We introduce ClothCode, a unified generative framework for garment state estimation and action-conditioned dynamics prediction in a shared discrete configuration space. A material-aligned tokenizer encodes garment configuration as residual vector-quantized patch tokens, while task-specific Masked Generative Transformers predict these tokens through parallel confidence-guided refinement. Across three garment templates, ClothCode reduces mean displacement error by 26.6% for state estimation and 37.0% for dynamics prediction relative to the strongest baselines. Dynamics prediction requires only 16.7 ms per window, compared with 556.2 ms for diffusion. Integrated into the same model-predictive controller, ClothCode improves real-world folding and flattening success from 56.7% to 78.9%. Further results and videos are available at: https://anonymous.4open.science/r/ClothCode-DE32

  PDF

  Summarize this
• arXiv cs.IROct 9

  Simple Linear Corrections for Compression in Late Interaction

  Mihir Agarwal, Sabyasachi Basu, Viraj Agrawal

  Late-interaction (multi-vector) models represent documents using a set of token-level vectors, providing a middle ground between cross-encoders and dense encoders in terms of quality and cost of embedding. However, compared to the latter, the indices are costly to store and score. Pooling and pruning based compression techniques are deployed widely but deteriorate in quality noticeably with aggressive compression. We investigate whether a lightweight change to the scoring function can recover retrieval quality after document-vector compression. For each combination of encoder, compressor, and vector budget, we learn a global linear correction by distilling the uncompressed model's score distributions over candidate documents. The encoder and compressor remain frozen, and the correction can be applied to each query at search time or once, offline, to the stored vectors. We evaluate the method with ColBERTv2 and Jina v4 on MS MARCO and BEIR, and with ColPali2 on the ViDoRe v3 visual document retrieval benchmark. Experiments cover four training-free compression methods, multiple vector budgets, in-domain adaptation, and transfer from MS MARCO to BEIR. In the in-domain setting, the…

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

  Laura Yao, Shixing Yu, Jie Peng, Promit Ghosal, Kyra Gan +1

  Explaining a vision--language model (VLM) prediction requires tracing training-example influence to the multimodal evidence that mediates it, yet existing attribution methods largely operate only at the sample level. Recent latent-mediated influence methods provide feature-level attribution in language models, but lack the multimodal and spatial structure needed to ground influence in VLMs. We introduce MOSAIC (Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization). It learns a shared sparse image--text latent space whose concepts are 1) sparse within each modality, 2) supported on contiguous image regions, 3) aligned to the phrases that describe them, and 4) approximately decorrelated. It then applies EK-FAC to decompose training-example influence through these concepts and localize their contributions to specific text phrases and image regions. We evaluate MOSAIC on Qwen and Gemma across captioning, chest X-ray understanding, and visual question answering. The learned features are more localized and coherent without degrading task behavior, and ablating influence-ranked latents changes model outputs more than ablating latents ranked by activation or…

  PDF

  Summarize this
• arXiv cs.NIOct 9

  Semantic Edge-Cloud Communication for Real-Time Urban Traffic Digital Twins

  Murat Arda Onsu, Arman Dogru, Poonam Lohan, Burak Kantarci, Aisha Syed

  Urban traffic systems are becoming increasingly complex, demanding real-time traffic perception, proactive collision prediction, and intelligent management to improve safety and reduce congestion. Although AI-driven traffic management has shown strong promise, its success relies on diverse, high-quality data spanning critical scenarios such as collisions, near-collisions, and traffic rule violations. Digital twins offer a powerful solution by enabling synchronized virtual replicas of physical traffic environments for real-time monitoring, collision prediction, traffic-flow optimization, and diverse realistic scenario generation. However, continuously synchronizing a high-fidelity urban traffic digital twin through edge surveillance cameras requires substantial edge-cloud communication, and transmitting raw video streams leads to severe bandwidth and latency constraints. To overcome this challenge, this paper proposes a semantic communication-based edge-cloud framework for real-time urban traffic digital twin applications. By leveraging edge intelligence to extract and transmit only task-relevant semantic traffic information instead of full video feeds, the proposed framework…

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Visualizing the World Model in an Agent's Value Function

  Shawheen Naderi, Fatemeh Akherati, Yubei Chen

  Could model-free reinforcement learning algorithms actually be learning a model of the environment? In this work, we apply counterfactual interventions which hold everything about a state fixed, besides a targeted change, to explore if the value function of a Proximal Policy Optimization agent forms a world model of the Crafter environment. Despite only being trained to optimize reward, we discover that the value function learns concepts such as the rules governing when a diamond is collectible, water and food having an impact on health, heuristics regarding inventory, and more. Finally, we discuss the safety implications of our visualizations and some shortcomings of current reinforcement learning agents that will prevent us from building truly advanced machine intelligence.

  PDF

  Summarize this
• arXiv cs.LGOct 9

  ULPack: Reference-Conditioned Lossless Compression of Checkpoint Trajectories

  Aniket Srinivasan Ashok

  Studying model training requires checkpoint trajectories, whose storage cost limits temporal resolution. Although adjacent checkpoints are numerically close, existing lossless codecs either operate on XOR bit patterns or ignore trajectory-specific conditional structure. We show that a reference weight's exponent predicts the bit length of its displacement in ordered floating-point space without side information. We introduce \textbf{ULPack}, a lossless codec which exploits this dependence by conditioning entropy models of ordered-integer residuals on the reference exponent. Across bfloat16, float16, and float32 trajectories, ULPack reduces storage by up to 27\% against ZipNN, 21\% against zstd-19, and 11.5\% against FM-Delta. Across two dense trajectories, compression improves as updates shrink and reference conditioning strengthens, while near uniform low-order bits set the remaining compression floor and earlier checkpoints add little information.

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Unconstrained CTI-REALM with Open-Weights

  Sahil Garg, Ruhi Gottumukkala, Vishal Agarwal, Shubham Vishwakarma

  Organisations that want language-model agents for security work face a choice between open-weight models, which they can run and audit themselves, and far more expensive closed frontier models. We study that choice on CTI-REALM (arXiv:2603.13517), a benchmark in which an agent turns a threat-intelligence report into a working detection by querying a live telemetry store. We evaluate twenty-six models, nineteen open-weight and seven closed, under a common harness, scorer and panel of LLM judges. Our main finding is that an open-weight model can match the strongest closed models at less than a tenth of their cost, provided it is given enough turns. The benchmark's published limit on agent messages is a confound: a task that exhausts the limit scores zero on every checkpoint, including those it had already passed, so results at that limit partly measure which models finish in time. At the published limit the leading open-weight and closed models cannot be distinguished. With the limit relaxed, the models it was truncating recover, and the highest rewards belong to open-weight models. Among the leading models, price is a poor guide to capability: models with similar rewards differ in…

  PDF

  Summarize this
• arXiv cs.DLOct 9

  Do you "see" what I "see"? A Multi-panel Visual Question Answer Dataset for Large Language Model Chart Analysis

  Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie

  Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable, accessible, and reusable remains an open challenge. Large language and multimodal models (LLMs/LMMs) provide the possibility to automate many figure parsing tasks such as determining data trends or extracting data from plots. In this work, we consider if these models can reliably support task-oriented visual data analysis when presented with synthetic, but realistic, scientific figures that contain multiple charts. Our Visual Question Answering (VQA) process scaffolds our tasks along a four-level description model from basic conceptual answers to interpretive and analytical understanding. Additionally, we outline our process for synthetically generating our single and multi-panel (charts in which there are multiple panels present) plots and testing the data with various LMMs using quantitative analysis. In general, we find LMMs become less accurate with increasing figure complexity as the number of panels, number of plot elements, and difficulty of questions increase. Finally, because LMM accuracy is expected to change rapidly, our…

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Can Language Models Play the Wikipedia Game?

  Alexander Hickey, Joanna Knight, James Bishop

  Language models increasingly support or automate information retrieval from document collections too large to index. This task is difficult when it requires specialist linguistic or scientific knowledge. We used the Wikipedia Game to test language models on an analogous information retrieval task. In the game, players must reach a target article using only article links. We compared sentence embedding models and LLMs across five Wikipedia languages and five academic-literature datasets. We construct very large local graphs of Wikipedia pages and of paper citations using OpenAlex and provide code to build and navigate them. This allows the study of language models' navigation capabilities without relying on third-party APIs. We compared the models with human and TF-IDF benchmarks. The Wikipedia Game format enables robust comparisons across techniques, languages, and domains. No model significantly outperformed humans, although the best models (GPT-o3 mini, GPT-4o, QwQ:32B, and DeepSeek-R1:70B) achieved comparable success. Performance and rankings varied across languages and topics, with implications for practitioners utilising these techniques.

  PDF

  Summarize this
• arXiv cs.CVOct 9▲ 23

  Towards Physically Grounded Articulated 3D Assets: A Multi-Agent Pipeline and Benchmark

  Xudong Cai, Ruijie Zhu, Haoliang Shang, Ruining Li, Marc Pollefeys +3

  In this paper, we study physically grounded articulated 3D asset generation---moving the goal from reproducing how objects look to capturing how they move and function. Inspired by the literature on procedural and agentic generation, we present Artisan, a system that extends the latest method Articraft by introducing a multi-agent pipeline with physical simulation feedback, enabling comprehensive attributes beyond geometry, appearance, and physically plausible articulation, such as collision-free motion, stability under gravity, and accurate mass and material properties. Furthermore, we construct a unified benchmark spanning four key aspects: spatial fidelity, physical plausibility, downstream task performance, and overall quality. The benchmark encompasses 100 reference assets from ArtVIP and LightWheel, 50 stability prompts, and 50 ABO objects with reference masses, with protocols evaluating various metrics. Through extensive evaluation of more than 10 representative models, Artisan achieves competitive spatial fidelity while attaining the lowest collision rates and highest stability rates among the evaluated methods. Together, our system and benchmark establish insights for…

  Code
• arXiv cs.ROOct 9

  Splat2Act: From Captured Scenes to Personalized VLA Manipulation

  Xiaomeng Ye, Krishna Jaganathan, Gyubeum Lim, Gunjan Chhablani, Muhammad Zubair Irshad +1

  A user who brings a generalist manipulation policy into their home or office wants it to pick up their objects from their shelves, counters, and tables. Yet even a strong generalist like $π_{0.5}$ achieves 0% pickup success in our captured deployment scenes without adaptation, and collecting real demonstrations in every user's environment is prohibitively expensive. We present Splat2Act, an automated real-to-sim-to-real pipeline that personalizes a pretrained vision-language-action (VLA) policy to a specific scene and object set using only a handheld phone scan and images of the user's objects. Unlike prior real-to-sim work that reconstructs individual tabletops, Splat2Act operates on building-scale indoor environments: it builds a photorealistic digital twin with 3D Gaussian Splatting and collision geometry, automatically discovers tabletop and non-tabletop receptacles, constructs feasible pickup tasks, and generates successful demonstrations with a privileged simulation oracle, with only scene and object capture required from a human. Applied to 101 scanned campus environments with a median navigable area of 104.87 m$^2$ and 7 receptacles, the pipeline yields 852 receptacles…

  PDF

  Summarize this
• arXiv cs.CVOct 9▲ 27

  InstantFusion: A Shared Latent Interface for Heterogeneous Image Generators

  Xuanlang Dai, Jiazi Bu, Yujie Zhou, Yuhong Liu, Beichen Zhang +3

  Diffusion/flow models have emerged as the de facto paradigms for image generation, giving rise to a diverse family of modern generators with increasingly heterogeneous architectures and representations. Despite their remarkable generation capabilities, the cross-model alignment of latent representations and the consistency of their underlying denoising dynamics remain underexplored. In this work, we uncover an intriguing cross-model compatibility: independently trained generators exhibit similar latent spatial organization, while directly handing off intermediate states between unaligned models can still preserve coarse visual content. Motivated by these observations, we propose InstantFusion, a lightweight shared latent interface that connects the denoising trajectories of heterogeneous image generators. Specifically, to enable flexible cross-model state translation, model-specific Latent AutoEncoders (LAEs) are introduced to project intermediate states from different generators into a shared latent space and decode them into the latent space of a target generator. This shared space is constrained by two complementary objectives: aligning intermediate states derived from the…

  Code
• arXiv cs.CVOct 9

  Images Talk Back: Guiding MMDiTs by Suppressing Image-to-Text Feedback

  Yucheng Yang, Xiaoxiao Sun, Shusheng Yang, Jiakang Chen, Serena Yeung-Levy

  Multimodal diffusion transformers (MMDiTs) allow the evolving image to update internal text representations through joint attention. These representations then guide subsequent image updates, raising a question: does this feedback help correct deviations from the prompt, or preserve them? We find that image-to-text (I2T) feedback can preserve the objects and attributes already represented in the evolving image, including errors. Transplanting I2T messages can rescue failures or disrupt successes. At matched intervention strengths, suppressing the influence of I2T feedback improves generation, whereas opposite controls degrade it. These findings motivate Semantic Feedback Guidance (SFG), which suppresses the influence of I2T feedback by guiding away from an auxiliary branch that strengthens this feedback and weakens text-to-image conditioning. The in-place variant, iSFG, directly suppresses I2T feedback and strengthens text-to-image conditioning, requiring no extra model evaluations. Temporal interventions motivate running the SFG branch during the first 20% of image sampling steps, requiring about $1.2\times$ the base NFE without classifier-free guidance (CFG). Experiments across…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  Hide and Seek: HIDE Concepts to SEEK explanations for black box models

  Bhanu Tokas, Rahul Nair, Rockwell Jackson, Hannah Kerner

  Interpreting machine learning model predictions is a central challenge, particularly for black-box vision models. Concept-based attribution methods interpret models using human-understandable concepts. Existing concept-based methods can only interpret classification and regression models. They require access to model weights or large concept-annotated datasets, and most methods cannot provide image-level attribution. We propose Hide and Seek (HnS), a new method for concept-level attribution that does not require access to model weights or labeled concept datasets. HnS can interpret a wide range of vision models. Given a concept and image set, HnS retrieves concept-aligned images using a vision-language model, localizes the concept within each image, masks the localized region, and measures attribution from the change in the black-box model's output between the original and masked images. This enables faithful attribution by directly measuring the model's behavior and supporting local attributions. We validated HnS on CelebA and COCO-Stuff. Compared to existing methods, HnS showed better agreement with ground-truth attributions. We demonstrated the applicability of HnS on diverse…

  PDF

  Summarize this
• arXiv math.ATOct 9

  Constructive Bounds for the Topological Expressivity of Neural Networks

  Kosio Beshkov

  The expressivity of a neural network $Φ$ describes the class of functions that a network can express. The term topological expressivity refers to a family of measures particularly suited to understand binary classification and is often formulated in terms of the Betti numbers of a decision boundary $β(Φ^{-1}(0))$. Despite its popularity, this measure is not as useful for regression and other problems that do not involve a decision boundary. Instead, we propose to study topological expressivity in terms of the Betti numbers of the output of a neural network $β(Φ(X))$. We formally prove that a ReLU network with architecture $(d,\sum\limits_{k=1}^{d } β_k(3k+2),\sum\limits_{k=1}^{d}β_k(k+1))$ can express spaces homotopy equivalent to wedge sums of spheres with arbitrary Betti numbers. This gives upper bounds for any arbitrary connected ($β_0=1$) sequence of Betti numbers, but by constructing spaces homotopy equivalent to products of $N$ spheres of dimension $d_1,...,d_N$ we achieve much better bounds $(\sum\limits_{n=1} d_n, 3\sum\limits_{n=1}\limits^N d_n + 2N, \sum\limits_{n=1}\limits^N d_n + N) $ for Betti numbers given by $\prod_n (1+x^{d_n})$. Finally we conjecture a much…

  PDF

  Summarize this
• arXiv cs.ROOct 9

  Chronos Policy: Learning Robot Policies on Physical Time

  Changhao Wang, Jaden Clark, Shuran Song

  Robot policies often convert sensing and control into fixed-rate observation-action sequences, leaving physical time implicit in sequence position. This assumption breaks when observation rates or delays vary, or when inference latency changes. We introduce Chronos Policy, which makes physical time explicit by conditioning on observation capture times and desired action execution times. We study two ways to incorporate time into a diffusion transformer: adding timestamp embeddings to token features, or encoding physical time in positional encoding through time-aware rotary positional encodings (time-aware RoPE). We find that time-aware RoPE is more robust, allowing attention to depend directly on physical time while keeping token content unchanged. During training, we temporally randomize which observations are sampled and when the action chunk begins, exposing the policy to a wider range of sensing and execution schedules that may occur during deployments. This also enables learning from asynchronous, multi-rate data without forcing all measurements onto a fixed temporal grid. Across simulation and real-world experiments, Chronos Policy improves performance on dynamic tasks,…

  PDF

  Summarize this

• Anomaly detectors27+92.9%
• AI tutors15+36.4%
• Multimodal emotion8+14.3%
• Channel state information140.0%
• Medical image segmentation146−20.2%
• Machine learned interatomic36−25.0%
• CAD models12−25.0%
• Character recognition OCR9−25.0%