Research

• Reliable request execution in streaming video generation1 paper raises it · 0 address it

  Models often fail to follow incremental user requests while maintaining state consistency in real-time video generation.

  • StreamEval: Evaluating Interactive Streaming Video Generation Modelsraises
• Calibration and contrast-group accuracy in decision models1 paper raises it · 0 address it

  Aggregate accuracy hides failures on controlled variations and unreliable confidence estimates in decision-making systems.

  • DecisionPivot: A Diagnostic Benchmark for Decision Models in LLM Systemsraises
• Length generalization in state tracking1 paper raises it · 1 addresses it

  Models struggle to maintain accuracy on problems much longer than their training data without explicit progress alignment.

  • PAR: Progress-Aligned Recurrence for Length Generalization in State Trackingraises · addresses
• Token collapse in graph soft prompting1 paper raises it · 0 address it

  Graph-specific tokens lose discriminative power, reducing the method to a generic prompt with no graph information.

  • On the Token Collapse Phenomenon in Textual Graph Soft Promptingraises
• Coupled evaluation of multimodal agents and world models1 paper raises it · 0 address it

  Separate evaluation of agents and world models fails to predict success when they are combined in real-world tasks.

  • Dual Exam: Controlled Evaluation of Multimodal Agents and World Modelsraises

Named by an AI from these papers' abstracts.

• Anomaly detectors27+92.9%
• AI tutors15+36.4%
• Multimodal emotion8+14.3%
• Channel state information140.0%
• Medical image segmentation146−22.3%
• CAD models12−25.0%
• Character recognition OCR9−25.0%
• Physics informed neural119−26.5%

• arXiv cs.LGOct 9

  Continuous-Time Cloth Dynamics with Physics-Informed Neural ODEs

  Yingdong Ru, Lipeng Zhuang, Zhuo He, Gerardo Aragon Camarasa

  We present ContinuousCloth, a continuous-time, physics-informed neural solver for deformable cloth dynamics that enforces a spring--mass ODE within a neural trajectory model. Unlike classical time-stepping simulators that return trajectories only on a discrete integration grid, ContinuousCloth supports querying at arbitrary timestamps and computing displacement, velocity, and acceleration via automatic differentiation. We evaluate ContinuousCloth along two dimensions. In a physics-only setting, ContinuousCloth achieves trajectory accuracy comparable to high-precision solvers across six diverse boundary-condition scenarios while reducing runtime by up to 17.4 times. When fused with sparse and noisy observations, the spring--mass ODE acts as a strong inductive bias for downstream tasks: interpolation and parameter estimation. That is, ContinuousCloth reduces interpolation error by 7.3 times over a data-only neural baseline in simulation and 2.4 times on real-world cloth sequences, and achieves 2.6 times lower parameter error than a differentiable-solver baseline under high-noise conditions, demonstrating consistent robustness under realistic measurement imperfections.

  PDF

  Summarize this
• arXiv math.OCOct 9

  STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization

  Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri, Amina Lamghari

  TREGO, a trust-region Bayesian optimization (BO) method for expensive blackbox problems, has convergence guarantees, but only for exact function values. In high dimension, its performance tends to deteriorate. This paper asks whether a trust-region BO method can be extended to handle noisy evaluations while being competitive in high dimension. We first generalize TREGO into a trust-region framework for noisy BO and extend its convergence analysis to noisy function estimates that are sufficiently accurate with sufficiently high probability. We then propose STREGO, a scalable implementation of this framework. Its global phase selects a batch of candidates by solving a bi-objective problem on the posterior mean of a Gaussian process and the integrated variance reduction. Unlike the predictive variance, this second objective does not favor boundary points merely because they are far from the data. Its local phase minimizes the posterior mean of a second Gaussian process, trained only on points close to the incumbent. STREGO is compared with TREGO, TURBO, BAXUS and NSMA-BO on synthetic and real-world problems of dimension $4$ to $124$, with and without noise. It gives the best average…

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions

  Xiaofan Li, Fengchun Qiao, Xing Gao

  In Model Context Protocol (MCP) systems, LLMs select tools from metadata rather than from their implementations. Names and descriptions summarize what a tool does, while the parameter schema specifies the fields a tool accepts and how they are organized. Our analysis of metadata for 367,174 MCP tools reveals substantial schema variation among functionally similar tools. Many of these tools share names and similar descriptions but expose different parameter schemas. This raises a key question: can parameter-schema differences influence tool selection before any call is made? We study this question through controlled metadata interventions across five LLMs and five tasks. Each comparison changes exactly one metadata factor (server name, tool name, description, or parameter schema) while keeping the others fixed. We balance tool positions and use conditional logistic regression to estimate position-adjusted effects. We find that: (1) parameter-schema changes produce the largest and most consistent selection shifts among the tested interventions. Models tend to favor separate top-level parameters over grouped or composite forms. (2) Descriptions that explain parameters also tend to…

  PDF

  Summarize this
• arXiv cs.LGOct 9

  Decentralized Bandit Learning of Markov Perfect Equilibria

  Felix Höfer, H. Mete Soner

  We study decentralized bandit learning in discounted infinite-horizon, general-sum Markov games with continuous actions. Agents observe the public state, but only their own realized rewards, and use simultaneous perturbation stochastic approximation to update their actions. Under strong monotonicity of the negative running-reward gradients and sufficiently small time steps, we prove that the fully decentralized procedure converges almost surely to the unique stationary Markov perfect equilibrium. We obtain an $O(n^{-1/2})$ convergence rate and present a natural extension to episodic finite-horizon learning.

  PDF

  Summarize this
• arXiv cs.CVOct 9

  TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling

  George Matthew, Guoqiang Zhang, Kenta Niwa

  Accurate sampling from continuous-time generative models can require many model evaluations. Low-step sampling reduces cost, but coarse updates introduce endpoint errors that can accumulate and degrade generation quality. We introduce TraceSampler, a post-training method that corrects these errors without modifying the pretrained model. Rather than optimising low-step updates directly, TraceSampler models endpoint error as a sample-specific prediction problem. A higher-step reference integration over the same interval provides the supervised target, while a compact representation makes high-dimensional error predictable from the sampling trace. This representation combines shared residual structure, trace-derived directions, and a low-rank remainder, so shallow per-step MLP heads predict sample-dependent coefficients. These coefficients drive a two-stage learned residual correction: the first stage predicts the full endpoint residual, while an intermediate backbone evaluation provides new trace information to predict the remaining residual; this evaluation is reused by the following step. TraceSampler is evaluated with Rectified Flow on CIFAR-10, SiT-XL/2 on ImageNet-256, and…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  Real-Time Traffic Density Estimation: A Hybrid Computer Vision and Predictive Machine Learning Pipeline

  Prince Kwabena Appiah Boadu

  Traffic density estimation from camera feeds requires connecting visual perception with short-horizon forecasting. We present a two-stage pipeline that combines vehicle detection and tracking with a spatial logic layer, followed by a temporal estimator. The spatial layer filters bounding boxes by their overlap with a precomputed road mask and extracts per-frame traffic features. LSTM, GRU, and Temporal Convolutional Network (TCN) estimators use these features to predict five future steps from a ten-frame input window. We benchmark ten detector variants from six families on a dataset of 5,716 annotated frames using a single NVIDIA GPU. YOLOv5l achieves the highest mAP@50:95 of 0.524 at 56 frames per second, while YOLOv8n reaches 258 frames per second at 0.487 mAP@50:95; these rates measure detector inference without tracking or spatial filtering. Temporal evaluation on 29,243 held-out windows yields aggregate R-squared values of 0.858 for GRU, 0.856 for LSTM, and 0.840 for TCN. These metrics measure forecasts of detector-derived telemetry and are dominated by direction variance, rather than establishing absolute traffic-density accuracy. No road mask was available for the…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  Manifold Navigation Learning Using Recurrent Networks: Efficient Training Techniques

  Sarah Harkins Dayton, Sathya Rengaswami, Vasileios Maroulas, David L. Boothe, Ioannis D. Schizas

  Position tracking is a well-studied problem in Euclidean navigation, yet comparatively little attention has been given to navigation on general manifolds, where motion is governed by the underlying geometry of the space. In this work, we formalize the problem of manifold navigation as a velocity-driven tracking task and propose a coordinate-chart-based learning framework in which learning models operate entirely in a Euclidean representation. Predicted trajectories are mapped to the manifold via an immersion function prior to error computation, allowing geometric consistency to be evaluated without explicitly enforcing manifold constraints during learning. Within this framework, we focus on Gated Recurrent Units (GRUs) for sequential position prediction on curved manifolds after comparisons with alternative recurrent architectures. We evaluate our GRU models, trained with pertinent custom loss functions, against both a standard MSE loss function and a naive vector-addition baseline. We evaluate our approach on both analytically (and numerically) defined manifolds, as well as a surface reconstructed from a LiDAR point cloud. Our results show that recurrent models can learn to…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference

  Anxhelo Diko, Jieyu Lin, Stefano Ambrogio, Ziyun Li, Mahmoud Assran +3

  Foundation models like V-JEPA are increasingly deployed across a heterogeneous edge-cloud continuum spanning smart glasses, phones, robots, and servers. Yet, they are typically trained as rigid, fixed-capacity networks bound to one device, supporting neither reconfigurable compute budgets nor collaborative execution. Existing approaches address only one axis at a time: slimmable and NAS-based methods provide width elasticity but assume single-device inference, whereas split-computing methods enable cross-device collaboration only for fixed partitions and static capacities. We introduce Elastic V-JEPA (EV-JEPA), a single model trained once and deployable everywhere along the continuum. Its width is reconfigurable, and it is natively partitionable across heterogeneous devices. At its core is Transition-Aware Triplet Training (T3). At each step we pick a narrow and a wide width, and train three paths: each width alone, plus a hybrid path that runs the early layers narrow and the rest wide. The hybrid switch point is where one device hands off to another, so the same weights learn both single-device and cross-device execution. Across various benchmarks, EV-JEPA falls within 1…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  MvFFN: Multi-view Floor plan Feed-forward Network for Unposed Wide-Baseline Panorama Layout Reconstruction

  Yuguang Li, Yichuan Deng, Zixuan Liu, Ivaylo Boyadzhiev, Xiuyuan Wu +3

  Reconstructing accurate camera poses and floor-plan layouts from sparse, unposed, wide-baseline RGB panoramas remains a challenging open problem. Direct multi-room floor-plan prediction is especially difficult in this setting: pose, geometry, semantics, and global layout structure are tightly coupled, yet a complete floor-plan is hard to learn as a single global output. Our key insight is to keep the model's predictions strictly local. Multi-view Ordered Wall Instance Segmentation (MOIS) is designed around this concept. Each panorama column predicts only the wall it observes plus the next two adjacent walls in clockwise order. The global wall partition, room layouts, and shared coordinates emerge from deterministic aggregation of overlapping local predictions. Building on MOIS, we propose Multi-view floor-plan Feed-Forward Network (MvFFN), which jointly predicts coarse camera poses, dense global geometry, and per-wall semantic outputs. A downstream Layout Chaining (LC) post processing pipeline then assembles these predictions into multi-room floor-plans with shared global coordinates, consistent wall identities, and connected room structure. The dense MOIS predictions recover…

  PDF

  Summarize this
• arXiv cs.CLOct 9

  ASSET: Adaptive Speech Segmentation via Reinforcement Learning

  Cihan Xiao, Yuan Tseng, Matthew Wiesner, Desh Raj, Leibny Paola Garcia Perera +2

  Speech large language models (LLMs) are often built using pretrained acoustic encoders as an audio interface to an LLM decoder. However, LLMs scale poorly with input sequence length, while acoustic encoders tend to produce substantially longer sequences than typical textual inputs. This motivates learning to compress encoder outputs while preserving task-oriented information. To this end, we introduce ASSET, a reinforcement learning (RL)-based framework that learns to adaptively compress encoder outputs by first partitioning them into variable-duration segments and then pooling each segment into a continuous audio token. We use group-relative policy optimization (GRPO) to train a segmenter with task performance feedback and a token-rate penalty. In automatic speech recognition (ASR) experiments, ASSET achieves performance comparable to a transcript-assisted character-boundary oracle with approximately 25% fewer audio tokens on English LibriSpeech, while improving on a phone-boundary oracle at a similar token rate on Mandarin AISHELL. Exploratory experiments on non-ASR tasks further illustrate how learned segmentation changes when the framework is adapted to tasks beyond…

  PDF

  Summarize this
• arXiv cs.AIOct 9

  ConicFlow: constrained Inference-time optimal control for flow matching models

  Shervin Khalafi, Kasra Fallah, Sung Cho, Dongsheng Ding, James Anderson +1

  We propose ConicFlow, which frames constrained sampling from a flow matching model at inference time as an optimal control problem that balances deviation from the pretrained trajectory against reward, subject to hard terminal constraints. We discretize this problem and adapt receding-horizon control, optimizing only a short window of the trajectory at each step for computational tractability. We address the resulting sequence of receding-horizon subproblems with a receding-horizon primal-dual algorithm. Its dual multipliers capture the difficulty of satisfying each constraint and are refined throughout sampling, steering generation toward feasible final samples. We further propose a terminal-estimate variant that avoids accumulating local truncation errors by evaluating the reward and constraints at the predicted terminal point rather than at the end of the current horizon. Under standard regularity assumptions, we establish finite-horizon guarantees on suboptimality. Under convex-affine structure, we derive a sharper bound that scales linearly with omitted horizon length, and we show that the terminal-estimate variant is provably tighter when the flow dynamics change slowly.…

  PDF

  Summarize this
• arXiv cs.LGOct 9

  Learning to See through Action: Predictive Curiosity for Robots

  Stelios Zarifis, Panagiotis Filntisis, George Retsinas, Petros Maragos

  Humans and animals do not learn vision from a fixed set of data: they move, interact, and learn from the visual consequences of their actions. Motivated by this developmental view, we study whether robots can improve visual learning by actively collecting experience through curiosity-driven interaction. We introduce DySCOVER, a self-supervised framework for active robot visual learning. A robot observes RGB frames, learns a visual autoencoder that encodes and reconstructs them, and trains a recurrent world model to predict the next visual latent state conditioned on the current state and action. The agent's policy is trained only with intrinsic rewards: reconstruction error encourages visually novel observations, while the world model's error encourages transitions whose consequences are not yet predictable. This prediction error biases the policy toward more intricate trajectories that enrich the agent's set of experiences with observations beyond those encouraged by visual novelty alone. Our experiments conducted with robotic arms as the embodied agents show that adding predictive curiosity improves the vision model over using visual curiosity alone. Across variations in…

  PDF

  Summarize this
• arXiv eess.ASOct 9

  RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can

  Yi-Jen Shih, Puyuan Peng, Haohe Liu, David Harwath, Abdelrahman Mohamed

  Recent Speech Full-Duplex (FDX) models are capable of engaging in naturalistic conversation by listening and speaking at the same time. Building on this foundation, recent architectures introduce internal thinking streams to enhance their intelligence. To guarantee these streams process successfully in real time, existing methods enforce a fixed decoding rate, which assumes a hardware profile capable of a specific latency. We argue this static approach is severely suboptimal. Because internal thinking is intrinsically asynchronous, an ideal FDX model should sustain real-time conversational flow while allowing each stream to decode at its own hardware-adaptive rate. Thinking streams must process as fast as the hardware allows to deliver accurate answers faster. To solve this, we propose RUBATO, a novel multistream positional encoding that assigns independent clocks to entirely decouple the streams. This structural isolation empowers the model to zero-shot interpolate across any inference decoding speed. It drastically simplifies dataset construction by requiring training on only two extreme decoding speeds while smoothly adapting to any intermediate rate during deployment. RUBATO…

  PDF

  Summarize this
• arXiv cs.SEOct 9

  Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances

  Konstantin Ievtushenko, Ralf Lämmel, Jan Jürjens

  High benchmark scores in learning-based vulnerability detection mask severe degradation under distribution shifts. This paper demonstrates that calibrated embedding geometry exposes these hidden generalization boundaries on syntactically similar vulnerable and patched code pairs. An evaluation of CodeBERT and Qwen2.5-Coder across pattern, project, and temporal shifts on a strictly paired dataset reveals three main findings. First, whitened intermediate-layer embeddings with local nearest-neighbour scoring separate unfamiliar samples with a mean out-of-distribution AUROC of 0.739 (RQ1). Second, distance scores reveal a shift-specific reliability gradient where accuracy drops by over 35 percentage points and cross-project shifts cause the sharpest degradation, even though temporal shifts drift geometrically farthest (RQ2). Third, false positives on patched code systematically correlate with geometric proximity to vulnerable training neighbourhoods, increasing from 9.4% in the lowest-risk decile to 58.8% in the highest (RQ3). These calibrated distances establish a diagnostic signal that maps representation geometry directly to model degradation without requiring auxiliary models.

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Controllable Diverse Sampling through Pullback Directions

  Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang, Elham Azizi

  Conditional diffusion models can generate high-quality images that align closely with a given prompt. However, samples generated from different random seeds may remain visually similar, limiting creative exploration. Existing approaches can increase variation, but often require additional optimization or sampling overhead, while stronger perturbations may introduce distortions that degrade image quality or prompt alignment. We propose a training-free method that leverages the local pullback geometry induced by the mapping from the conditioning-embedding space to the image space to identify directions of maximal output sensitivity. By steering different sampling trajectories along these directions, we encourage distinct generations while maintaining consistency with the conditioning signal. Experiments on MS-COCO and BrushNet show the efficacy of the method in improving diversity on prompt-conditioned image generation and inpainting.

  PDF

  Summarize this
• arXiv cs.CVOct 9▲ 30

  GeoFix: Geometry Fixing for 4D Consistent Generation

  Yeobin Hong, Hyojun Go, Dohun Lee, Jeongjae Lee, Sunghwan Hong +5

  Video diffusion models increasingly serve as priors for 4D generation, but the generated videos often lack geometric consistency. Recent approaches address this through post-training by designing geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves geometric consistency and visual quality with a single geometric reward, applicable to both training and inference-time reward alignment. Our reward measures cross-view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field-of-view overlap even in real videos and 4D datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap-dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.

  PDF

  Summarize this
• arXiv cs.LGOct 9

  Layer-wise Input Recoverability in Deep Neural Networks

  Maxwell Spiridonov, Alireza Rahmansetayesh, Pierre Baldi

  The inner workings of deep neural networks are still not well understood. The information bottleneck framework, which characterizes deep networks by the mutual information between their hidden representations and the network's input, has been used to identify compression of input information with respect to an input image and output labels. We instead study how much of a layer's own input is recoverable from the layer's output in convolutional neural networks (CNNs). A convolutional layer applies its filters to one input patch at a time, so we evaluate recoverability on sampled input patches with a reconstruction-based probe and not on the entire feature map. The probe's size thus depends on channels and kernel size and not on feature map size, which lets our method scale to deep CNNs with wide feature maps. For each layer, we compare pretrained, random, and reconstruction-optimized encoders, using linear and linear-plus-residual-MLP decoders. We measure recoverability as $R^2$ between input patches and their reconstructions on a held-out set. Across nine ImageNet CNNs, recoverability generally decreases with depth but is more consistently predicted by the compression of the…

  PDF

  Summarize this
• arXiv cs.CVOct 9

  TGSR: Text-Grounded Diffusion for Faithful Image Super-Resolution

  Zhiming Hu, Ran Zhang, Angela Ye, Raghav Goyal, Faraz Ali +5

  Diffusion models have achieved remarkable success in image super-resolution (SR) by leveraging strong generative priors to enhance perceptual quality. However, for textual content, these priors often produce text hallucinations: non-existent or distorted text, which alter scene semantics and compromise model reliability. While recent methods focus on enhancing text generation, they do so indiscriminately, leaving hallucinations fundamentally unresolved. We argue the core problem is a lack of selective control. Therefore, we reformulate text SR as a selective generation problem: text is resolved only when reliable generation is expected, while other scene content is fully super-resolved. We introduce TGSR (Text-Grounded Super-Resolution), a test-time framework that progressively identifies and suppresses likely hallucinated text regions during generation. To enable this, we train dedicated detectors on a large-scale dataset curated from diverse diffusion-based SR outputs. TGSR consistently improves text fidelity across multiple backbones, increasing recognition precision by up to 17.68 points without degrading perceptual quality, underscoring the value of selective control in…

  PDF

  Summarize this
• arXiv cs.LGOct 9

  The RNN Homunculus: Learning to Gate Long-Term Memory

  Zeyu Fan, Alexandru-Dragos Manolache, Jan van Gemert

  Learning long-range dependencies with recurrent neural networks (RNNs) is difficult because of the vanishing gradient problem. Gated recurrent neural networks like GRUs and LSTMs address this problem and are commonly explained by their ability to create additive copy paths through time. This explanation is correct, but incomplete: useful RNN gate values must themselves be learned, which is typically done with another RNN. We study this gate-learning problem through carefully controlled sequence classification tasks. Our analysis shows that gated architectures struggle when all training samples require long-range memory and thus equally suffer from the vanishing gradient problem. We intervene by adding short-dependency samples which let gated models learn solutions that capture long-dependency samples, but also generalize across different class labels. When properly learned, gated RNNs match or even exceed the test accuracy of modern architectures like Mamba and xLSTM. Overall, our results highlight gate learning as a central part of learning long-range memory and show how training data can support this process by exposing the same task over shorter dependencies.

  PDF

  Summarize this
• arXiv cs.AIOct 9

  Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference

  Xingyu Xie, Xiaoyong Xie, Zhouchen Lin, Kim-Chuan Toh

  Tensor parallelism ends every layer with an all-reduce that sits between two dependent matrix multiplications, so its cost cannot be hidden and grows as compute outpaces interconnect bandwidth. Sending 4-bit instead of 16-bit values would cut this cost by nearly 4x, but the all-reduce must add the values it carries, and rounding the sum injects an error at every call. Model quality depends on how these errors accumulate over layers, yet existing methods treat each call on its own. We present Echo, a 4-bit all-reduce for training and inference that controls the accumulated error. Echo moves data in 4 bits and sums in high precision, so each value is quantized at most twice per call for any number of GPUs. The summation reveals each injected error, and Echo feeds it back into later calls, so old errors cancel instead of accumulating. We prove that the accumulated error stays bounded independently of depth. On A100 GPUs, Echo is 2.09x faster than the 16-bit NCCL all-reduce on 1 GB messages. It speeds up the time to first token by up to 1.16x and the training step by up to 1.09x on an idle interconnect, and by up to 1.44x and 1.38x as background traffic grows. It keeps the average…

  PDF

  Summarize this