Research

• Comprehensive and reliable benchmarking for AI capabilities3 papers raise it · 3 address it

  Current benchmarks often lack the depth to fully characterize model capabilities, especially for complex, real-world, or long-horizon tasks.

  • SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Modelsraises · addresses
  • Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimizationraises · addresses
  • One Table, Many Queries: Benchmarking Generalist Tabular Models Across Generation, Imputation, and Predictionraises · addresses
• Effective AI agents in complex, dynamic real-world environments3 papers raise it · 3 address it

  AI agents struggle with real-world tasks requiring adaptation, learning from limited data, or debugging live systems, limiting practical deployment.

  • Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoningraises
  • World In-Context Learning for Agentic Robot Manipulationraises · addresses
  • Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimizationraises · addresses
  • Sage-Taiyi: A Metacognitive Architecture for Scientific Discoveryaddresses
• Efficient and scalable generation of complex data2 papers raise it · 2 address it

  Generating high-quality, complex data like streaming video or diverse tabular data efficiently and with fewer computational resources is crucial.

  • Few-Step Streaming Video Generation via Condition Shiftingraises · addresses
  • One Table, Many Queries: Benchmarking Generalist Tabular Models Across Generation, Imputation, and Predictionraises · addresses
• Unified models for diverse data modalities and tasks2 papers raise it · 2 address it

  Creating single models that handle diverse tasks or data variations within a complex domain reduces complexity and improves generalization.

  • Prompt2Echo (P2Echo): Text-Prompted Multi-View Echocardiography Segmentation with One Modelraises · addresses
  • One Table, Many Queries: Benchmarking Generalist Tabular Models Across Generation, Imputation, and Predictionraises · addresses
• Reducing computational cost of language model attribution1 paper raises it · 1 addresses it

  Understanding how prompt parts influence language model output is crucial for interpretability, but direct methods are computationally expensive.

  • Sketching Language Models for Context Attributionraises · addresses

Named by an AI from these papers' abstracts.

• arXiv cs.CLRAGSep 19

  From Lexical Baselines to Agentic Retrieval-Augmented Generation: Structured Skill and Responsibility-Level Extraction with the SFIA Framework

  Ranuga Disansa, U. S. Samarasinghe, Lasith Gunawardena

  Automated skill extraction underpins workforce planning, yet most systems represent skills as flat labels with no notion of the responsibility level at which a skill is practiced. The Skills Framework for the Information Age (SFIA) captures exactly this dimension, defining 147 professional skills across seven responsibility levels, but no automated LLM-based extraction targeting SFIA has been reported. We formalize the task as structured prediction of (skill, level) pairs from free text and ask three questions: how accurately can text be mapped onto SFIA's closed vocabulary, which strategies reliably predict the level alongside the skill, and do agentic designs improve on simpler retrieval and prompting? We evaluate five strategies (a lexical baseline, dense retrieval with LLM reranking, a zero-shot schema-constrained LLM, single-agent agentic RAG, and a three-agent retriever--matcher--verifier crew) against expert-mapped European ICT role profiles, all drawing on an SFIA~9 corpus built by a fully automated agentic pipeline that we release. Retrieval-based matching identifies the most skills while generative strategies are markedly more precise; only strategies assigning the…

  AIAgents70PDF

  Summarize this
• arXiv cs.CVRoboticsSep 19

  Image Frame Dynamic Object Segmentation and Ego Motion Estimation using Radar Image Fusion

  Astik Srivastava, Suhani Grover, Avinash Sharma, Madhava Krishna

  Dynamic object segmentation and ego-motion estimation are closely coupled problems in autonomous driving, as accurate ego-motion estimation typically requires static scene observations, while identifying static observations requires knowledge of the ego motion. We present Radar-Dot, a radar--RGB framework that exploits radar Doppler measurements to address this coupling. Radar returns are first used to estimate ego velocity through a linear Doppler constraint, with residual-based static/dynamic segmentation and robust estimation used to reduce the influence of moving objects. The estimated motion is then combined with metric depth and dense optical flow to identify image regions whose observed motion is inconsistent with the rigid scene motion. Experiments on 10 nuScenes scenes (part of nuscenes-mini) demonstrate that the resulting geometric pipeline achieves 20.24% dynamic IoU and 33.67% F1-score over 394 frame pairs, while radar-based static-point filtering improves ego-velocity estimation compared with using all radar returns. These results demonstrate the potential of radar as a modality for jointly improving ego-motion estimation and dynamic object segmentation.

  AIComputer vision70PDF
• arXiv stat.MLSep 19

  FREESIA: Covariance-Aware Posterior Transport for Expressive and Scalable Data Assimilation

  Shiwei Ni, Yangwen Zhang, Hang Qi, Xiaofei Guan, Lili Ju

  Data assimilation aims to infer the state of complex dynamical systems based on observational data. However, accurate inference of the multimodal posteriors induced by nonlinear or non-injective observation operators remains a key challenge under high-dimensional and sparse observation conditions. Ensemble filters scale to high dimensions but are confined by restrictive distributional assumptions, while training-free generative filters (e.g., EnSF, EnFF) alleviate this limitation but may introduce structural errors and hinder information propagation under sparse observations. To address these issues, we propose a training-free, asymptotically exact posterior transport method. Firstly, a covariance-aware posterior transport scheme is designed, which embeds the forecast cross-covariance into flow-based transport and accurately recovers unobserved states while preserving the non-Gaussian posterior structure. Furthermore, the method combines a tractable observation-adaptive proposal with posterior correction, ensuring accurate approximation of the nonlinear posterior distribution. Finally, we establish the corresponding posterior flow theory, from which the asymptotic exactness of…

  PDF

  Summarize this
• arXiv cs.RORoboticsSep 19

  SmoLSTM: A Compact Vision-Language-Action Model with Recurrent Memory that Persists

  Jan-Gerrit Habekost, Parsa Mastouri Kashani, Connor Gäde, Matthias Kerzel, Philipp Allgeuer +3

  Vision-language-action models often predict actions from only the current observation, which can leave tasks involving object occlusion or visually identical objects ambiguous without episode history. The usual countermeasure, widening the observation window, turns the horizon into a hyperparameter and lets per-step cost grow with it. We instead capture the episode in a recurrent state. SmoLSTM couples a frozen 256M-parameter SmolVLM backbone to a matrix-memory LSTM control layer in which observation tokens and action queries are unified in a single causal stream that is never reset throughout the entire episode. Recurrent-state storage is therefore O(1) in episode length. A flow-matching action head predicts chunks of 10 end-effector pose deltas and gripper commands at each control step. Our single policy, trained jointly on 7,461 demonstrations across 140 tasks and evaluated on held-out initial states, performs best, reaching 85.1% subgoal coverage and 77.5% full-task success on LIBERO-Mem with 0.04B trainable parameters, surpassing both the benchmark's own object-centric baseline and recent memory-based approaches. Resetting the recurrent state at every control step reduces…

  AIMultimodal80PDF
• arXiv cs.SDMultimodalSep 19

  Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs

  Jing Peng, Zichao Nie, Zhisheng Zhang, Jingran Xie, Zhiyong Wu

  Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio understanding remains debated. Existing benchmarks often focus on narrow domains or evaluate encoders outside LALM contexts. To address these gaps, we systematically evaluate continuous and discrete representations across speech, sound and music. Utilizing our UniARC framework with dual evaluation strategies across model scales from SmolLM2-135M to Llama-3-8B, we analyze the dynamic relationships of data volume, model capacity, and computational efficiency. Our results reveal the pivotal role of semantic constraints in tokenization for audio understanding and demonstrate that scaling backbones fail to compensate for information loss in audio representation, especially in data-limited tasks. These findings offer practical guidance for balancing semantic density, fidelity, and efficiency in future LALMs.

  AIVoice AI80PDF

  Summarize this
• arXiv cs.LGSep 19

  Same Outcome, Different Readout: What Does a Steerable Valence Direction in LLMs Represent?

  Weihan Li, Xinlei Chen, Yuhan Song, Xiaofeng Lin, Tianshi Zheng

  Claims about what an internal direction in an LLM represents need evidential constraints beyond an observer's prior beliefs about the system. Decodability and successful activation steering do not, by themselves, establish which construct the direction tracks. This gap is especially consequential for welfare-relevant interpretations, where a proposed functional state must be distinguished from correlated features of the extraction contrast. We treat the question as one of construct validity and study a good-bad outcome direction in a maze task, using controlled interventions that separate the realised outcome from the informational history through which it became known. Across multiple LLM checkpoints, directions fitted on one explicit outcome encoding transfer well to another, indicating that the readout is not tied to surface form. When the same realised outcome is reached through announced and unannounced histories, however, transfer degrades substantially: even after both histories receive the same explicit outcome, the post-event readout remains strongly conditioned on the earlier announcement. In the base model, steering along the direction changes actions, yet removing it…

  PDF

  Summarize this
• arXiv cs.CVComputer visionSep 19

  LINGO: Latent Initialization and Gradient Optimization for Sparse-view X-ray Novel View Synthesis and CT Reconstruction with 3D Gaussian Splatting

  Lifeng Xing, Dequan Jin, Kunpeng Bu, Peigeng He, Shihui Ying

  In novel view synthesis and Computed Tomography (CT) reconstruction with sparse-view X-ray imaging, insufficient angular coverage leads to structural ambiguity and accumulated noise. Integrating 3D Gaussian Splatting (3DGS) with X-ray absorption physics can achieve promising results, but it suffers from noisy initialization, positional insensitivity, and weak gradients in low-density regions. In this paper, we propose a unified Latent Initialization and Gradient Optimization (LINGO) framework to address these issues. LINGO combines latent mask-space initialization with dynamic gradient optimization to improve point cloud structural completeness while accelerating training. It constructs voxel-level 3D filters from X-ray masks to robustly suppress background noise and provide reliable geometric priors. By employing an adaptive voxel scaling strategy and dynamically scaling loss, LINGO can adjust spatial resolution and explicitly amplify gradients in low-density structures. To evaluate the quality of initialization, we introduce the Initialization Point Cloud Structural Deviation (IPSD) metric. Experiments on the X3D dataset indicate that for the novel view synthesis task, LINGO…

  AIImage & video generation50
• arXiv eess.SYSep 19

  Intelligent Degradation Monitoring in Lithium-ion Batteries via Discharge Incremental Capacity Feature Estimation

  Amir Madmolilvand, Farzaneh Abdollahi

  Accurate and timely detection of degradation in lithium-ion batteries is crucial to ensure safety, reliability, and longevity in high-demand applications such as electric vehicles and energy storage systems. Traditional incremental capacity (IC) analysis methods require low-current cycling for discharge measurements, limiting their practical use in real-time battery management. This paper proposes a novel neural network-based framework that predicts discharge IC features directly from charging signals, eliminating the need for low-current discharge. Trained on a comprehensive dataset of 53 battery cells cycled under diverse fast-charging protocols, the model demonstrates robust generalization ability, effectively estimating degradation indicators on unseen battery data. Among several architectures evaluated, the LSTM model provides the best balance of prediction accuracy and computational efficiency. The proposed approach enables real-time integration into Battery Management Systems (BMS), enhancing degradation monitoring without disrupting normal battery operation. This paper presents a study to bridge IC analysis with practical, fast-charging scenarios, marking a significant…

  PDF

  Summarize this
• arXiv cs.CLAI safetySep 19

  Alignment Forecasting: Predicting Misalignment From Training Data

  Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak

  Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge, and today it is caught only after training, by auditing the resulting model. To complement post-hoc audits, we introduce Alignment Forecasting: the task of predicting alignment failures before training. Given a target model, a fine-tuning dataset, and a failure mode such as deception or sycophancy, a forecaster outputs the probability that fine-tuning would meaningfully increase that failure mode. To measure progress on alignment forecasting, we introduce ALIGNMENTFORECASTBENCH, a benchmark of over 5,000 forecasting questions spanning 17 target models, 32 datasets, and 16 failure modes. Frontier models prompted directly perform poorly on ALIGNMENTFORECASTBENCH. We therefore propose a forecasting scaffold in which an LLM reads the dataset and rates how strongly and broadly it pushes the model toward misbehavior, and a simple learned model combines that rating with the failure mode's base rate and the target model's prior tendency. This forecasts well above chance, and beats a model fine-tuned on the…

  AIEvals80PDF

  Summarize this
• arXiv cs.GTSep 19

  A Horizon-Independent Regret Bound for Optimistic Hedge in General-Sum Games

  Junsoo Ha

  Can simple learning rules keep their regret bounded in self-play? Recent work achieves constant regret bounds through modified regularization and higher-order prediction. Yet for Optimistic Hedge, arguably the most canonical method in games, the best known individual regret bound remains logarithmic. In this work, we prove that plain Optimistic Hedge with a constant step size can attain $O_{n,d}(1)$ individual regret in general-sum games with $n$ players and $d=(d_1,\ldots,d_n)$ actions, under expected loss-vector feedback. As a corollary, its time-averaged play enjoys an $O_{n,d}(1/T)$ coarse correlated equilibrium (CCE) gap. Our analysis represents Optimistic Hedge as a real-analytic recurrence on a compact space, which yields an exact finite-order difference relation that eliminates horizon dependence. Our proof hinges on nonconstructive Noetherianity argument of Frisch (1967), so the $(n,d)$-dependence remains implicit.

  PDF

  Summarize this
• arXiv cs.LGSep 19

  A Hybrid Attention Model Learning Unified Time-aware Patch Representation for Irregular Multivariate Time Series Forecasting

  Li Lin, Zhihao Lin, Qi Zhang, Kaiwen Xia, Shuai Wang +1

  Time series foundation models (TSFMs) have recently delivered impressive zero-shot performance across diverse forecasting tasks. However, real-world decision-making frequently relies on \emph{irregular multivariate time series} (IMTS), where inconsistent inter-observation intervals and asynchronous sampling across variables coexist with informative missingness. Existing TSFMs handle such inputs either through imputation that injects spurious values or through index-based positional encodings that ignore continuous time. There is still a gap in the foundation model that follows the original IMTS patterns. In this paper, we propose a hybrid attention model that learns a unified time-aware patch representation for IMTS forecasting. We first design a \emph{time-aware patch encoding} that maps a variable number of intra-patch timestamps into a fixed-size embedding, producing a uniform format for irregular patches without resorting to imputation. We then introduce a \emph{time bias attention} mechanism that calibrates inter-patch temporal misalignment and asynchronous cross-channel dependencies as auxiliary attention offset. Finally, on top of a decoder-only Transformer backbone, we…

  AI30PDF

  Summarize this
• arXiv cs.CVSep 19

  SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery

  Changhao Zhao, Linglin Zeng, Hai Liu

  Open-vocabulary semantic segmentation (OVS) of remote sensing imagery is a challenging pixel-level task requiring strong generalization and adaptation to the spatial characteristics of remote sensing data. Although existing vision-language foundation models perform well in general domains, their image-level classification design weakens the spatial priors needed for high-resolution remote sensing segmentation: structural spatial relations are degraded during deep feature transformation, and fine-grained spatial details are lost during downsampling. To address these complementary deficiencies, we propose SatOV, a training-free framework for open-vocabulary remote sensing segmentation that restores spatial priors at two stages of the representation pipeline. Specifically, Residual QQ Attention (ResQQ) extracts Query-Key self-attention from an intermediate CLIP layer and fuses it with final-layer Query-Query attention via a residual combination, restoring structural spatial priors suppressed by the final-layer representation. Spatially Modulated Upsampling (SatUp) uses the original high-resolution RGB image as spatial guidance, combining spatial feature modulation with guided…

  PDF

  Summarize this
• arXiv cs.LGSep 19

  Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients

  Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon

  We study personalized federated reinforcement learning, in which $n$ agents, each acting in its own Markov decision process, collaborate through a server to learn a shared MAML-style policy initialization that becomes effective for an individual agent once that agent adapts it with a single local policy-gradient step. We propose Per-FedAvg-PG, in which agents take $τ$ local stochastic meta-policy-gradient steps between communication rounds, and prove that it reaches an $\varepsilon$-approximate first-order stationary point of the personalized objective in $K=\mathcal O(\varepsilon^{-3/2})$ rounds with $τ=Θ(\varepsilon^{-1/2})$ local steps. The analysis rests on a structural feature of the reinforcement learning setting: under standard policy-class regularity, the per-agent objectives have uniformly bounded gradients and Hessians with explicit constants, so the bounded-gradient and bounded-heterogeneity conditions imposed by the supervised theory hold automatically and no separate heterogeneity assumption is needed. The exact meta-gradient requires the inner-loop policy Hessian, which our experiments identify as the practical bottleneck. We therefore analyze the Hessian-free…

  AI30PDF

  Summarize this
• arXiv cs.CVSep 19

  Modernising the Compressed-Domain Video Captioner: A Controlled Study of SigLIP2 and GPT-2 Substitutions

  Ashim Nepal, Ashok B. K

  Compressed-domain video captioning avoids full video decoding by operating directly on I-frames, motion vectors and residuals, trading a small amount of accuracy for a large gain in inference speed. CoCap established this pipeline using a CLIP vision encoder and a shallow BERT-style multimodal decoder. Both components predate substantially stronger alternatives. We ask a narrow, controlled question: how much of CoCap's accuracy is limited by these two components, and which of the two is the binding constraint? We replace the CLIP I-frame encoder with SigLIP2 and the BERT-style decoder with GPT-2, and evaluate three configurations (the original pairing, the encoder substitution alone, and both substitutions together) under identical data, sampling budget and optimisation schedule. All comparisons are made against our own reproduction of CoCap rather than its published numbers, because we train on a 4,999-clip subset of VATEX at a reduced sampling budget; absolute values are therefore not comparable with the original work. Our reproduction tracks the published result closely: CIDEr and METEOR run slightly above it (54.9 against 52.7; 23.4 against 23.2), BLEU-4 and ROUGE-L slightly…

  PDF

  Summarize this
• arXiv quant-phSep 19

  From IceCube to IT-Sphere: A Hybrid Quantum-Classical GNN for Banking IT Root Cause Analysis

  Antonio Greco, Riccardo Paoletti, Roberto Cappuccio, Mario Onorato

  We present Hybrid Quantum Root Cause Analysis (HQ-RCA), an industrially grounded workflow for root cause analysis in banking IT operations, built on a hybrid Quantum Graph Neural Network (QGNN): the classical backbone of DynEdge (the IceCube neutrino-reconstruction GNN, which we call standalone DynEdge), with its classification head replaced by a Variational Quantum Circuit (VQC). On 13 months of anonymised IT data (13k alarm clusters) from a major European bank, the hybrid QGNN matches standalone DynEdge -- the strongest classical baseline -- on $F_1$, while standalone DynEdge leads the ranking metrics. A readout-sensitivity and layout-robustness study, analysed via Dimensional Expressivity Analysis (DEA), shows that the effective parameter dimensionality (rank) of the quantum observable has no measurable correlation with $F_1$; we therefore keep the simplest readout $\langle Z_0\rangle$ (the Pauli-$Z$ expectation on the first qubit), which in the deployed layout is rank-1, collapsing optimisation to a 1-D problem solvable by a gradient-free grid scan. Execution on IBM Heron r2 (no error mitigation) shows this gradient-free readout is executable on NISQ hardware after threshold…

  PDF

  Summarize this
• arXiv cs.LGEvalsSep 19

  Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecasting

  Xu Lin, Runheng Zuo, Shengxuan Xu, Qitai Tan, Hongyu Lin +1

  Average squared error cannot reveal whether forecasting performance degrades because the future becomes less predictable or because forecasts move farther from the conditional mean. We introduce paired, mechanism-controlled stress tests that decompose changes in expected squared error at each lead time into environmental risk and forecast-oracle distance, using an origin-conditioned predictive oracle unavailable to the evaluated models. Three end-to-end controls have known attribution. Specifically, the null, environmental-only, and information-gap controls verify that the pipeline assigns changes to the correct component. We then apply the benchmark to 24 deployable forecasters. Under frequent switching, 14 methods have higher realized MSE but lower oracle distance; under outlier-variance feedback, 19 have higher MSE but lower scale-standardized MSE. Short- and long-lead stress-response rankings have Spearman correlation 0.624, revealing substantial horizon-dependent reordering. We then study multivariate relation shifts. Across six models and three coupling severities, oracle distance accounts for only 0.7-3.9% of the decomposed expected-risk increase, and environmental-risk…

  AI60PDF

  Summarize this
• arXiv cs.LGAgentsSep 19

  Counterfactual Tool Ranking under Utility, Cost, and Privilege Constraints

  Jiapeng Li

  Counterfactual tool evaluation must distinguish authority, historical support, and what a comparison actually estimates. We study these distinctions with eleven executable enterprise-inspired tools, exact-propensity logs, and real local Model Context Protocol transport. An initial 45-run synthetic study is retained, then challenged by 30 realized-return control runs and 15 experiments on 1,930 independently released Berkeley Function Calling Leaderboard (BFCL) tasks. Full-return direct regression reverses an initially favorable doubly robust (DR) evaluation result in the linear setting: mean absolute errors are 0.0139 for direct regression and 0.0272 for DR. Under a shifted environment, DR retains an advantage, with errors 0.0227 versus 0.0948. On function-name-group-disjoint BFCL-derived splits, direct and DR selectors obtain balanced accuracies of 81.85% and 79.83%. Two pinned local Qwen2.5 models are evaluated on the same 200 held-out tasks, exposing a strong failure to abstain under the fixed prompt. We further characterize policy differences under missing support: unsupported actions shared by two policies cancel, allowing point identification of an incremental change when…

  AIEvals50PDF

  Summarize this
• arXiv cs.CRSep 19

  The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents

  Pritom Bhowmik

  Memory-poisoning defenses for LLM agents are typically evaluated by their ability to prevent attacks. However, the traffic they process is rarely adversarial. The cost of implementing a defense is paid with each interaction, while its benefits are only seen in a small percentage of cases. We developed a measurement setup that keeps the memory backend, retrieval process, and judge consistent across different conditions, changing only the defense itself. We test each condition three times across five conversations to distinguish the defense's real effects from noise inherent in the pipeline's runs, which remains significant even at temperature zero. Across three write-time defenses (input sanitization, provenance checking, and LLM-based anomaly detection) and one read-time defense (reranking), tested on entirely benign traffic, the write-time defenses show no utility cost we can resolve, with 95% confidence intervals spanning roughly +/-4.5 points and including zero. The reranker is different: it lowers core accuracy by 4.4 points (95% CI [-9.0,-0.05], bootstrap; McNemar p=0.064), a result that survives replication but sits at the edge of our resolution. Its clearer cost is…

  PDF

  Summarize this
• arXiv cs.LGSep 19

  FIRM-WM: State-factorized factual-interventional recurrent modeling for reward-free visual planning

  Yilun Wu, Yunjian Zhang, Aobo Li, Mujiangshan Wang, Haitao Wu +1

  Reward-free latent world models can learn from offline videos and solve new image--goal tasks by optimizing actions through predicted latent futures. This setting places two demands on the planning state: its coordinates must be comparable with a goal image. Moreover, its dynamics must retain velocity, motion trend, contact, and other history--dependent information beyond those goal coordinates. Offline training creates a second mismatch: each recorded trajectory reveals one factual future, whereas a sampling--based planner compares many actions that were not taken from the same state. We introduce FIRM-WM (Factual--Interventional Recurrent World Model), a compact pixel world model designed around these two gaps. Its recurrent state separates a typed, goal--comparable configuration from a 128-dimensional dynamic fiber used for prediction but excluded from the terminal goal cost. Broad factual trajectories provide state coverage, while common--reset intervention branches provide observed outcomes for alternative action sequences. Before executing each branch, we reset the environment and restore the same recorded values exposed by the environment's state--setting interface. Under…

  PDF

  Summarize this
• arXiv cs.ROSep 19

  Commonsense-Grounded Path Planning from Abstract Instructions

  Masafumi Endo, Kohei Honda, Ryo Yonetani

  We present \emph{commonsense ranked search} (CoRS), a novel path planner that turns an abstract instruction into a route that follows commonsense. While existing methods respect the considerations written down in advance, a robot working among people must follow those left unstated too, as with a wet floor that a worker avoids without being told. CoRS leverages large language models (LLMs) and vision-language models (VLMs) as commonsense knowledge to reason about these latent considerations in its planning. Given an abstract instruction (\emph{e.g.}, ``move carefully'') and visual observations of each region in the environment, CoRS derives a consideration for each region, as in ``this wet floor is slippery and worth a detour.'' It then compares the considerations between regions to see which of the two the robot should avoid more, as in ``the crowd is worse than the wet floor.'' These judgments sort the regions into a commonsense ranking, whose costs drive a conventional search that always returns a valid route. We build a benchmark for planning under latent considerations, with three environments, 1350 problems, and five instructions at three levels of abstraction. Experiments…

  PDF

  Summarize this

• Spiking neural networks11+175.0%
• Quantum machine learning34+25.9%
• Character recognition OCR44+25.7%
• Speech emotion23+21.1%
• Voice AI76+13.4%
• Forecasting time133−2.9%
• Robotics195−3.0%
• Channel state information18−5.3%