Papers

Field

Top tools here

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking realism for interactive or dynamic use.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality views when camera poses differ greatly from training inputs.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Dense correspondence matching without spatio-temporal priors1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity, limiting applications like image editing.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Physics and geometry coherence in 3D scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack fine-grained spatial and physical plausibility between interacting objects.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy, a fundamental requirement for many 3D tasks.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

3d scene · 3d reconstruction · depth estimation268 papers41 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d212 papers36 this week
small target detection · infrared small target · target detection122 papers26 this week
remote sensing · satellite imagery · earth observation133 papers23 this week
point cloud registration · 3d point clouds · point cloud72 papers16 this week
camera lidar · 3d object detection · point clouds96 papers14 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.ROSep 28
mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar
Junqiao Fan, Yuxuan Hu, Bofan Lyu, Yanshuo Lu, Pengfei Liu +5
Assistive robots increasingly operate in many human-centered environments and perform various human-robot interaction (HRI) tasks, such as object delivery. However, most existing HRI systems rely on RGB cameras that continuously observe humans to respond to non-verbal commands, such as hand gestures. This raises privacy concerns in privacy- critical environments, such as hospital wards or restaurants, where direct camera observation of humans is restricted. To develop privacy-preserving HRI, we leverage millimeter-wave (mmWave) radar, which can sense human motion through privacy barriers without identifiable imagery. We propose mmHRI, the first multi-modal robot manipulation framework that achieves mmWave radar-guided privacy-preserving HRI. mmHRI introduces two key designs to mitigate the sparsity and temporal inconsistency of radar data in cluttered robot manipulation environments. First, we propose a dual-stream architecture that jointly learns from unfiltered raw radar tensors and radar point clouds to estimate both human actions and 3D poses. To mitigate signal inconsistency, mmHRI further incorporates a memory-based state-space model (MSSM) that retains historical radar…
PDF arXiv
arXiv cs.CVSep 28
AGILE-GS: Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting
Amirhossein Mollaei Khass, Nader Motee
Radiance fields need hundreds of views, and their placement matters as much as their number. Next-best-view (NBV) selection for 3D Gaussian Splatting (3DGS) usually scores every candidate in the pool and keeps one. Searching for information and choosing a camera, however, are separable problems. We present AGILE-GS, an anchor-guided NBV method that separates the two. A virtual anchor pose is optimized on SE(3) by Riemannian gradient ascent on expected information gain. It need not be reachable or in the pool; it marks where the model is most uncertain. Candidates are scored against the anchor's viewing geometry, and a greedy ridge-leverage step distills the pool into a small, non-redundant shortlist without rendering any candidate. The shortlist can be used in two ways. AGILE-GS takes the first view on it as the next view, so no Fisher information is computed for any candidate. AGILE-GS+ computes the Fisher information gain of each shortlisted view and picks the best, so the expensive evaluation runs on a handful of views rather than the whole pool. On standard benchmarks and in closed-loop embodied acquisition, both match or exceed existing baselines while cutting selection…
PDF arXiv
arXiv cs.AISep 28
A Differentiable Optimization Framework for Registering Sequential Bounding Boxes with Point Cloud Stream
Xuesong Li, Jinguang Tong, Jie Hong
Refining a sequence of coarse 3D bounding boxes against a LiDAR point-cloud stream demands tracks that are geometrically accurate (high IoU) and temporally coherent (low roughness), preferably without training data. The usual recipe keeps the two concerns apart: register each frame independently, then smooth the trajectory afterwards with a Kalman~RTS or Savitzky--Golay filter. Smoothing displaces boxes from a geometric optimum and never re-optimises, so it trades accuracy for smoothness. We instead fold the temporal smoothness constraint into a training-free registration objective and solve for all poses jointly with L-BFGS. The payoff depends on how well the object is seen. On well-observed tracks it is large: within the low-roughness budget, the joint objective beats both post-hoc smoothers on paired multi-seed statistics and cuts roughness several-fold relative to frame-wise registration at matched accuracy. Treating visibility as an experimental variable exposes the limit. The advantage decays monotonically as views become one-sided, until it is indistinguishable from zero for near-edge-on objects and slightly negative under a ray-cast simulator with range-dependent density…
PDF arXiv
arXiv cs.CVSep 28
Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata
Yuzhuo Li, Di Zhao, Tingrui Qiao, Yihao Wu, Bo Pang +1
Identifying individual animals is crucial for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision have shown promise in animal re-identification (Animal ReID) by leveraging data from camera traps. However, existing Animal ReID datasets rely exclusively on visual data, overlooking environmental metadata that ecologists have identified as highly correlated with animal behavior and identity, such as temperature and circadian rhythms. Meanwhile, modern vision-language models (VLMs) offer rich multimodal reasoning capabilities, but existing resources underutilize their text-processing potential. To address these limitations, we propose MetaWild, a multimodal Animal ReID dataset comprising 20,890 images across six species, paired with environmental metadata extracted from embedded camera trap overlays and scene contexts. Additionally, to facilitate the use of metadata in existing ReID methods, we propose the Meta-Feature Adapter (MFA), a lightweight module that can be incorporated into existing VLM-based ReID methods, allowing ReID models to leverage both environmental metadata and visual information to improve ReID performance. Experiments on…
PDF arXiv
arXiv cs.CVSep 27
3D Point Tracking with State Space Models
Masahiro Ogawa, Qi An, Atsushi Yamashita
Tracking any point of a dynamic scene in metric 3D - in absolute meters, not up to an unknown scale - underpins 3D and 4D reconstruction, robot navigation, and autonomous driving, where decisions are made in meters, not pixels. Our objective is a 3D point tracker accurate in those absolute terms and operating within a single commodity GPU, pose-free, monocular budget. Our method rests on one observation: once a point's 2D image trajectory is fixed, the quantity that governs its metric accuracy is the depth along its pixel ray. Rather than learning tracking end-to-end, we therefore compose two frozen front-ends - dense optical flow for 2D correspondence and a monocular metric-depth network for the third dimension - and learn only the residual they cannot supply: that depth, refined by a compact state space model (Mamba-3) conditioned on appearance features (DINOv3). A state space model rather than the transformers the strongest 3D trackers adopt is what makes a single-GPU budget attainable: it summarises a track in a fixed-size recurrent state whose memory cost is constant in the number of frames, whereas attention requires a key-value cache that grows linearly with them. On the…
PDF arXiv
arXiv cs.LGSep 27
Structure-Adaptive Tree Field Integrators
Millend Roy, Soham Samal, Ivan Zelich, Krzysztof Marcin Choromanski
We present a new class of near-linear algorithms for efficiently integrating general tensor fields defined on trees with distance dependent kernels, the Structure-Adaptive Tree Field Integrators (STAD-TFIs). STAD-TFIs exploit the tree's underlying structure through decompositions built around path backbones and single vertex separators, and use two-dimensional fast Fourier transforms to compute interactions jointly. By exploiting this structural information, STAD-TFIs achieve more computationally efficient integration than their regular efficient tree field integrators (TFI) counterparts. We provide a detailed theoretical analysis of our proposed approach and complement it with an exhaustive empirical evaluation, ranging from speed tests on synthetic trees, through accelerated Sinkhorn-based relaxations of the Optimal Transport algorithms on real meshes, to Topological Attention Transformers for vision tasks. To the best of our knowledge, we provide some of the first results showing that efficient to compute and accurate relaxations of the geodesic Sinkhorn-based solutions of the Optimal Transport problem can be derived by applying fast TFI methods.
PDF arXiv

Papers

Topics in this field

3d scene · 3d reconstruction · depth estimation268 papers41 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d212 papers36 this week
small target detection · infrared small target · target detection122 papers26 this week
remote sensing · satellite imagery · earth observation133 papers23 this week
point cloud registration · 3d point clouds · point cloud72 papers16 this week
camera lidar · 3d object detection · point clouds96 papers14 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.ROSep 28
mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar
Junqiao Fan, Yuxuan Hu, Bofan Lyu, Yanshuo Lu, Pengfei Liu +5
Assistive robots increasingly operate in many human-centered environments and perform various human-robot interaction (HRI) tasks, such as object delivery. However, most existing HRI systems rely on RGB cameras that continuously observe humans to respond to non-verbal commands, such as hand gestures. This raises privacy concerns in privacy- critical environments, such as hospital wards or restaurants, where direct camera observation of humans is restricted. To develop privacy-preserving HRI, we leverage millimeter-wave (mmWave) radar, which can sense human motion through privacy barriers without identifiable imagery. We propose mmHRI, the first multi-modal robot manipulation framework that achieves mmWave radar-guided privacy-preserving HRI. mmHRI introduces two key designs to mitigate the sparsity and temporal inconsistency of radar data in cluttered robot manipulation environments. First, we propose a dual-stream architecture that jointly learns from unfiltered raw radar tensors and radar point clouds to estimate both human actions and 3D poses. To mitigate signal inconsistency, mmHRI further incorporates a memory-based state-space model (MSSM) that retains historical radar…
PDF arXiv
arXiv cs.CVSep 28
AGILE-GS: Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting
Amirhossein Mollaei Khass, Nader Motee
Radiance fields need hundreds of views, and their placement matters as much as their number. Next-best-view (NBV) selection for 3D Gaussian Splatting (3DGS) usually scores every candidate in the pool and keeps one. Searching for information and choosing a camera, however, are separable problems. We present AGILE-GS, an anchor-guided NBV method that separates the two. A virtual anchor pose is optimized on SE(3) by Riemannian gradient ascent on expected information gain. It need not be reachable or in the pool; it marks where the model is most uncertain. Candidates are scored against the anchor's viewing geometry, and a greedy ridge-leverage step distills the pool into a small, non-redundant shortlist without rendering any candidate. The shortlist can be used in two ways. AGILE-GS takes the first view on it as the next view, so no Fisher information is computed for any candidate. AGILE-GS+ computes the Fisher information gain of each shortlisted view and picks the best, so the expensive evaluation runs on a handful of views rather than the whole pool. On standard benchmarks and in closed-loop embodied acquisition, both match or exceed existing baselines while cutting selection…
PDF arXiv
arXiv cs.AISep 28
A Differentiable Optimization Framework for Registering Sequential Bounding Boxes with Point Cloud Stream
Xuesong Li, Jinguang Tong, Jie Hong
Refining a sequence of coarse 3D bounding boxes against a LiDAR point-cloud stream demands tracks that are geometrically accurate (high IoU) and temporally coherent (low roughness), preferably without training data. The usual recipe keeps the two concerns apart: register each frame independently, then smooth the trajectory afterwards with a Kalman~RTS or Savitzky--Golay filter. Smoothing displaces boxes from a geometric optimum and never re-optimises, so it trades accuracy for smoothness. We instead fold the temporal smoothness constraint into a training-free registration objective and solve for all poses jointly with L-BFGS. The payoff depends on how well the object is seen. On well-observed tracks it is large: within the low-roughness budget, the joint objective beats both post-hoc smoothers on paired multi-seed statistics and cuts roughness several-fold relative to frame-wise registration at matched accuracy. Treating visibility as an experimental variable exposes the limit. The advantage decays monotonically as views become one-sided, until it is indistinguishable from zero for near-edge-on objects and slightly negative under a ray-cast simulator with range-dependent density…
PDF arXiv
arXiv cs.CVSep 28
Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata
Yuzhuo Li, Di Zhao, Tingrui Qiao, Yihao Wu, Bo Pang +1
Identifying individual animals is crucial for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision have shown promise in animal re-identification (Animal ReID) by leveraging data from camera traps. However, existing Animal ReID datasets rely exclusively on visual data, overlooking environmental metadata that ecologists have identified as highly correlated with animal behavior and identity, such as temperature and circadian rhythms. Meanwhile, modern vision-language models (VLMs) offer rich multimodal reasoning capabilities, but existing resources underutilize their text-processing potential. To address these limitations, we propose MetaWild, a multimodal Animal ReID dataset comprising 20,890 images across six species, paired with environmental metadata extracted from embedded camera trap overlays and scene contexts. Additionally, to facilitate the use of metadata in existing ReID methods, we propose the Meta-Feature Adapter (MFA), a lightweight module that can be incorporated into existing VLM-based ReID methods, allowing ReID models to leverage both environmental metadata and visual information to improve ReID performance. Experiments on…
PDF arXiv
arXiv cs.CVSep 27
3D Point Tracking with State Space Models
Masahiro Ogawa, Qi An, Atsushi Yamashita
Tracking any point of a dynamic scene in metric 3D - in absolute meters, not up to an unknown scale - underpins 3D and 4D reconstruction, robot navigation, and autonomous driving, where decisions are made in meters, not pixels. Our objective is a 3D point tracker accurate in those absolute terms and operating within a single commodity GPU, pose-free, monocular budget. Our method rests on one observation: once a point's 2D image trajectory is fixed, the quantity that governs its metric accuracy is the depth along its pixel ray. Rather than learning tracking end-to-end, we therefore compose two frozen front-ends - dense optical flow for 2D correspondence and a monocular metric-depth network for the third dimension - and learn only the residual they cannot supply: that depth, refined by a compact state space model (Mamba-3) conditioned on appearance features (DINOv3). A state space model rather than the transformers the strongest 3D trackers adopt is what makes a single-GPU budget attainable: it summarises a track in a fixed-size recurrent state whose memory cost is constant in the number of frames, whereas attention requires a key-value cache that grows linearly with them. On the…
PDF arXiv
arXiv cs.LGSep 27
Structure-Adaptive Tree Field Integrators
Millend Roy, Soham Samal, Ivan Zelich, Krzysztof Marcin Choromanski
We present a new class of near-linear algorithms for efficiently integrating general tensor fields defined on trees with distance dependent kernels, the Structure-Adaptive Tree Field Integrators (STAD-TFIs). STAD-TFIs exploit the tree's underlying structure through decompositions built around path backbones and single vertex separators, and use two-dimensional fast Fourier transforms to compute interactions jointly. By exploiting this structural information, STAD-TFIs achieve more computationally efficient integration than their regular efficient tree field integrators (TFI) counterparts. We provide a detailed theoretical analysis of our proposed approach and complement it with an exhaustive empirical evaluation, ranging from speed tests on synthetic trees, through accelerated Sinkhorn-based relaxations of the Optimal Transport algorithms on real meshes, to Topological Attention Transformers for vision tasks. To the best of our knowledge, we provide some of the first results showing that efficient to compute and accurate relaxations of the geodesic Sinkhorn-based solutions of the Optimal Transport problem can be derived by applying fast TFI methods.
PDF arXiv

Activity · 12 weeks

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking realism for interactive or dynamic use.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality views when camera poses differ greatly from training inputs.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Dense correspondence matching without spatio-temporal priors1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity, limiting applications like image editing.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Physics and geometry coherence in 3D scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack fine-grained spatial and physical plausibility between interacting objects.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy, a fundamental requirement for many 3D tasks.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 24m ago

3D scene

Found by the topic model: 3d scene, gaussian splatting 3dgs, remote sensing.

156papers this week−13.8% vs last week

All papers · 903 With code · 28

Newest Most relevant

797 papers this month