Papers

Field

Top tools here

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality scenes when camera poses differ significantly from training views.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack physically plausible object interactions and spatial compatibility.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D geometry.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

3d scene · 3d reconstruction · depth estimation262 papers44 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d209 papers40 this week
remote sensing · satellite imagery · earth observation127 papers24 this week
small target detection · infrared small target · target detection114 papers21 this week
point cloud registration · 3d point clouds · point cloud69 papers19 this week
camera lidar · 3d object detection · point clouds95 papers14 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.CVOct 12
UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models
Lei Yang, Yuner Zhang, Ziyang Liu, Ziying Song, Kai WU +3
Recent advances in multimodal foundation models, exemplified by GPT-6 Astra, raise the prospect of general-purpose embodied reasoning and planning. Using Internet videos for embodied pretraining calls for reliable physical-scale annotations. Yet visual geometry foundation models often recover scene structure and camera motion without reliable metric scale. We show that uncertain size priors of familiar objects can jointly constrain a reconstruction's shared scale. Building on this insight, we present UniMetric, a universal and training-free metric scale recovery framework for geometry foundation models, without additional sensors or task-specific metric supervision. Specifically, Metric Prior Acquisition elicits object height intervals from a vision-language model. Semantic-Geometric Alignment measures reconstructed object heights along the estimated support-surface normal in each frame and converts the priors into scale intervals. Global Scale Recovery reconciles these intervals through a convex objective, yielding a jointly feasible scale when they overlap and a unique compromise otherwise. The estimated sequence-level scale is applied uniformly to reconstructed points and…
PDF arXiv
arXiv cs.CVOct 12
Mining Cross-Modal Perception Gaps in Autonomous Driving: When Camera and LiDAR Disagree
Hongsen Cao, Mona Jaber, Shanxin Yuan, Ahmed Sayed
Multi-modal fusion for 3D object detection assumes that cameras and LiDAR are broadly complementary. In practice, how much each modality is trusted at a given location is decided by learned weights that are neither characterized nor auditable. We label every ground-truth object by which modalities detected it, then mine the resulting 0.4M-object database across nuScenes, KITTI, and Waymo. Because the patterns shift with distance from the ego vehicle, we read the mined model through Spatially Stratified Feature Attribution (SSFA), which conditions Shapley attribution on distance zones and exposes structure that global attribution averages away. Three assumptions built into the current fusion design do not hold. The modality-preference boundary tracks the object itself: a traffic cone is nearly six times as likely as a truck to be seen by the camera alone (16.8% vs. 2.9%), regardless of range. Camera-only detections peak at 10-30 m, inside the band usually ceded to LiDAR, and fall away on either side. The drivers of failure are not additive either: the distance - density interaction alone outweighs the entire main effect of viewing direction. We further find that modality dominance…
PDF arXiv
arXiv cs.CVOct 12
The Midas Touch for Metric Depth: Interpretable Metricization for Generalizable Depth Perception
Mengtan Zhang, Yu Ma, Shiqi Jiang, Yi Feng, Wei Ye +4
Recent foundation models have drastically improved the generalization of monocular relative depth estimation, yet recovering accurate metric depth from their estimates for depth perception tasks remains challenging due to scale ambiguity and local metric-scale inconsistency. Existing approaches recover metric depth from sparse observations using either explicit global affine transformation algorithms or implicit data-driven depth completion networks. The former cannot accommodate spatially varying relative-to-metric relationships, while the latter are sensitive to the distribution of sparse observations and may not fully exploit generalizable relative depth priors. To simultaneously address these issues, this study extends our previous work MTD and proposes MTDv2, a hybrid metric depth perception framework that combines explicit depth metricization with data-driven metric depth refinement. The explicit depth metricization process performs segment-wise affine transformations to recover spatially varying metric scales while preserving the geometric structure of relative depth, followed by pixel-wise depth refinement that propagates metric observations along paths with minimal…
PDF arXiv
arXiv cs.CVOct 12
3D-VITAL: Visibility-aware Identity Training from Artificial Liftings of 3D Reconstruction model for Cross-View Object Re-Identification
William Grolleau, Astrid Sabourin, Guillaume Lapouge, Catherine Achard
We address the Cross-View Object Re-Identification (ReID) task, aiming to match identities across drastically different viewpoints. We focus on Single-View Aerial-Ground ReID (SV AG-ReID), where models train on one viewpoint and must generalize to another at inference. To bridge this viewpoint gap, related work employs generative models to synthesize novel views from source images as additional training data. We leverage recent advances in large-scale single-view 3D mesh reconstruction to lift source images into textured meshes and to render them from new viewpoints. However, this pipeline produces unreliable data in two distinct ways. First, the reconstruction may fail and produce geometrically incorrect meshes, resulting in severe corruption in rendered novel views. Second, even when the geometry is correct, the rendered novel views contain hallucinated textures in regions not visible in the source image. The reconstructed mesh carries a per-pixel partition between regions visible from the source view and regions that the reconstruction model had to hallucinate, available as a deterministic byproduct of rendering. We introduce this per-pixel visibility map and exploit it in a…
PDF arXiv
arXiv cs.CVOct 12
Dense Endoluminal Mapping and Localisation from Monocular Bronchoscopy
Salvatore Esposito, Matias Mattamala, Daniel Rebain, Liang Zhao, Kev Dhaliwal +2
Bronchoscopy inserts a camera through the airways to diagnose endoluminal disease. Locating the camera along the airway usually requires additional equipment, such as an electromagnetic tracker and a preoperative CT scan. In this paper, we investigate the use of the monocular camera stream to localise the bronchoscope without requiring additional sensing. We introduce a monocular SLAM system that localises the tip of a bronchoscope and reconstructs the airway surface around it from RGB video alone. A Depth Anything 3 backbone, fine-tuned on over a million photorealistic airway renderings, returns metric per-pixel depth and is the system's only source of scale. DINOv3 features routed through a custom cross-attention matcher supply sub-pixel correspondences on mucosa where classical descriptors fail. Both enter a factor graph alongside a dense constraint from projective iterative closest point (ICP) registration against the system's own running reconstruction, so surface evidence stabilises the directions the visual factors leave weakly observable. On six airways rendered from patient CT scans, the system reaches 26.1 mm mean absolute trajectory error and 77.5% mean surface…
PDF arXiv
arXiv cs.CVOct 12
Recovering from Pose-Selection Failures in Repeated Visual Localization
Jaehyun Kim, Wonseok Kang, Tae-Wan Kim
Repetitive structures can give an incorrect pose candidate more inlier support than an accurate one, causing visual localization to select it. History-Guided Pose Reselection (HPR) uses coarse position forecasts from up to five past localization outputs to distinguish well-separated current-query candidates. The best-fitting forecast proposes an alternative; HPR accepts it only if every well-fitting forecast is much closer to it than to the localizer's pose. HPR needs no odometry and returns the selected candidate unchanged, using prediction for selection rather than pose refinement. With one development-set threshold, HPR improves recall within 5 meters and 10 degrees in all 12 main configurations of Hierarchical Localization (HLoc), MASt3R-based pose generation, and JIST-based retrieval, losing successes on fewer than 0.4 percent of queries. On NAVER Gangnam B1, HPR corrects 97 HLoc failures and loses six successes; gains persist within 0.25 meters and 2 degrees on all three HLoc test sets. Median HPR runtime on HLoc candidates is below one millisecond, excluding pose generation. Recovery weakens under less frequent localization.
PDF arXiv

Papers

Topics in this field

3d scene · 3d reconstruction · depth estimation262 papers44 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d209 papers40 this week
remote sensing · satellite imagery · earth observation127 papers24 this week
small target detection · infrared small target · target detection114 papers21 this week
point cloud registration · 3d point clouds · point cloud69 papers19 this week
camera lidar · 3d object detection · point clouds95 papers14 this week

Active contributors · 90 days

Activity · 12 weeks

Coming with more sources: repos

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality scenes when camera poses differ significantly from training views.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack physically plausible object interactions and spatial compatibility.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D geometry.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Recent papers

All in Research →

arXiv cs.CVOct 12
UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models
Lei Yang, Yuner Zhang, Ziyang Liu, Ziying Song, Kai WU +3
Recent advances in multimodal foundation models, exemplified by GPT-6 Astra, raise the prospect of general-purpose embodied reasoning and planning. Using Internet videos for embodied pretraining calls for reliable physical-scale annotations. Yet visual geometry foundation models often recover scene structure and camera motion without reliable metric scale. We show that uncertain size priors of familiar objects can jointly constrain a reconstruction's shared scale. Building on this insight, we present UniMetric, a universal and training-free metric scale recovery framework for geometry foundation models, without additional sensors or task-specific metric supervision. Specifically, Metric Prior Acquisition elicits object height intervals from a vision-language model. Semantic-Geometric Alignment measures reconstructed object heights along the estimated support-surface normal in each frame and converts the priors into scale intervals. Global Scale Recovery reconciles these intervals through a convex objective, yielding a jointly feasible scale when they overlap and a unique compromise otherwise. The estimated sequence-level scale is applied uniformly to reconstructed points and…
PDF arXiv
arXiv cs.CVOct 12
Mining Cross-Modal Perception Gaps in Autonomous Driving: When Camera and LiDAR Disagree
Hongsen Cao, Mona Jaber, Shanxin Yuan, Ahmed Sayed
Multi-modal fusion for 3D object detection assumes that cameras and LiDAR are broadly complementary. In practice, how much each modality is trusted at a given location is decided by learned weights that are neither characterized nor auditable. We label every ground-truth object by which modalities detected it, then mine the resulting 0.4M-object database across nuScenes, KITTI, and Waymo. Because the patterns shift with distance from the ego vehicle, we read the mined model through Spatially Stratified Feature Attribution (SSFA), which conditions Shapley attribution on distance zones and exposes structure that global attribution averages away. Three assumptions built into the current fusion design do not hold. The modality-preference boundary tracks the object itself: a traffic cone is nearly six times as likely as a truck to be seen by the camera alone (16.8% vs. 2.9%), regardless of range. Camera-only detections peak at 10-30 m, inside the band usually ceded to LiDAR, and fall away on either side. The drivers of failure are not additive either: the distance - density interaction alone outweighs the entire main effect of viewing direction. We further find that modality dominance…
PDF arXiv
arXiv cs.CVOct 12
The Midas Touch for Metric Depth: Interpretable Metricization for Generalizable Depth Perception
Mengtan Zhang, Yu Ma, Shiqi Jiang, Yi Feng, Wei Ye +4
Recent foundation models have drastically improved the generalization of monocular relative depth estimation, yet recovering accurate metric depth from their estimates for depth perception tasks remains challenging due to scale ambiguity and local metric-scale inconsistency. Existing approaches recover metric depth from sparse observations using either explicit global affine transformation algorithms or implicit data-driven depth completion networks. The former cannot accommodate spatially varying relative-to-metric relationships, while the latter are sensitive to the distribution of sparse observations and may not fully exploit generalizable relative depth priors. To simultaneously address these issues, this study extends our previous work MTD and proposes MTDv2, a hybrid metric depth perception framework that combines explicit depth metricization with data-driven metric depth refinement. The explicit depth metricization process performs segment-wise affine transformations to recover spatially varying metric scales while preserving the geometric structure of relative depth, followed by pixel-wise depth refinement that propagates metric observations along paths with minimal…
PDF arXiv
arXiv cs.CVOct 12
3D-VITAL: Visibility-aware Identity Training from Artificial Liftings of 3D Reconstruction model for Cross-View Object Re-Identification
William Grolleau, Astrid Sabourin, Guillaume Lapouge, Catherine Achard
We address the Cross-View Object Re-Identification (ReID) task, aiming to match identities across drastically different viewpoints. We focus on Single-View Aerial-Ground ReID (SV AG-ReID), where models train on one viewpoint and must generalize to another at inference. To bridge this viewpoint gap, related work employs generative models to synthesize novel views from source images as additional training data. We leverage recent advances in large-scale single-view 3D mesh reconstruction to lift source images into textured meshes and to render them from new viewpoints. However, this pipeline produces unreliable data in two distinct ways. First, the reconstruction may fail and produce geometrically incorrect meshes, resulting in severe corruption in rendered novel views. Second, even when the geometry is correct, the rendered novel views contain hallucinated textures in regions not visible in the source image. The reconstructed mesh carries a per-pixel partition between regions visible from the source view and regions that the reconstruction model had to hallucinate, available as a deterministic byproduct of rendering. We introduce this per-pixel visibility map and exploit it in a…
PDF arXiv
arXiv cs.CVOct 12
Dense Endoluminal Mapping and Localisation from Monocular Bronchoscopy
Salvatore Esposito, Matias Mattamala, Daniel Rebain, Liang Zhao, Kev Dhaliwal +2
Bronchoscopy inserts a camera through the airways to diagnose endoluminal disease. Locating the camera along the airway usually requires additional equipment, such as an electromagnetic tracker and a preoperative CT scan. In this paper, we investigate the use of the monocular camera stream to localise the bronchoscope without requiring additional sensing. We introduce a monocular SLAM system that localises the tip of a bronchoscope and reconstructs the airway surface around it from RGB video alone. A Depth Anything 3 backbone, fine-tuned on over a million photorealistic airway renderings, returns metric per-pixel depth and is the system's only source of scale. DINOv3 features routed through a custom cross-attention matcher supply sub-pixel correspondences on mucosa where classical descriptors fail. Both enter a factor graph alongside a dense constraint from projective iterative closest point (ICP) registration against the system's own running reconstruction, so surface evidence stabilises the directions the visual factors leave weakly observable. On six airways rendered from patient CT scans, the system reaches 26.1 mm mean absolute trajectory error and 77.5% mean surface…
PDF arXiv
arXiv cs.CVOct 12
Recovering from Pose-Selection Failures in Repeated Visual Localization
Jaehyun Kim, Wonseok Kang, Tae-Wan Kim
Repetitive structures can give an incorrect pose candidate more inlier support than an accurate one, causing visual localization to select it. History-Guided Pose Reselection (HPR) uses coarse position forecasts from up to five past localization outputs to distinguish well-separated current-query candidates. The best-fitting forecast proposes an alternative; HPR accepts it only if every well-fitting forecast is much closer to it than to the localizer's pose. HPR needs no odometry and returns the selected candidate unchanged, using prediction for selection rather than pose refinement. With one development-set threshold, HPR improves recall within 5 meters and 10 degrees in all 12 main configurations of Hierarchical Localization (HLoc), MASt3R-based pose generation, and JIST-based retrieval, losing successes on fewer than 0.4 percent of queries. On NAVER Gangnam B1, HPR corrects 97 HLoc failures and loses six successes; gains persist within 0.25 meters and 2 degrees on all three HLoc test sets. Median HPR runtime on HLoc candidates is below one millisecond, excluding pose generation. Recovery weakens under less frequent localization.
PDF arXiv

Fields · AI · updated 18m ago

3D scene

Found by the topic model: 3d scene, gaussian splatting 3dgs, remote sensing.

162papers this week−12.4% vs last week

All papers · 876 With code · 27

Newest Most relevant

806 papers this month