Papers

Field

Top tools here

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality scenes when camera poses differ significantly from training views.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack physically plausible object interactions and spatial compatibility.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D geometry.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

3d scene · 3d reconstruction · depth estimation261 papers46 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d207 papers41 this week
remote sensing · satellite imagery · earth observation127 papers27 this week
point cloud registration · 3d point clouds · point cloud69 papers22 this week
small target detection · infrared small target · target detection113 papers21 this week
camera lidar · 3d object detection · point clouds94 papers15 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.CVOct 17
Directionally Factorized Illumination with Spherical Harmonic Rotation
Yimeng Dou, Adam M. Saunders, Andre T. S. Hucke, Michael E. Kim, Adam W. Anderson +3
Many real-world capture scenarios involve an active light source rigidly attached to the camera: camera flashes for dark-environment imaging, underwater vehicle lights, and endoscopic/laparoscopic medical procedures. As the camera moves, the co-mounted light moves with it, causing the illumination direction at each surface point to change across views. Neural rendering methods (NeRF, Gaussian Splatting) encode this pose-dependent illumination as surface coloration, producing appearance inconsistencies and making it harder to recover the underlying scene. We present \textbf{DaFISH}, which can be plugged into any novel-view synthesis (NVS) pipeline such as NeRF. It factorizes per-point emitted color into surface albedo and a spherical harmonic (SH)-represented illumination factor. During training, the SH illumination is evaluated directly, enabling stable joint optimization of albedo, lighting, and geometry. \textbf{DaFISH} provides analytical generalization of illumination to arbitrary camera poses without retraining. Building on the closed-form SH-to-irradiance conversion, the training pipeline is fully differentiable from SH evaluation through volume rendering. Averaged over the…
PDF arXiv
arXiv cs.CVOct 17
Modeling Uncertainty Propagation in 3D Morphable Models
Megh Shukla, Edo Collins, Prashanth Chandran, Karel Petránek, Sergio Orts-Escolano +2
We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications…
PDF arXiv
arXiv cs.CVOct 17
Spectral-Aware Proposal Enhancement for Diffusion-Based Multispectral UAV Multi-Object Tracking
Xu Lin, Tingfa Xu, Tianhao Li, Haolin Qin, Huawei Gao +2
Multispectral UAV multi-object tracking (MOT) with oriented bounding boxes remains challenging due to small targets, cluttered backgrounds, and unstable inter-frame associations. Although diffusion-based MOT provides an effective iterative proposal refinement paradigm, existing diffusion-based trackers lack dedicated designs for exploiting multispectral cues in proposal initialization and confidence calibration, limiting tracking robustness in complex UAV scenarios. This letter proposes DSATrack, a diffusion-guided spectral-aware tracking framework for multispectral UAV MOT. Specifically, a lightweight spectral encoder injects multispectral information into the RPN branch through feature fusion, enabling the RPN head to generate high-quality object-aware proposals. These proposals are then injected into the initial proposal set to provide reliable spatial priors for diffusion refinement. Furthermore, a spectral-saliency confidence modulation strategy recalibrates refined tracking confidence according to local multispectral saliency contrast before score thresholding, enhancing spectrally reliable candidates while suppressing unreliable ones. Without modifying the original…
PDF arXiv
arXiv cs.CVOct 17
Beyond Geometric Closure: Measuring and Diagnosing Mechanical Consistency in Visual Dynamics
Qibin Gu, Yunfeng Yan
Visual accuracy leaves open whether predicted motion obeys dynamics. We introduce a framework for measuring and diagnosing mechanical consistency in visual dynamics. An independent SI audit of rigid closed chains separates geometric admissibility, reaction-free actuator effort, and work--energy balance. Continuous scoring measures progress against fixed engineering targets, while reference replay quantifies measurement error. With three seeds per setting, we evaluate KeyCLD and CON across 343 planar assemblies at 64/256 pixels and V-JEPA + AC and LingBot-VA on 100 shared spatial test assemblies. CON reduces image MSE by 37.6%/37.2% relative to persistence, yet neither planar model attains valid native geometry. At 256 pixels, calibrated RGB replay places forecast errors far above measurement error. With exact states, KeyCLD preserves geometry and RK4 training raises its score from 52.65 to 64.30. In both spatial pathways, CAD/IK reconstruction restores geometry while mean actuator residuals exceed five times the engineering targets. The interventions distinguish gains in visual prediction and geometric admissibility from remaining force and energy errors.
PDF arXiv
arXiv cs.CVOct 17
IntriAdapt: Generalizable Intrinsic-Guided Test-Time Adaptation for Metric Geometry Estimation
Jiaxing Chen, Zizheng Yan, Jingyu Zhuang, Jinwei Chen, Shuguang Cui +2
Recovering metric 3D geometry from images is fundamentally ill-posed due to perspective projection ambiguity. Current geometry models incorporate supplied or predicted intrinsics during inference, but do not use them to supervise model adaptation. To refine these predictions, test-time adaptation (TTA) typically relies on external depth measurements. In this paper, we instead use camera intrinsics as an output-side geometric verifier. Comparing the point cloud's viewing rays with those defined by the known intrinsics yields a direct measure of geometric inconsistency. We propose \textit{IntriAdapt}, a TTA framework that minimizes this closed-form geometric residual. By optimizing a lightweight LoRA adapter on a frozen geometry foundation model, our method aligns the prediction with the specific camera model without depth sensors or external priors. We further extend this approach to video sequences, demonstrating that adapting a short sequence generalizes to other sequences captured by the same camera. The adapted model processes held-out images in a feed-forward pass, without repeating optimization for each frame. Experiments on ten single-image and seven sequence benchmarks…
PDF arXiv
arXiv cs.CVOct 17
SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation
Bowen He, Li Zhang, Yujie Chen, Chang Cheng
Self-supervised monocular depth estimation remains challenging due to the loss of fine-grained details and limited adaptability to varying scene structures. In this paper, we propose SAP-Depth, a novel architecture based on Scene-Adaptive Probabilistic. Unlike traditional methods, we introduce a Dynamic Bins mechanism powered by a Learnable Parameter Predictor, which adaptively estimates scene-specific depth distributions conditioned on the input image context. Furthermore, a Bin-wise Probability Distribution Decoder is designed to effectively fuse multi-scale representations to predict dense depth probability distributions, significantly enhancing boundary sharpness and structural recovery. Through the collaborative coupling of these proposed modules, our model achieves dynamic prediction of scene-specific depth distributions. Extensive experiments on the KITTI dataset show that SAP-Depth outperforms existing state-of-the-art methods in accuracy, particularly in its ability to better preserve complex edge details. Further, tests on the Make3D dataset validate the model's excellent generalization capability and robustness.
PDF arXiv

Papers

Topics in this field

3d scene · 3d reconstruction · depth estimation261 papers46 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d207 papers41 this week
remote sensing · satellite imagery · earth observation127 papers27 this week
point cloud registration · 3d point clouds · point cloud69 papers22 this week
small target detection · infrared small target · target detection113 papers21 this week
camera lidar · 3d object detection · point clouds94 papers15 this week

Active contributors · 90 days

Recent papers

All in Research →

arXiv cs.CVOct 17
Directionally Factorized Illumination with Spherical Harmonic Rotation
Yimeng Dou, Adam M. Saunders, Andre T. S. Hucke, Michael E. Kim, Adam W. Anderson +3
Many real-world capture scenarios involve an active light source rigidly attached to the camera: camera flashes for dark-environment imaging, underwater vehicle lights, and endoscopic/laparoscopic medical procedures. As the camera moves, the co-mounted light moves with it, causing the illumination direction at each surface point to change across views. Neural rendering methods (NeRF, Gaussian Splatting) encode this pose-dependent illumination as surface coloration, producing appearance inconsistencies and making it harder to recover the underlying scene. We present \textbf{DaFISH}, which can be plugged into any novel-view synthesis (NVS) pipeline such as NeRF. It factorizes per-point emitted color into surface albedo and a spherical harmonic (SH)-represented illumination factor. During training, the SH illumination is evaluated directly, enabling stable joint optimization of albedo, lighting, and geometry. \textbf{DaFISH} provides analytical generalization of illumination to arbitrary camera poses without retraining. Building on the closed-form SH-to-irradiance conversion, the training pipeline is fully differentiable from SH evaluation through volume rendering. Averaged over the…
PDF arXiv
arXiv cs.CVOct 17
Modeling Uncertainty Propagation in 3D Morphable Models
Megh Shukla, Edo Collins, Prashanth Chandran, Karel Petránek, Sergio Orts-Escolano +2
We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications…
PDF arXiv
arXiv cs.CVOct 17
Spectral-Aware Proposal Enhancement for Diffusion-Based Multispectral UAV Multi-Object Tracking
Xu Lin, Tingfa Xu, Tianhao Li, Haolin Qin, Huawei Gao +2
Multispectral UAV multi-object tracking (MOT) with oriented bounding boxes remains challenging due to small targets, cluttered backgrounds, and unstable inter-frame associations. Although diffusion-based MOT provides an effective iterative proposal refinement paradigm, existing diffusion-based trackers lack dedicated designs for exploiting multispectral cues in proposal initialization and confidence calibration, limiting tracking robustness in complex UAV scenarios. This letter proposes DSATrack, a diffusion-guided spectral-aware tracking framework for multispectral UAV MOT. Specifically, a lightweight spectral encoder injects multispectral information into the RPN branch through feature fusion, enabling the RPN head to generate high-quality object-aware proposals. These proposals are then injected into the initial proposal set to provide reliable spatial priors for diffusion refinement. Furthermore, a spectral-saliency confidence modulation strategy recalibrates refined tracking confidence according to local multispectral saliency contrast before score thresholding, enhancing spectrally reliable candidates while suppressing unreliable ones. Without modifying the original…
PDF arXiv
arXiv cs.CVOct 17
Beyond Geometric Closure: Measuring and Diagnosing Mechanical Consistency in Visual Dynamics
Qibin Gu, Yunfeng Yan
Visual accuracy leaves open whether predicted motion obeys dynamics. We introduce a framework for measuring and diagnosing mechanical consistency in visual dynamics. An independent SI audit of rigid closed chains separates geometric admissibility, reaction-free actuator effort, and work--energy balance. Continuous scoring measures progress against fixed engineering targets, while reference replay quantifies measurement error. With three seeds per setting, we evaluate KeyCLD and CON across 343 planar assemblies at 64/256 pixels and V-JEPA + AC and LingBot-VA on 100 shared spatial test assemblies. CON reduces image MSE by 37.6%/37.2% relative to persistence, yet neither planar model attains valid native geometry. At 256 pixels, calibrated RGB replay places forecast errors far above measurement error. With exact states, KeyCLD preserves geometry and RK4 training raises its score from 52.65 to 64.30. In both spatial pathways, CAD/IK reconstruction restores geometry while mean actuator residuals exceed five times the engineering targets. The interventions distinguish gains in visual prediction and geometric admissibility from remaining force and energy errors.
PDF arXiv
arXiv cs.CVOct 17
IntriAdapt: Generalizable Intrinsic-Guided Test-Time Adaptation for Metric Geometry Estimation
Jiaxing Chen, Zizheng Yan, Jingyu Zhuang, Jinwei Chen, Shuguang Cui +2
Recovering metric 3D geometry from images is fundamentally ill-posed due to perspective projection ambiguity. Current geometry models incorporate supplied or predicted intrinsics during inference, but do not use them to supervise model adaptation. To refine these predictions, test-time adaptation (TTA) typically relies on external depth measurements. In this paper, we instead use camera intrinsics as an output-side geometric verifier. Comparing the point cloud's viewing rays with those defined by the known intrinsics yields a direct measure of geometric inconsistency. We propose \textit{IntriAdapt}, a TTA framework that minimizes this closed-form geometric residual. By optimizing a lightweight LoRA adapter on a frozen geometry foundation model, our method aligns the prediction with the specific camera model without depth sensors or external priors. We further extend this approach to video sequences, demonstrating that adapting a short sequence generalizes to other sequences captured by the same camera. The adapted model processes held-out images in a feed-forward pass, without repeating optimization for each frame. Experiments on ten single-image and seven sequence benchmarks…
PDF arXiv
arXiv cs.CVOct 17
SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation
Bowen He, Li Zhang, Yujie Chen, Chang Cheng
Self-supervised monocular depth estimation remains challenging due to the loss of fine-grained details and limited adaptability to varying scene structures. In this paper, we propose SAP-Depth, a novel architecture based on Scene-Adaptive Probabilistic. Unlike traditional methods, we introduce a Dynamic Bins mechanism powered by a Learnable Parameter Predictor, which adaptively estimates scene-specific depth distributions conditioned on the input image context. Furthermore, a Bin-wise Probability Distribution Decoder is designed to effectively fuse multi-scale representations to predict dense depth probability distributions, significantly enhancing boundary sharpness and structural recovery. Through the collaborative coupling of these proposed modules, our model achieves dynamic prediction of scene-specific depth distributions. Extensive experiments on the KITTI dataset show that SAP-Depth outperforms existing state-of-the-art methods in accuracy, particularly in its ability to better preserve complex edge details. Further, tests on the Make3D dataset validate the model's excellent generalization capability and robustness.
PDF arXiv

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 47m ago

3D scene

Found by the topic model: 3d scene, gaussian splatting 3dgs, remote sensing.

172papers this week−7.0% vs last week

All papers · 871 With code · 26

Newest Most relevant

813 papers this month

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality scenes when camera poses differ significantly from training views.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in scene generation1 paper raises it · 1 addresses it
Generated 3D scenes often lack physically plausible object interactions and spatial compatibility.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing correspondence methods fail when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D geometry.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.