Papers

Field

Top tools here

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality views when camera poses differ significantly from training data.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in 3D scene generation1 paper raises it · 1 addresses it
Generated objects often lack fine-grained spatial and physical compatibility with their surroundings.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing methods fail to match dense correspondences when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D scenes.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

3d scene · 3d reconstruction · depth estimation250 papers37 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d198 papers33 this week
remote sensing · satellite imagery · earth observation123 papers24 this week
small target detection · infrared small target · target detection112 papers21 this week
point cloud registration · 3d point clouds · point cloud63 papers16 this week
camera lidar · 3d object detection · point clouds88 papers9 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.CVOct 18
ReDT: Resolution-Decoupled Visual Geometry Transformer
Abteen Arab, Guile Wu, Dongfeng Bai
We introduce ReDT, a feed-forward visual geometry transformer designed to scale multi-view reconstruction to high-definition (HD) inputs. Existing visual geometry transformers are typically trained at fixed, standard definition (SD) resolutions and exhibit degraded reconstruction quality and poor computational scaling to high input resolutions. While recent efficient architectures reduce the complexity of inter-frame attention, expensive intra-frame and encoder attention remain largely unaddressed, making HD reconstruction impractical with existing architectures. ReDT addresses both limitations with a resolution-aware architecture designed to scale to high-resolution inputs. The key idea of ReDT is to first map each view to a fixed-size set of basis tokens, compute correspondences in this compact constant-size basis space, and then project the outputs back to the original resolution for dense prediction. By decoupling the internal representation from input resolution during attention, ReDT substantially reduces the computational cost of high-resolution 3D reconstruction and improves model performance at all resolutions. Extensive experiments on multiple datasets spanning SD to…
PDF arXiv
arXiv cs.CVOct 18
UBD-Bench: Benchmarking and Analyzing Unimodal Branch Degradation in Multimodal Perception for Autonomous Driving
Ang Liu, Xiang Gu, Ruikai Li, Qiyue Wang, Yizhe Fan +8
Multimodal perception improves autonomous-driving performance by combining complementary sensor information. However, stronger multimodal predictions do not necessarily imply stronger representations in each modality-specific branch. Such branch-level weaknesses can remain hidden in conventional output-level evaluation. We introduce UBD-Bench, a systematic benchmark for studying unimodal branch degradation (UBD) in multimodal perception. UBD-Bench uses a Paired Frozen-Branch Probe to compare the task-relevant representation quality of the same modality after independent unimodal training and multimodal joint training under matched decoding conditions. Across four autonomous-driving perception tasks and 10 models, we find that Camera branches consistently degrade after multimodal joint training, whereas LiDAR branches exhibit weaker and more model-dependent changes, revealing a pronounced modality asymmetry. Hierarchical probing further shows that Camera-branch degradation extends from the visual backbone through the Camera-to-BEV pathway and is most severe at the pre-fusion Camera BEV representation. Guided by this diagnosis, we introduce unimodal-to-multimodal distillation at…
PDF arXiv
arXiv cs.CVOct 18
RAHF-Net: Range-Aware Heterogeneous Fusion for LiDAR-Assisted RGB Semantic Segmentation
R. D. Lin, Pengcheng Weng, Ge Wang, Han Ding, Fei Wang
This paper studies LiDAR-assisted RGB semantic segmentation, where LiDAR point-cloud features improve dense pixel-wise predictions in the RGB image plane rather than 3D point labels or detections. Existing 2D fusion methods often project point clouds into shallow depth/range maps and apply spatially uniform fusion, discarding rich 3D structure and ignoring the range-dependent reliability of outdoor LiDAR. We propose RAHF-Net, a Range-Aware Heterogeneous Fusion Network. Its Cylindrical-to-Perspective (C2P) module projects pretrained cylindrical voxel features into the image plane, preserving structured 3D semantic priors. A Range-Aware Heterogeneous Fusion (RAHF) module then uses a soft depth-derived range mask to perform LiDAR-guided geometric enhancement in reliable near-field regions and visual-context enhancement in sparse far-field regions. On KITTI-360, RAHF-Net achieves 65.76 mIoU, outperforming representative RGB-only and RGB-LiDAR baselines, including CMX by 1.45 points. Experiments on DELIVER further demonstrate improved robustness under adverse weather, illumination degradation, and sensor corruption.
PDF arXiv
arXiv cs.CVOct 18
PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes
Ruilin Huang, Jue Wang, Zhen Tian, Xia Hua, F. Richard Yu
Reflections, glare, transparent structures, and material-dependent appearance variations can introduce ambiguity into RGB-based visual perception in vehicle-relevant environments. Polarization provides complementary reflection- and material-sensitive information, but its acquisition requires dedicated sensing hardware. This paper presents RGB-P Wild, a real-world dataset containing 10,000 paired RGB and four-angle linear-polarization image sets, together with PolarVis, a diffusion-based framework for recovering polarization representations from a single RGB image. PolarVis combines hierarchical RGB feature extraction, diffusion-based feature refinement, four angle-specific decoders, and multi-scale attention recalibration to reconstruct polarization intensity images at $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$. On RGB-P Wild, PolarVis achieves a normalized-domain MSE of 0.0465, a PSNR of 27.83 dB, and an SSIM of 0.8597. Compared with the released PolarAnything model under the adopted evaluation protocol, PolarVis obtains an AoLP error of $25.66^\circ$ and a DoLP MAE of 0.0404, compared with $46.03^\circ$ and 0.1084, respectively. Cross-dataset evaluation on…
PDF arXiv
arXiv cs.CVOct 18
A 3D-Driven Framework for Moving-Camera Perspective-to-360° Video Generation
Michael Wakeham, Mrinmoy Kumar Das, Wenqi Wang, Ayush Tewari, Boqing Gong +1
Turning a moving-camera perspective video into a 360° video requires more than just generating content outside the input frame: observations revealed over time must be preserved, unseen regions completed, and the resulting scene kept consistent as the camera viewpoint changes. Existing methods assign these different requirements to a single conditional video model, requiring them to be jointly learned from panoramic video data. We instead present a modular, 3D-driven framework that distributes these three responsibilities across specialized components, all connected through a shared scene representation. Input observations are reconstructed explicitly in 3D, while a pretrained panoramic image model completes unobserved content from a selected panoramic render. This generated content is lifted into the shared 3D scene, where it remains spatially anchored across camera viewpoints. Without requiring any additional panoramic-video training, our framework achieves state-of-the-art input-view fidelity, perceptual quality, and geometric consistency on a static-scene subset of the Argus evaluation set. These results support our decomposed, 3D-driven formulation as a promising alternative…
PDF arXiv
arXiv cs.LGOct 18
Structured Relevance Determination and Channel Additive Gaussian Processes for Image Data
Harald Roßels, Ludovico Scarton
We introduce radial basis function (RBF) kernels for Gaussian processes that use structured hyperparameter sharing for image-valued inputs. For high-dimensional inputs such as images, a fully anisotropic RBF kernel assigns a separate lengthscale to each input dimension, which can introduce many hyperparameters and may lead to overfitting and poorly calibrated predictive uncertainty when data are limited. We propose structured relevance determination by tying lengthscales across groups of adjacent pixels and within each colour channel. This yields patch-wise and channel-wise constrained anisotropic RBF parametrisations that reduce the number of independently learned lengthscales. We further introduce channel-additive Gaussian processes for multi-channel images and combine them with structured relevance determination, resulting in a parameter-efficient decomposition of the latent function into components defined on channel subsets, enabling some non-local generalisation. The proposed kernels are evaluated on MNIST and a compact binary aircraft-detection dataset of low-resolution RGB satellite image chips. We also show how channel-wise structured relevance determination can be…
PDF arXiv

Papers

Active contributors · 90 days

Topics in this field

3d scene · 3d reconstruction · depth estimation250 papers37 this week
gaussian splatting 3dgs · 3d gaussian splatting · feed forward 3d198 papers33 this week
remote sensing · satellite imagery · earth observation123 papers24 this week
small target detection · infrared small target · target detection112 papers21 this week
point cloud registration · 3d point clouds · point cloud63 papers16 this week
camera lidar · 3d object detection · point clouds88 papers9 this week

Recent papers

All in Research →

arXiv cs.CVOct 18
ReDT: Resolution-Decoupled Visual Geometry Transformer
Abteen Arab, Guile Wu, Dongfeng Bai
We introduce ReDT, a feed-forward visual geometry transformer designed to scale multi-view reconstruction to high-definition (HD) inputs. Existing visual geometry transformers are typically trained at fixed, standard definition (SD) resolutions and exhibit degraded reconstruction quality and poor computational scaling to high input resolutions. While recent efficient architectures reduce the complexity of inter-frame attention, expensive intra-frame and encoder attention remain largely unaddressed, making HD reconstruction impractical with existing architectures. ReDT addresses both limitations with a resolution-aware architecture designed to scale to high-resolution inputs. The key idea of ReDT is to first map each view to a fixed-size set of basis tokens, compute correspondences in this compact constant-size basis space, and then project the outputs back to the original resolution for dense prediction. By decoupling the internal representation from input resolution during attention, ReDT substantially reduces the computational cost of high-resolution 3D reconstruction and improves model performance at all resolutions. Extensive experiments on multiple datasets spanning SD to…
PDF arXiv
arXiv cs.CVOct 18
UBD-Bench: Benchmarking and Analyzing Unimodal Branch Degradation in Multimodal Perception for Autonomous Driving
Ang Liu, Xiang Gu, Ruikai Li, Qiyue Wang, Yizhe Fan +8
Multimodal perception improves autonomous-driving performance by combining complementary sensor information. However, stronger multimodal predictions do not necessarily imply stronger representations in each modality-specific branch. Such branch-level weaknesses can remain hidden in conventional output-level evaluation. We introduce UBD-Bench, a systematic benchmark for studying unimodal branch degradation (UBD) in multimodal perception. UBD-Bench uses a Paired Frozen-Branch Probe to compare the task-relevant representation quality of the same modality after independent unimodal training and multimodal joint training under matched decoding conditions. Across four autonomous-driving perception tasks and 10 models, we find that Camera branches consistently degrade after multimodal joint training, whereas LiDAR branches exhibit weaker and more model-dependent changes, revealing a pronounced modality asymmetry. Hierarchical probing further shows that Camera-branch degradation extends from the visual backbone through the Camera-to-BEV pathway and is most severe at the pre-fusion Camera BEV representation. Guided by this diagnosis, we introduce unimodal-to-multimodal distillation at…
PDF arXiv
arXiv cs.CVOct 18
RAHF-Net: Range-Aware Heterogeneous Fusion for LiDAR-Assisted RGB Semantic Segmentation
R. D. Lin, Pengcheng Weng, Ge Wang, Han Ding, Fei Wang
This paper studies LiDAR-assisted RGB semantic segmentation, where LiDAR point-cloud features improve dense pixel-wise predictions in the RGB image plane rather than 3D point labels or detections. Existing 2D fusion methods often project point clouds into shallow depth/range maps and apply spatially uniform fusion, discarding rich 3D structure and ignoring the range-dependent reliability of outdoor LiDAR. We propose RAHF-Net, a Range-Aware Heterogeneous Fusion Network. Its Cylindrical-to-Perspective (C2P) module projects pretrained cylindrical voxel features into the image plane, preserving structured 3D semantic priors. A Range-Aware Heterogeneous Fusion (RAHF) module then uses a soft depth-derived range mask to perform LiDAR-guided geometric enhancement in reliable near-field regions and visual-context enhancement in sparse far-field regions. On KITTI-360, RAHF-Net achieves 65.76 mIoU, outperforming representative RGB-only and RGB-LiDAR baselines, including CMX by 1.45 points. Experiments on DELIVER further demonstrate improved robustness under adverse weather, illumination degradation, and sensor corruption.
PDF arXiv
arXiv cs.CVOct 18
PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes
Ruilin Huang, Jue Wang, Zhen Tian, Xia Hua, F. Richard Yu
Reflections, glare, transparent structures, and material-dependent appearance variations can introduce ambiguity into RGB-based visual perception in vehicle-relevant environments. Polarization provides complementary reflection- and material-sensitive information, but its acquisition requires dedicated sensing hardware. This paper presents RGB-P Wild, a real-world dataset containing 10,000 paired RGB and four-angle linear-polarization image sets, together with PolarVis, a diffusion-based framework for recovering polarization representations from a single RGB image. PolarVis combines hierarchical RGB feature extraction, diffusion-based feature refinement, four angle-specific decoders, and multi-scale attention recalibration to reconstruct polarization intensity images at $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$. On RGB-P Wild, PolarVis achieves a normalized-domain MSE of 0.0465, a PSNR of 27.83 dB, and an SSIM of 0.8597. Compared with the released PolarAnything model under the adopted evaluation protocol, PolarVis obtains an AoLP error of $25.66^\circ$ and a DoLP MAE of 0.0404, compared with $46.03^\circ$ and 0.1084, respectively. Cross-dataset evaluation on…
PDF arXiv
arXiv cs.CVOct 18
A 3D-Driven Framework for Moving-Camera Perspective-to-360° Video Generation
Michael Wakeham, Mrinmoy Kumar Das, Wenqi Wang, Ayush Tewari, Boqing Gong +1
Turning a moving-camera perspective video into a 360° video requires more than just generating content outside the input frame: observations revealed over time must be preserved, unseen regions completed, and the resulting scene kept consistent as the camera viewpoint changes. Existing methods assign these different requirements to a single conditional video model, requiring them to be jointly learned from panoramic video data. We instead present a modular, 3D-driven framework that distributes these three responsibilities across specialized components, all connected through a shared scene representation. Input observations are reconstructed explicitly in 3D, while a pretrained panoramic image model completes unobserved content from a selected panoramic render. This generated content is lifted into the shared 3D scene, where it remains spatially anchored across camera viewpoints. Without requiring any additional panoramic-video training, our framework achieves state-of-the-art input-view fidelity, perceptual quality, and geometric consistency on a static-scene subset of the Argus evaluation set. These results support our decomposed, 3D-driven formulation as a promising alternative…
PDF arXiv
arXiv cs.LGOct 18
Structured Relevance Determination and Channel Additive Gaussian Processes for Image Data
Harald Roßels, Ludovico Scarton
We introduce radial basis function (RBF) kernels for Gaussian processes that use structured hyperparameter sharing for image-valued inputs. For high-dimensional inputs such as images, a fully anisotropic RBF kernel assigns a separate lengthscale to each input dimension, which can introduce many hyperparameters and may lead to overfitting and poorly calibrated predictive uncertainty when data are limited. We propose structured relevance determination by tying lengthscales across groups of adjacent pixels and within each colour channel. This yields patch-wise and channel-wise constrained anisotropic RBF parametrisations that reduce the number of independently learned lengthscales. We further introduce channel-additive Gaussian processes for multi-channel images and combine them with structured relevance determination, resulting in a parameter-efficient decomposition of the latent function into components defined on channel subsets, enabling some non-local generalisation. The proposed kernels are evaluated on MNIST and a compact binary aircraft-detection dataset of low-resolution RGB satellite image chips. We also show how channel-wise structured relevance determination can be…
PDF arXiv

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 10m ago

3D scene

Found by the topic model: 3d scene, gaussian splatting 3dgs, remote sensing.

140papers this week−25.9% vs last week

All papers · 834 With code · 26

Newest Most relevant

778 papers this month

Open problems in 3D scene

3D consistency in generated scenes3 papers raise it · 3 address it
Generated videos and scenes often lack geometric and multi-view consistency, breaking immersion and realism.
- OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphsraises · addresses
- Orbia: Do Generated Videos Form a Consistent 3D World?raises · addresses
- GeoFix: Geometry Fixing for 4D Consistent Generationraises · addresses
High-quality novel view synthesis under large viewpoint changes3 papers raise it · 3 address it
Feed-forward methods struggle to render high-quality views when camera poses differ significantly from training data.
- InfiniSplat v2: Analyzing and Improving Feed-Forward 3DGS from Any Viewsraises · addresses
- LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generationraises · addresses
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesisraises · addresses
Physics and geometry coherence in 3D scene generation1 paper raises it · 1 addresses it
Generated objects often lack fine-grained spatial and physical compatibility with their surroundings.
- Tetris3D: 3D Scene Generation With Objects That Fit Togetherraises · addresses
Dense correspondence matching beyond rigid motion1 paper raises it · 1 addresses it
Existing methods fail to match dense correspondences when transformations break physical continuity but preserve identity.
- Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matchingraises · addresses
Metric distance estimation in 3D reconstruction1 paper raises it · 1 addresses it
Current benchmarks do not directly evaluate point-to-point distance accuracy in reconstructed 3D scenes.
- Pumpire: Unified Benchmark for Metric Distance Estimationraises · addresses

Named by an AI from these papers' abstracts.