TodayNewsResearchFieldsStacksMarketAnalytics
⌘K

Research

New papers ranked by attention, with their code, models and discussion linked. Pick a field on the left.

Research · AI field

Diffusion priors

Found by the topic model: diffusion priors, computed tomography.

Open the field page
28papers this week+75.0% vs last week89 this month

Papers

Open problems

Rising research topics

Ask the research

Sample
ByteTrail · AI and developer-tools intelligence
NewsResearchStacksAPI
Fields · this week
All fields3070Multimodal533Frontier LLMs402Recursive self improvement332Inference & serving291Agents & indirect prompt injection247Neural operators222Clinical reasoning207Robotics178Linear bandits119Forecasting time117Voice AI70AI safety68Molecular property prediction57Federated learning44Character recognition OCR29Diffusion priors28All 23 fields
WeekMonthQuarterAll
NewestMost relevant

Papers

Open problems in Diffusion priors

  • Generalization of diffusion priors across domains3 papers raise it · 3 address it

    Limits the reuse of pre-trained diffusion models for tasks like medical imaging or cosmology without domain-specific retraining.

    • Diffusion Plug-in Priors for Self-Supervised Ultrasound Video Restorationraises · addresses
    • Sparse-View 3D Coronary Reconstruction from Vessel Segmentations Using a Diffusion Priorraises · addresses
    • 3D dark matter map of the JWST COSMOS-Web field using diffusion-based generative priorsraises · addresses
  • Handling sparse or missing sensor data in reconstruction3 papers raise it · 3 address it

    Real-world sensing often has incomplete or asynchronous measurements, degrading reconstruction quality.

    • Fault-tolerant shallow recurrent decoders for sparse sensing with missing sensor measurementsraises · addresses
    • Sparse-View 3D Coronary Reconstruction from Vessel Segmentations Using a Diffusion Priorraises · addresses
    • Spatial and Temporal Super-Resolution and Denoising of Intracardiac 4D Flow MRIraises · addresses
  • Task-specific optimization of diffusion priors without paired data3 papers raise it · 3 address it

    Many applications lack paired low-high quality data, making supervised training of priors infeasible.

    • Diffusion Plug-in Priors for Self-Supervised Ultrasound Video Restorationraises · addresses
    • Sparse-View 3D Coronary Reconstruction from Vessel Segmentations Using a Diffusion Priorraises · addresses
    • Learning from high-resolution 2D images to reconstruct 3D microstructuresraises · addresses
  • Robustness to noise in diffusion sampling1 paper raises it · 1 addresses it

    Noise injection during sampling can break or make generalization, affecting sample quality and consistency.

    • The role of injected noise in diffusion model generalizationraises · addresses
  • Efficient inference with diffusion priors for high-dimensional data2 papers raise it · 2 address it

    Slow sampling or high computational cost limits practical deployment in time-sensitive applications.

    • Spatial and Temporal Super-Resolution and Denoising of Intracardiac 4D Flow MRIraises · addresses
    • Prior-Guided Physics-Informed Deep Learning for Fast 4D MRI Reconstructionraises · addresses

Named by an AI from these papers' abstracts.

Rising research topics

  • Linear bandits119+98.3%
  • Diffusion priors28+75.0%
  • Channel state information20+66.7%
  • Molecular property prediction57+62.9%
  • Algorithmic trading27+42.1%
  • Federated learning44+33.3%
  • Quantum machine learning25+25.0%
  • Inference & serving290+13.3%

Papers

  • arXiv cs.CVNov 4

    FLINT: Coupling Proximal Initialization and Bounded Stochasticity for Flow-Matching Inverse Problems

    Junseo Bang, Daewon Choi, Dong Ju Mun, Se Young Chun

    Flow matching models have emerged as a dominant paradigm for generative posterior sampling in imaging inverse problems. Existing solvers follow two distinct lines of work: trajectory-space methods, which refine sampling dynamics from random initial noise, and source-space methods, which optimize the initial noise itself. While most prior work treats these as independent design choices, we show that they are inherently coupled. We derive a unified stability bound that links initialization proximity and cumulative sampling stochasticity through a multiplicative envelope, complemented by two empirical observations. First, we identify a sufficiency regime in which a moderately optimized proximal source initialization is enough to attain the performance plateau, rendering exhaustive source-space optimization unnecessary. Second, this initial advantage is fragile: it is eroded by excessive stochasticity accumulation during guided sampling, necessitating a mechanism that bounds the cumulative noise injection. These findings motivate FLINT, a two-stage framework that operationalizes this coupling. FLINT establishes a high-fidelity starting point via a manifold-constrained proximal…

    PDF
  • arXiv eess.IVNov 4

    Dual Flow: Joint Image-Measurement Flow for Inverse Problems

    Sevket Ege Yesilyurt, Figen S. Oktem

    Existing generative model-based approaches to linear inverse problems typically use the acquired noisy measurement to condition or guide sampling in the image domain. In this work, we introduce Dual Flow, a novel joint flow model for solving linear inverse problems by jointly estimating the clean image and its corresponding noise-free measurement from the observation. Specifically, Dual Flow formulates image reconstruction and measurement denoising as a coupled estimation problem, using coupled flows in the image and measurement domains. The acquired noisy measurement $\mathbf{y}_0$ is used only to initialize the measurement state; thereafter, the image and measurement states are jointly updated based on their current values. By construction, the final image and measurement estimates are guaranteed to satisfy the underlying forward model. We derive the sampling updates for the joint flow and show that they are governed by the conditional mean of the clean image given the current joint state. We further derive a closed-form expression for a sufficient statistic of the joint state and show that the required conditional mean can be obtained by solving an image-domain Gaussian…

    PDF
  • arXiv cs.CVNov 3

    Diffusion Plug-in Priors for Self-Supervised Ultrasound Video Restoration

    Chu Chen, Mingduo He, Yining Li, Yinhao Wu, Wei Tang +4

    Ultrasound videos are limited by hardware-constrained spatial resolution and signal-dependent speckle, while paired low- and high-resolution acquisitions for supervised restoration are rarely available. We aim to jointly super-resolve and despeckle ultrasound videos without paired training data or network retraining. We propose UltraDiff, which uses a frozen pre-trained diffusion model (DM) as a plug-in prior and restores each frame by optimizing the latent seed of the DM sampler under a range-constrained maximum a posteriori formulation. We derive a speckle-aware data-fidelity term from a Rayleigh scale-family model and characterize its relative-residual and variance-adaptive weighting properties. We further introduce a stabilized fidelity that is bounded below and strictly convex in log intensity for positive observations in its unclamped form. A total-variation regularizer complements the DM prior, and a temporal seed update warm-starts each frame from the optimized seed of its predecessor. On EchoNet-LVH videos with simulated speckle and additive noise, UltraDiff with a natural-image prior improves PSNR by 1.24\,dB over the strongest baseline, and ultrasound-related priors…

    PDF
  • arXiv cs.LGNov 3

    Fault-tolerant shallow recurrent decoders for sparse sensing with missing sensor measurements

    Matteo Romilio Pasqual, Matteo Tomasetto, J. Nathan Kutz, Andrea Manzoni

    Real-world sensing systems are inherently imperfect: sensors may fail, intermittently disconnect, operate with asynchronous sampling rates, and be subject to noise, leading to measurement streams that are irregular, incomplete, and heterogeneous. Nevertheless, sensing strategies for real-time state reconstruction typically assume synchronous and continuously active sensors, a constraint that substantially limits their applicability in operational environments. In this work, we introduce fault-tolerant sparse sensing strategies in the context of shallow recurrent decoders (SHRED) to enable high-dimensional state estimation from sparse, asynchronous, and missing sensor measurements. Specifically, we consider naive imputation strategies, fault-tolerant recurrent temporal encoders, and we introduce four different transformer-based masking mechanisms to make SHRED explicitly aware of sensor availability through the binary data-availability mask. Moreover, we further introduce a multi-step decoding training objective for SHRED, resulting in smoother latent trajectories and more robust predictions. Throughout a synthetic benchmark and three real-world problems in hydrodynamic and ocean…

    PDF
  • arXiv physics.opticsNov 3

    LIPS-PC: Learning Illumination Patterns for Single-Pixel Classification

    Tomer Porat, Talya Eden, Eliahu Cohen

    In single-pixel (ghost) imaging, known illumination patterns probe an object and a detector with no spatial resolution returns one number per pattern. Each measurement costs acquisition time and photons and, for ionizing probes, contributes to delivered dose. Motivated by automated radiographic defect screening, where a class label suffices, we introduce Learning Illumination Patterns for Single-Pixel Classification (LIPS-PC). The patterns are learned end-to-end while the measurement operator remains one fixed linear matrix, identical for every input. We parameterize it as a residual correction to a two-dimensional discrete cosine transform (2D-DCT), $D=D_{\mathrm{base}}+Δ$, and jointly train the correction and a compact classifier using cross-entropy, without a reconstruction objective. This yields a task-driven linear embedding through optimization. We evaluate LIPS-PC numerically on MNIST and Fashion-MNIST under a controlled protocol, then on RIAWELC, a real industrial X-ray weld-radiograph dataset. Across five budgets, $M\in{8,16,32,64,128}$, corresponding to sampling ratios from below 1% to 12.5%, the learned operator outperforms the best tested fixed transform by 9-15…

    PDF
  • arXiv eess.IVNov 3

    Sparse-View 3D Coronary Reconstruction from Vessel Segmentations Using a Diffusion Prior

    Tianyuan Wang, Kenan Niu, Stefano Stramigioli

    Reconstructing three-dimensional (3D) coronary anatomy from sparse angiographic views could support percutaneous coronary intervention, but limited angular coverage and imperfect vessel segmentations introduce substantial geometric ambiguity. We propose a vessel-specific reconstruction framework combining a learned 3D diffusion prior, visual-hull initialization, and projection-based data consistency. The unconditional diffusion model is trained on coronary vessel masks without paired 2D--3D data and reused across different view counts without retraining. During inference, a scaled visual hull initializes reconstruction, while projection-based updates constrain diffusion refinement using patient-specific vessel observations. We evaluated the framework on simulated angiograms derived from ImageCAS computed tomography angiography volumes using two, four, and eight views at two contrast levels. Across 50 evaluation coronary-tree cases, the proposed method achieved higher mean Dice and clDice than visual hull, deep image prior (DIP), and hash-grid implicit neural representation (INR) at all view counts under reduced contrast. With eight views, improvements over DIP, the…

    PDF
  • arXiv cs.CVNov 2

    Spatial and Temporal Super-Resolution and Denoising of Intracardiac 4D Flow MRI

    Yusuf A. Ozturk, Jie Zheng, Fanwei Kong

    Intracardiac flow is driven by complex, and time-varying cardiac motion and interaction with the heart valves, producing complex flow features that may provide valuable information for disease diagnosis and treatment planning. 4D Flow MRI enables non-invasive, time-resolved measurement of three-dimensional cardiovascular velocity fields. However, substantial noise and limited spatial and temporal resolution limit its ability to accurately resolve flow features. We propose 4D-MRAR (4D Magnetic Resonance Augmented Reconstruction), a deep learning (DL) method to super-resolve and denoise 4D Flow MRI data. Unlike prior DL methods that train and infer on static vascular regions, our method predicts intracardiac flow within hearts undergoing large deformation. Moreover, it uniquely allows for simultaneous super-resolution in 3D space and time. The method combines a residual convolutional neural network, a recurrent neural network, and an implicit neural field to predict high-resolution velocity and pressure fields while efficiently enforcing the governing equations of fluid flow during training. Trained on a synthetic dataset obtained from computational fluid dynamics (CFD) simulations…

    PDF
  • arXiv astro-ph.CONov 2

    3D dark matter map of the JWST COSMOS-Web field using diffusion-based generative priors

    Diana Scognamiglio, Brandon Zhao, Katherine L. Bouman, Olivier Doré, Caitlin M. Casey +15

    Mapping the three-dimensional distribution of dark matter is key to tracing the growth of cosmic structure. Two-dimensional weak lensing mass maps recover its projected distribution, but they blend structures at different distances along the line of sight. We reconstruct the three-dimensional matter distribution in the 0.54 deg$^2$ COSMOS-Web field using deep James Webb Space Telescope imaging, with an effective source density of 121 galaxies arcmin$^{-2}$ We combine a physical weak lensing forward model with a diffusion-based Bayesian generative prior learned from cosmological simulations, yielding a posterior distribution of the matter field across 20 lens planes to $z \simeq 2$, doubling the redshift reach of previous Hubble Space Telescope-based 3D reconstructions and approaching cosmic noon, an epoch of rapid galaxy growth. The reconstruction separates structures that overlap in projection and recovers non-Gaussian matter structure across a range of spatial scales, including over-densities associated with independently identified galaxy clusters that were not detected in earlier weak lensing maps. Posterior sampling quantifies the range of three-dimensional matter…

    PDF
  • arXiv cs.LGNov 2

    The role of injected noise in diffusion model generalization

    Vincent Emmerling, Felix Günther, David Kappel

    Diffusion models trained on distinct datasets can produce highly similar samples when prompted with the same noise input, suggesting that they learn a generalized data distribution. We investigate the role of noise injected during the sampling process, and show that it plays an important role in making or breaking this strong generalization effect. Based on this finding we investigate the robustness of sample similarity to changes in the injected noise, and find that its impact is largest in the sampling regime where broad structure already exists but the precise identity of the sample is not yet determined.

    PDF
  • arXiv cs.CVNov 1

    Learning from high-resolution 2D images to reconstruct 3D microstructures

    Wenbo Sui, Lars P. Mikkelsen, Jens W. Andreasen, Daniel Lichau, Harold Phelippeau +1

    High-resolution three-dimensional (3D) microstructure characterization remains difficult to undertake at scale because volumetric imaging at the required spatial resolution is expensive, time-consuming or destructive. In many settings, however, fine-resolution two-dimensional (2D) imaging and lower-resolution volumetric measurements are both readily available. We introduce a Privileged Cross-modal Relational Distillation (PCRD) framework that leverages fine-resolution 2D observations during training to improve specimen-specific 3D reconstruction from lower-resolution volumetric imaging alone. We demonstrate PCRD on pultruded carbon fiber-reinforced polymer composites, using scanning electron microscopy as the privileged modality and micro-CT as the inference input. The framework transfers spatial organization by matching similarity distributions defined around corresponding spatial anchors in the teacher and student representations, avoiding direct alignment of heterogeneous feature vectors. In 2D cross-sectional reconstruction, PCRD improves mAP by 1.66 percentage points over direct micro-CT training. Independent 3D validation against high-resolution synchrotron X-ray CT data…

    PDF
  • arXiv cs.CVNov 1

    Training-Free Motion-Corrected 3D Brain MRI Reconstruction via Event-Guided References

    Wei Jiang, Hongli Chen, Tianyi Ding, Feng Liu, Nan Ye +1

    Head motion causes artefacts in 3D brain MRI and can bias structural measurements. Model-based correction alternates between image and motion estimation, but artefacts in the reconstructed reference can reinforce motion errors. Learned priors can reduce these artefacts but require pretraining and may add test-time computation. We propose EventMC (event-guided motion correction), a training-free method for interleaved Cartesian acquisitions. It iteratively detects motion events and estimates motion using reference images reconstructed from stable intervals. It then refines the image and motion from low to high spatial frequencies. In a fair comparison in simulation (same reconstruction), EventMC's motion estimates improve mean PSNR by up to 4.0 dB over those of the learning-based MotionTTT. A single run of EventMC is up to 7.5 times faster than MotionTTT with less than half the GPU memory. On in vivo motion-corrupted scans, EventMC achieves similar mean image quality to MotionTTT without any training. Code is available at https://github.com/bayesbreeze/EventMC.

    PDF
  • arXiv cs.CVNov 1

    Prior-Guided Physics-Informed Deep Learning for Fast 4D MRI Reconstruction

    Mojtaba Safari, Parag J. Parikh, Shansong Wang, Mingzhe Hu, Philip Connell +4

    Background and purpose: We propose PGPI4MR (prior-guided, physics-informed 4D MRI reconstruction), a deep learning method that leverages information from other respiratory phases or a setup scan, the first 30 s of the same scan, to improve 4D MRI reconstruction without extra scan time. Materials and methods: PGPI4MR is an unrolled network that reconstructs 16 respiratory phases from 60, 30, or 15 s of data, and single 3D volumes, each from 0.29-0.49 s of data, using a setup-scan image. The networks were trained once, on other volunteers, and were not retrained for new volunteers. We evaluated PGPI4MR on measured k-space from three datasets: free-breathing MR-linac (1.5T) and abdominal (3T) scans, and thigh scans with cuff compression (1.5T). All methods, including comparative models, received identical samples and were scored similarly. Results: In vivo, PGPI4MR had the lowest held-out error for 4D MRI at every scan time, and its 16 phases were ready a median of 7.5-13 s after the scan on the 2 GPUs tested. Coupling the 16 respiratory phases reduced the error at 60 s by 9.1% compared with reconstructing each phase alone. For sub-second MRI, the error at 0.49 s was 15.5% lower…

    PDF
  • arXiv cs.LGNov 1

    DyRadio: Spatiotemporal Diffusion for Dynamic Radio Map Reconstruction

    Zhijie Fan, Zhifeng Zhao, Rongpeng Li

    Accurate radio map (RM) reconstruction is essential for environment-aware wireless networks. Sampling-based methods, which infer radio conditions from sparse measurements, are widely adopted because they do not require explicit knowledge of transmitter configurations. However, in dynamic environments, time-varying propagation conditions cause RMs to evolve, complicating reconstruction from temporally distributed sparse observations. Although cross-frame observations provide complementary spatial information, their direct aggregation can introduce temporal state mismatch. Furthermore, unknown measurement noise adds to the reconstruction difficulties. To address these challenges, we propose DyRadio, a framework that integrates spatiotemporal RM reconstruction with unknown noise estimation. Specifically, a propagation-prior-guided, noise-conditioned spatiotemporal diffusion model (DM) jointly reconstructs consecutive RMs by capturing cross-frame dependencies, while adapting to measurement noise levels. To estimate the noise level, an iterative predictive likelihood method alternates between DM-based blind-area prediction and noise-level updates. The estimated noise level then…

    PDF
  • arXiv eess.IVNov 1

    Task-driven sparse-view planar X-ray computed tomography for axial metrology

    Yibin Tian, Shuming Yang, Wolfgang Heidrich

    Computational imaging can reduce the acquisition burden when the system is designed around task-relevant information rather than complete image recovery. We investigate this framework for sparse-view planar X-ray tomography (XCT) of layered objects, using measurement of the axial termination gap (ATG) of back-drilled printed-circuit-board (PCB) vias as a representative task. Instead of using 3D reconstruction, the proposed method estimates ATG directly from polychromatic projection data through a physics-based sensitivity model. A gated hierarchical estimator first fits ATG together with radiometric perturbations and selectively introduces global geometry and specimen-warp models only when a reduced chi-squared adequacy test and Bayesian information criterion justify the added degrees of freedom. The same low-dimensional perturbation model is incorporated into marginalized Fisher-information analyses, yielding task-specific conditional and calibration-aware Cram'er-Rao bounds. Numerical experiments show that, under a fixed total photon budget, task-driven metrology changes little over 12-64 views, while reconstruction-driven metrology is strongly view-dependent with much larger…

    PDF
  • arXiv cs.LGOct 31

    Accelerating posterior sampling in Bayesian inverse problems with diffusion models: an online-offline perspective

    Duc-Lam Duong

    Bayesian inverse problems require both informative prior models and computationally feasible posterior inference. Score-based diffusion models provide expressive learned priors, but their use for inversion introduces a conditioning problem: the score of a noised posterior is generally unavailable from a prior score alone. This selective survey reviews the principal approaches to that problem, including likelihood guidance, spectral and projection methods, sequential Monte Carlo corrections, and amortized conditional training. We organize methods by how they split cost between offline preparation and online inference, and compare their assumptions, advantages and limitations, with emphasis on computational speedups. Some ongoing directions and open questions are gathered at the end.

    PDF
  • arXiv cs.CVOct 31

    Robust Pixel-wise Digital Image Correlation under Large Motions and Illumination Variations via a Global-to-Local Hybrid Neural Field

    Xianglong Guo, Jianyang Li

    Digital image correlation (DIC) is a widely used non-contact technique for full-field displacement measurement. Data-driven deep-learning DIC offers rapid inference but may generalize poorly to unseen speckle patterns, deformation modes, noise levels, and illumination conditions. Classical optimization-based DIC and recent image-constrained neural formulations avoid labeled displacement data, yet their convergence and accuracy remain vulnerable to initialization, large rigid-body motion, fine non-uniform deformation, and gray-value disturbances. This paper presents a global-to-local hybrid neural DIC framework for robust pixelwise measurement. Stage 1 estimates a dominant in-plane rigid transformation through coarse rotation search, phase correlation, and masked refinement, providing a reliable initialization under large translation and rotation. Stage 2 represents the residual displacement with a hybrid Kolmogorov-Arnold network (KAN) and multilayer perceptron (MLP), and optimizes it using a valid-overlap, illumination-robust structural loss. The local gray-value model explicitly accommodates multiplicative and additive intensity changes. Simulated and real experiments…

    PDF
  • arXiv cs.CVOct 31

    PSF-Informed Reconstruction: An Enhanced 2D Directional Filtered Backprojection Method for Cone-Beam Rotary Computed Laminography

    Chengxiang Wang, Wei Yu, Baodong Liu

    Cone-beam rotary computed laminography (CL) presents a viable alternative to cone-beam CT for imaging objects with high aspect ratios. Conventional reconstruction techniques, such as CL-FBP and CL-FDK, often yield suboptimal results charac?terized by pronounced superimposition artifacts, reduced contrast,and blurred edges, primarily due to their reliance on 1D row-wise filtering along detector rows. To address these limitations and improve reconstruction quality, we proposed a novel 2D directional filtered backprojection (DFBP) method based on the point spread function (PSF). First, an exact expression for the projection process is derived by establishing the relationship between a reconstructed pixel and its corresponding projection point on the detector. Second, a revised backprojection process incorporating two weighting factors is introduced to counteract the nonlinearity inherent in conventional backprojection, leading to a simplified PSF formulation. Third, to circumvent the computational burden of 3D image space filtering,an approximate method by linearizing the coordinate relationship in Fourier space under some assumptions is developed that transfers the 3D filtering…

    PDF
  • arXiv cs.CVOct 31

    Spatial-Spectral Interactive POCS Network for Large-Gap Seismic Data Interpolation

    Baisong Jiang, Chunxia Zhang, Anxiang Di, Xiaoli Wei, Xiangyu Zhao +2

    Large gaps in seismic acquisition disrupt reflection events and hinder spectral recovery. Existing seismic interpolation deep unfolding networks primarily rely on localized spatial priors, limiting their ability to aggregate the global context necessary for reconstructing these extensive missing regions. We propose a spatial-spectral interactive projection-onto-convex-sets network (SSI-POCS) to tackle this severely ill-posed problem. By unfolding the classical iterative solver, our network integrates physical consistency with dual-domain deep priors via learned transforms, soft thresholding, and exact data reinsertion. Within the learned prior, a spatial Restormer branch captures local waveform structures, while a fast Fourier convolution (FFC) branch extracts global context, effectively suppressing incoherent components and enhancing spectral energy concentration in the $f-k$ domain. A dual-domain fusion module exchanges these complementary features via bidirectional channel attention and gated modulation, yielding substantial interpolation improvements. To overcome the severe loss of lateral continuity caused by large gaps, we introduce structural similarity (SSIM) supervision,…

    PDF
  • arXiv cs.CVOct 31

    3D Neural Rendering for Automated Ultrasonic Laser Scans

    Thomas Pierson, Matthew Mitchell, Kira Perna, Anand Iyer, Eric Flynn +1

    Laser doppler vibrometer (LDV) scans can be used to detect defects and cracks in manufactured parts before they are deployed. This work presents a method for reconstructing the steady-state surface wavefield of a three-dimensional part by combining automated robotic manipulation with neural rendering. A six-degree-of-freedom (6-DOF) robotic manipulator (UFactory xArm6) with an embedded piezoelectric transducer presented an acoustically excited test article to a stationary LDV at a sequence of known orientations, yielding a set of complex-valued 2D scans with poses derived analytically from the arm kinematics rather than from image-based pose estimation. These scans were used to train a novel complex-nerfacto model, a modified Neural Radiance Field (NeRF) pipeline that replaces the standard three-channel RGB output head with a two-channel output. Complex-nerfacto represents the real and imaginary velocity components of a wavefield, preserving the complex measurement without quantization. Reconstructions of a welded aluminum cylinder and a welded T-bracket were evaluated against measured scans using Z-normalized cross-correlation (ZNCC). The complex-valued pipeline achieved a mean…

    PDF
  • arXiv cs.CVOct 30

    Learning Clean Generative Processes from Noisy Measurements

    Fabian Kruse, Lorenzo Baldassari, Ivan Dokmanić

    We study how to learn a generative model of clean images using only noisy observations. Diffusion processes and their generalizations to non-Gaussian noise require posterior predictions at every noise level. We first show that it is possible to learn these predictions from independent noisy views using Noise2Noise regression, but only at or above the noise level of the training data. To learn the predictions below that level, needed to generate clean images, we impose consistency: a prediction should equal its expectation after continuing the sampling process. We derive conditions under which consistency uniquely determines the continuation to the clean image distribution and verify them for Poisson, binomial, Gamma, and negative-binomial measurements. Experiments on CIFAR-10, photon-counting microscopy, and synthetic aperture radar show that the required posterior predictions can be learned from noisy views alone and that the consistency relation repairs the continuation beyond the noise level of the data, which severely deteriorates without consistency.

    PDF
Load more