Spotlight

Papers

Field

Top tools here

Open problems in Supervised & diffusion models

Stable one-step generation in diffusion models1 paper raises it · 1 addresses it
Multi-step inference in diffusion models is slow, and one-step methods often collapse in performance due to unstable dynamics.
- Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Modelsraises · addresses
Evaluating model uncertainty with exact ground truth1 paper raises it · 1 addresses it
Real-world data lacks instance-level class probabilities, making it hard to assess model calibration and uncertainty.
- ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertaintyraises · addresses
Scalable optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit the benefits of optimal transport in flow matching, and scaling is computationally expensive.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Prior-consistent sampling for ambiguous inverse problems1 paper raises it · 1 addresses it
Diffusion-based solvers for nonlinear inverse problems can produce prior-inconsistent solutions when multiple modes are measurement-consistent.
- Beyond Best-of-$k$: Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problemsraises · addresses
High-dimensional Bayesian optimization with limited budget1 paper raises it · 1 addresses it
Latent spaces in generative models can be extremely high-dimensional, making optimization with limited evaluations inefficient.
- SABLE: Scalable Affine Bayesian Latent-space Explorationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

supervised · labeled · classification162 papers24 this week
amortized bayesian · bayesian networks · variational inference107 papers17 this week
diffusion models · generative models · monte carlo154 papers10 this week
bayesian inverse · diffusion prior · diffusion posterior58 papers9 this week
conformal prediction · split conformal prediction · prediction intervals80 papers7 this week
flow matching models · flow matching flow · matching flow matching93 papers5 this week

Topics in this field

Active contributors · 90 days

Henry MossSABLE: Scalable Affine Bayesian Latent-space Exploration5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Xiaoyu WangAccelerating Non-Smooth and Heavy-Tailed Sampling4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4
Eric MoulinesOne-Step Generation via Riemannian Wasserstein Gradient Flows4

Active contributors

Activity · 12 weeks

Activity

★ Spotlight this month

arXiv cs.AIOct 15

Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe, Khuushi Maheshwari

Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible through the score itself. We introduce RADAR (Repair, Adopt, and Diagnose After Recalibration), a reliability framework that separates confidence repair from deciding whether to deploy the repair and auditing the errors that remain. Under squared loss, we decompose prediction risk into outcome noise, information hidden by the score, and mapping error; restricting repair to order-preserving maps additionally introduces a pooling gap. For repair, we propose Probe-Anchored Isotonic Calibration (PAIC), which learns a monotone map from correctness-probe predictions on unlabeled target prompts. PAIC's excess risk depends on the probe's score-conditional bias and fitting error rather than its overall accuracy. For adoption, we derive a paired procedure with finite-sample control of false improvements. For auditing, we develop a prompt-clustered kernel test that detects feature-aligned residual errors while accounting for dependent answers from the same prompt. Across multiple LLM checkpoints, benchmarks, and distribution shifts, richer…

PDF arXiv page Details

Spotlight

Recent papers

All papers in Research

arXiv cs.CVOct 22
Beyond Best-of-$k$: Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problems
Merve Kocabas, Benjamin A. Burns, Andreas Geiger, Kent Gauen, Sara Fridovich-Keil
Diffusion models have become strong priors for solving inverse problems. However, nonlinear inverse problems such as phase retrieval remain challenging when multiple measurement-consistent solutions are distinguishable only by the prior; we call such measurements ambiguous. Existing diffusion-based samplers rely on local approximations during reconstruction. For an ambiguous measurement, these approximations can allow measurement guidance to favor a measurement-consistent mode that is close to a prior-supported mode but is itself not supported by the prior. Evaluation often masks this type of failure by reporting the best-of-$k$ runs, where the best is selected by comparison to the ground truth. We introduce Prior-Consistent DAPS (PC-DAPS), a variant of Decoupled Annealing Posterior Sampling (DAPS) designed to address this failure case by maintaining multiple candidate solutions within a single sampling run. As annealing noise decreases, PC-DAPS prunes and replaces weaker candidates according to their prior consistency, without any ground-truth access. PC-DAPS reduces the incidence of prior-inconsistent samples and improves sample quality on nonlinear inverse problems while…
PDF arXiv
arXiv cs.LGOct 22
SABLE: Scalable Affine Bayesian Latent-space Exploration
Erik Bodin, Lisa Maria Kreusser, Donney Fan, Henry Moss, Geoff Pleiss +1
Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces can have thousands to tens of thousands of dimensions, creating a regime where the latent dimension $D$ greatly exceeds the evaluation budget. We introduce SABLE, a Bayesian optimisation method for high-dimensional latent spaces with an isotropic distribution and limited budget. SABLE uses a Bayesian affine surrogate for prediction and acquisition, treating higher-order structure as surrogate discrepancy. After $N$ observations, the slope update has dimension at most $N-1$, yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm. We show that the off-span norm implicitly selected by affine EI can lead to underexploration, and SABLE instead prescribes it using a finite-horizon schedule, without restricting search to a learned subspace. A recompression scheme reduces information to…
PDF arXiv
arXiv eess.IVOct 22
Self-Supervised Multibeam Sonar Restoration via Coherent Forward Modeling
Zixin Wang, James Hopgood, Mike Davies
Multibeam imaging sonar produces intensity images degraded by coherent speckle and system-induced blur, while clean reflectivity targets are rarely available. Existing despeckling methods often rely on simplified intensity-domain noise models, synthetic clean targets, or blind-spot self-supervision, and typically do not jointly model deblurring and speckle suppression. We propose a physics-consistent self-supervised framework for joint despeckling and deblurring of multibeam imaging sonar. A stochastic coherent image-formation model yields a conditional-mean forward operator that relates latent scene reflectivity to expected intensity through the squared magnitude of the system point spread function. We then use registered multi-frame sonar sequences to train without clean ground truth images, showing that, under an idealized inter-frame conditional-mean condition, the expected paired loss equals the corresponding forward-prediction loss up to a network-independent constant. To stabilize the ill-conditioned inverse problem, we further adapt robust equivariant imaging using stochastic coherent simulation and introduce a paired forward prediction error metric for ground-truth-free…
PDF arXiv
arXiv stat.MLOct 22
Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis
Yuansi Chen, Francesco Pedrotti, Peter A. Whalley
The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$. In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$. In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$, we prove that dimension-free guarantees are impossible in general.
PDF arXiv
arXiv stat.MLOct 22
Metric Conformal Prediction via the Conditional Expected Local Radius
Rui Qiu, Fang Yao, Zhou Yu
We introduce the conditional expected local radius (CELR) and develop a new conformal prediction method for non-Euclidean responses in general metric spaces. CELR measures local conditional probability concentration, allowing prediction sets to accommodate heterogeneity and multimodality without requiring densities or a vector space structure. Its conditional distribution yields a nonconformity score that is comparable across covariate values and avoids costly transport comparisons between distance profiles. We further adapt a metric distributional random forest that handles multivariate covariates and nonlinear interactions, while a single forest fit supplies all score components, reducing computation by eliminating repeated local fitting. We establish uniform consistency of the estimators and asymptotic conditional validity under suitable regularity conditions, together with finite-sample marginal coverage under exchangeability. Numerical studies demonstrate effective uncertainty quantification across different metric spaces and substantial computational savings over the conditional profile score method.
PDF arXiv
arXiv cs.LGOct 22
The Bayesian Learning Rule Beyond KL Geometry
Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo, Thomas Moellenhoff +1
The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use $f$-divergences suggest that this is only possible with the KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum-entropy families, dual coordinates, and generalized-BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that \textit{cannot} be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed $q$-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable…
PDF arXiv

Papers

Topics in this field

supervised · labeled · classification162 papers24 this week
amortized bayesian · bayesian networks · variational inference107 papers17 this week
diffusion models · generative models · monte carlo154 papers10 this week
bayesian inverse · diffusion prior · diffusion posterior58 papers9 this week
conformal prediction · split conformal prediction · prediction intervals80 papers7 this week
flow matching models · flow matching flow · matching flow matching93 papers5 this week

Active contributors · 90 days

Henry MossSABLE: Scalable Affine Bayesian Latent-space Exploration5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Xiaoyu WangAccelerating Non-Smooth and Heavy-Tailed Sampling4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4
Eric MoulinesOne-Step Generation via Riemannian Wasserstein Gradient Flows4

★ Spotlight this month

arXiv cs.AIOct 15

Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe, Khuushi Maheshwari

Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible through the score itself. We introduce RADAR (Repair, Adopt, and Diagnose After Recalibration), a reliability framework that separates confidence repair from deciding whether to deploy the repair and auditing the errors that remain. Under squared loss, we decompose prediction risk into outcome noise, information hidden by the score, and mapping error; restricting repair to order-preserving maps additionally introduces a pooling gap. For repair, we propose Probe-Anchored Isotonic Calibration (PAIC), which learns a monotone map from correctness-probe predictions on unlabeled target prompts. PAIC's excess risk depends on the probe's score-conditional bias and fitting error rather than its overall accuracy. For adoption, we derive a paired procedure with finite-sample control of false improvements. For auditing, we develop a prompt-clustered kernel test that detects feature-aligned residual errors while accounting for dependent answers from the same prompt. Across multiple LLM checkpoints, benchmarks, and distribution shifts, richer…

PDF arXiv page Details

Activity · 12 weeks

Open problems in Supervised & diffusion models

Stable one-step generation in diffusion models1 paper raises it · 1 addresses it
Multi-step inference in diffusion models is slow, and one-step methods often collapse in performance due to unstable dynamics.
- Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Modelsraises · addresses
Evaluating model uncertainty with exact ground truth1 paper raises it · 1 addresses it
Real-world data lacks instance-level class probabilities, making it hard to assess model calibration and uncertainty.
- ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertaintyraises · addresses
Scalable optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit the benefits of optimal transport in flow matching, and scaling is computationally expensive.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Prior-consistent sampling for ambiguous inverse problems1 paper raises it · 1 addresses it
Diffusion-based solvers for nonlinear inverse problems can produce prior-inconsistent solutions when multiple modes are measurement-consistent.
- Beyond Best-of-$k$: Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problemsraises · addresses
High-dimensional Bayesian optimization with limited budget1 paper raises it · 1 addresses it
Latent spaces in generative models can be extremely high-dimensional, making optimization with limited evaluations inefficient.
- SABLE: Scalable Affine Bayesian Latent-space Explorationraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 41m ago

Supervised & diffusion models

Found by the topic model: supervised, diffusion models, amortized bayesian.

77papers this week−47.6% vs last week

All papers · 737 With code · 2

Newest Most relevant

623 papers this month

Recent papers

All papers in Research

arXiv cs.CVOct 22
Beyond Best-of-$k$: Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problems
Merve Kocabas, Benjamin A. Burns, Andreas Geiger, Kent Gauen, Sara Fridovich-Keil
Diffusion models have become strong priors for solving inverse problems. However, nonlinear inverse problems such as phase retrieval remain challenging when multiple measurement-consistent solutions are distinguishable only by the prior; we call such measurements ambiguous. Existing diffusion-based samplers rely on local approximations during reconstruction. For an ambiguous measurement, these approximations can allow measurement guidance to favor a measurement-consistent mode that is close to a prior-supported mode but is itself not supported by the prior. Evaluation often masks this type of failure by reporting the best-of-$k$ runs, where the best is selected by comparison to the ground truth. We introduce Prior-Consistent DAPS (PC-DAPS), a variant of Decoupled Annealing Posterior Sampling (DAPS) designed to address this failure case by maintaining multiple candidate solutions within a single sampling run. As annealing noise decreases, PC-DAPS prunes and replaces weaker candidates according to their prior consistency, without any ground-truth access. PC-DAPS reduces the incidence of prior-inconsistent samples and improves sample quality on nonlinear inverse problems while…
PDF arXiv
arXiv cs.LGOct 22
SABLE: Scalable Affine Bayesian Latent-space Exploration
Erik Bodin, Lisa Maria Kreusser, Donney Fan, Henry Moss, Geoff Pleiss +1
Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces can have thousands to tens of thousands of dimensions, creating a regime where the latent dimension $D$ greatly exceeds the evaluation budget. We introduce SABLE, a Bayesian optimisation method for high-dimensional latent spaces with an isotropic distribution and limited budget. SABLE uses a Bayesian affine surrogate for prediction and acquisition, treating higher-order structure as surrogate discrepancy. After $N$ observations, the slope update has dimension at most $N-1$, yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm. We show that the off-span norm implicitly selected by affine EI can lead to underexploration, and SABLE instead prescribes it using a finite-horizon schedule, without restricting search to a learned subspace. A recompression scheme reduces information to…
PDF arXiv
arXiv eess.IVOct 22
Self-Supervised Multibeam Sonar Restoration via Coherent Forward Modeling
Zixin Wang, James Hopgood, Mike Davies
Multibeam imaging sonar produces intensity images degraded by coherent speckle and system-induced blur, while clean reflectivity targets are rarely available. Existing despeckling methods often rely on simplified intensity-domain noise models, synthetic clean targets, or blind-spot self-supervision, and typically do not jointly model deblurring and speckle suppression. We propose a physics-consistent self-supervised framework for joint despeckling and deblurring of multibeam imaging sonar. A stochastic coherent image-formation model yields a conditional-mean forward operator that relates latent scene reflectivity to expected intensity through the squared magnitude of the system point spread function. We then use registered multi-frame sonar sequences to train without clean ground truth images, showing that, under an idealized inter-frame conditional-mean condition, the expected paired loss equals the corresponding forward-prediction loss up to a network-independent constant. To stabilize the ill-conditioned inverse problem, we further adapt robust equivariant imaging using stochastic coherent simulation and introduce a paired forward prediction error metric for ground-truth-free…
PDF arXiv
arXiv stat.MLOct 22
Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis
Yuansi Chen, Francesco Pedrotti, Peter A. Whalley
The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$. In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$. In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$, we prove that dimension-free guarantees are impossible in general.
PDF arXiv
arXiv stat.MLOct 22
Metric Conformal Prediction via the Conditional Expected Local Radius
Rui Qiu, Fang Yao, Zhou Yu
We introduce the conditional expected local radius (CELR) and develop a new conformal prediction method for non-Euclidean responses in general metric spaces. CELR measures local conditional probability concentration, allowing prediction sets to accommodate heterogeneity and multimodality without requiring densities or a vector space structure. Its conditional distribution yields a nonconformity score that is comparable across covariate values and avoids costly transport comparisons between distance profiles. We further adapt a metric distributional random forest that handles multivariate covariates and nonlinear interactions, while a single forest fit supplies all score components, reducing computation by eliminating repeated local fitting. We establish uniform consistency of the estimators and asymptotic conditional validity under suitable regularity conditions, together with finite-sample marginal coverage under exchangeability. Numerical studies demonstrate effective uncertainty quantification across different metric spaces and substantial computational savings over the conditional profile score method.
PDF arXiv
arXiv cs.LGOct 22
The Bayesian Learning Rule Beyond KL Geometry
Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo, Thomas Moellenhoff +1
The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use $f$-divergences suggest that this is only possible with the KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum-entropy families, dual coordinates, and generalized-BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that \textit{cannot} be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed $q$-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable…
PDF arXiv