Fields · AI · updated 27m ago

Supervised & diffusion models

Found by the topic model: supervised, diffusion models, amortized bayesian.

96papers this week−21.3% vs last week

All papers · 811 With code · 2

Newest Most relevant

625 papers this month

Papers

Supervised & diffusion models · ByteTrail

Fields · AI · updated 27m ago

Supervised & diffusion models

Found by the topic model: supervised, diffusion models, amortized bayesian.

96papers this week−21.3% vs last week

All papers · 811 With code · 2

Newest Most relevant

625 papers this month

Recent papers

All papers in Research

arXiv cs.LGOct 1
Efficient Neural Field Learning via Adaptive Coverage and Focused Sampling
Guang Zhao, Xihaier Luo, Huan-Hsin Tseng, Seungjun Lee, Shinjae Yoo +2
Implicit neural representations (INRs) provide a flexible framework for modeling high-dimensional continuous fields, but their training is often inefficient due to uniform subsampling that ignores spatial heterogeneity. Existing adaptive sampling methods partially address this issue by prioritizing high-error samples, but typically operate at the point level, often leading to redundant sampling in localized regions and insufficient coverage of the domain. We propose ACES (Adaptive Coverage-aware Efficient Sampling), a structured sampling framework that improves training efficiency by decoupling coverage and importance. ACES constructs adaptive spatial partitions to ensure domain coverage and reduce redundancy, and applies region-level importance weighting to prioritize informative regions during training. We provide a theoretical analysis showing that adaptive partitioning reduces gradient variance by increasing within-region homogeneity, and that controlled bias in region-level weighting may improve optimization efficiency relative to standard unbiased estimators. Experiments on scientific field learning tasks demonstrate that ACES achieves faster convergence and lower error…
PDF arXiv
arXiv math.NAOct 1
Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems
Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke, Björn Sprungk
Sampling from the posterior is the central task of computational Bayesian inverse problems. The standard workhorse in Bayesian inference - Markov chain Monte Carlo (MCMC) - is sequential, yields correlated samples, and must be rerun for each observation. Conditional flow matching offers an amortized alternative: a transport map, trained once on joint samples of parameter and data, that yields independent approximate posterior samples for any observation at negligible online cost, without new likelihood evaluations. We give a careful, MCMC-literate assessment of flow matching for PDE-based inverse problems with function-valued parameters. Exploiting the flow's tractable density, we derive computable accuracy estimates of the underlying approximate posterior in total-variation distance and Kullback-Leibler divergence and, moreover, propose a hybrid sampler that is asymptotically exact by Metropolization. We validate the accuracy estimates and demonstrate the amortization in several numerical examples, including electrical impedance tomography and a likelihood-free Lotka-Volterra model.
PDF arXiv
arXiv stat.MLOct 1
Conformal Prediction for Time Series with Deep Sequence Models
Junghwan Lee, Jonghyeok Lee, Yao Xie
Recent advances in deep learning for time series prediction have amplified the need for reliable uncertainty quantification. Conformal prediction has gained attention as a distribution-free framework for constructing prediction intervals with coverage guarantees. However, its coverage guarantees rely on data exchangeability, an assumption generally violated in time series. Active research has focused on developing conformal prediction methods for time series that overcome this limitation. While deep sequence models, such as recurrent neural networks and Transformers, have often been used in conformal prediction for time series, limited work has systematically studied how deep sequence models can be utilized in conformal prediction for time series. In this work, we systematically investigate the use of deep sequence models in conformal prediction for time series through three approaches: conditional quantile regression, conditional quantile function estimation, and localized conformal prediction. We provide a theoretical analysis establishing asymptotic conditional coverage guarantees for all three approaches under suitable assumptions. Through comprehensive experiments on…
PDF arXiv
arXiv cs.LGOct 1
Muon meets Tamed Langevin: Momentum Preconditioning beyond Convex and gradient-Lipschitz Potentials
Nikolaos Makras, Sotirios Sabanis
We consider the problem of sampling from Gibbs distributions on matrix spaces whose potential energies are neither convex nor globally gradient-Lipschitz. We introduce a family of non-quadratic kinetic energies that lead to a new underdamped Langevin system with momentum preconditioning, in which the gradient of the kinetic energy acts as a smooth spectral taming of the momentum. We prove that, under these relaxed assumptions on the potential, the resulting dynamics leaves the target Gibbs measure invariant, and we establish exponential convergence to equilibrium in a weighted total variation distance. Finally, we show that the corresponding Euler-Maruyama discretization admits moment bounds that are uniform in time, without any modification of the potential gradient, which ensures the stability of the resulting sampling algorithm.
PDF arXiv
arXiv cs.LGOct 1
$Ψ$-Resilience: Model-Free Feature Importance from 1D Topological Signals
Fabian Galis, Darian Onchis, Pedro Real Jurado
We introduce $Ψ$-Resilience, a model-free feature importance method that derives explanations directly from the data itself via 1D topological signals. Our method constructs a class-disagreement landscape by estimating class-conditional densities and taking their pointwise absolute difference along the feature axis. Then, the 0-dimensional persistence of this 1D signal defines a resilience functional that aggregates only those topological features that survive perturbations up to a robustness scale which is set by the user. This gives us a context-robust importance score that is inherently auditable via the underlying 1D landscapes and their persistence. We evaluate our method on both synthetic and real datasets. On synthetic generators with specified ground-truth importance, $Ψ$-Resilience recovers the ranking of features with high fidelity, achieving Spearman rank correlations up to 0.8 and performing competitively with multiple feature importance methods, including SHAP and mutual information. On real datasets with no known ground truth, our technique agrees with these methods, with correlations up to 0.9. These results show that $Ψ$-Resilience is a stable explanation method…
PDF arXiv
arXiv math.STOct 1
Sample complexity bounds for categorical Markov random fields via Discrete Diffusions
Shivam Kumar, Nabarun Deb
Many applications in statistics, economics, and physics require sampling from high-dimensional categorical distributions with local dependence structures. Examples include finite memory language models, Ising and Potts systems in statistical physics and protein folding, etc. In modern machine learning, discrete diffusions have emerged as a flexible approach for sampling such data, with strong empirical performance. Motivated by this, we develop learning methods with end-to-end sample complexity bounds for discrete diffusion with uniform noising under local dependence, which we model through low order Markov random fields (MRFs). Our main technical insight is a new \emph{pinning decomposition} of the discrete score. It shows that unlike in continuous diffusions, the score decomposes into components where the dependence on time separates multiplicatively from the dependence on the target. Building on this decomposition, we propose a \emph{weight-sharing neural score learner} and combine it with $τ$-leaping to obtain an end-to-end sampling procedure. Rather than treating score-learning error as a black-box input, as is common in existing sampling analyses, we study the score…
PDF arXiv

Papers

Open problems in Supervised & diffusion models

Lack of benchmarks for model uncertainty evaluation1 paper raises it · 1 addresses it
Evaluating model uncertainty requires exact ground-truth probabilities, which are typically unavailable in real-world data.
- ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertaintyraises · addresses
High inference latency in diffusion-based action generation1 paper raises it · 1 addresses it
Multi-step flow matching introduces computational overhead, limiting real-time applicability in robotics.
- Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Modelsraises · addresses
Scalability and stability in flow matching with optimal transport1 paper raises it · 1 addresses it
Small minibatches limit the benefits of OT coupling, while large minibatches are computationally expensive, and theoretical gaps remain in gradient variance reduction.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Failure to learn symmetries in diffusion models1 paper raises it · 1 addresses it
Symmetry breaking in diffusion models reduces sample efficiency and generalization, with initialization and finite-sample effects as key sources.
- Quantifying the failure of symmetry learning in linear diffusion models due to initialization and finite-sample effectsraises · addresses
Unreliable label-free monitoring in self-supervised learning1 paper raises it · 0 address it
Standard loss and feature statistics can fail to detect degradation in learned representations, leading to misleading model selection.
- Improving Signals, Declining Encoder: A Known-Answer Test of Label-Free Monitoring in Self-Supervised Learningraises

Named by an AI from these papers' abstracts.

Open problems

Topics in this field

supervised · labeled · classification181 papers23 this week
diffusion models · generative models · monte carlo167 papers16 this week
amortized bayesian · bayesian networks · variational inference116 papers14 this week
flow matching models · flow matching flow · matching flow matching103 papers13 this week
bayesian optimization · bayesian optimization bo · black box optimization91 papers11 this week
conformal prediction · split conformal prediction · prediction intervals89 papers11 this week

Topics in this field

Active contributors · 90 days

Henry MossSABLE: Scalable Affine Bayesian Latent-space Exploration5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Xiaoyu WangAccelerating Non-Smooth and Heavy-Tailed Sampling4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4
Eric MoulinesOne-Step Generation via Riemannian Wasserstein Gradient Flows4

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGOct 1
Efficient Neural Field Learning via Adaptive Coverage and Focused Sampling
Guang Zhao, Xihaier Luo, Huan-Hsin Tseng, Seungjun Lee, Shinjae Yoo +2
Implicit neural representations (INRs) provide a flexible framework for modeling high-dimensional continuous fields, but their training is often inefficient due to uniform subsampling that ignores spatial heterogeneity. Existing adaptive sampling methods partially address this issue by prioritizing high-error samples, but typically operate at the point level, often leading to redundant sampling in localized regions and insufficient coverage of the domain. We propose ACES (Adaptive Coverage-aware Efficient Sampling), a structured sampling framework that improves training efficiency by decoupling coverage and importance. ACES constructs adaptive spatial partitions to ensure domain coverage and reduce redundancy, and applies region-level importance weighting to prioritize informative regions during training. We provide a theoretical analysis showing that adaptive partitioning reduces gradient variance by increasing within-region homogeneity, and that controlled bias in region-level weighting may improve optimization efficiency relative to standard unbiased estimators. Experiments on scientific field learning tasks demonstrate that ACES achieves faster convergence and lower error…
PDF arXiv
arXiv math.NAOct 1
Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems
Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke, Björn Sprungk
Sampling from the posterior is the central task of computational Bayesian inverse problems. The standard workhorse in Bayesian inference - Markov chain Monte Carlo (MCMC) - is sequential, yields correlated samples, and must be rerun for each observation. Conditional flow matching offers an amortized alternative: a transport map, trained once on joint samples of parameter and data, that yields independent approximate posterior samples for any observation at negligible online cost, without new likelihood evaluations. We give a careful, MCMC-literate assessment of flow matching for PDE-based inverse problems with function-valued parameters. Exploiting the flow's tractable density, we derive computable accuracy estimates of the underlying approximate posterior in total-variation distance and Kullback-Leibler divergence and, moreover, propose a hybrid sampler that is asymptotically exact by Metropolization. We validate the accuracy estimates and demonstrate the amortization in several numerical examples, including electrical impedance tomography and a likelihood-free Lotka-Volterra model.
PDF arXiv
arXiv stat.MLOct 1
Conformal Prediction for Time Series with Deep Sequence Models
Junghwan Lee, Jonghyeok Lee, Yao Xie
Recent advances in deep learning for time series prediction have amplified the need for reliable uncertainty quantification. Conformal prediction has gained attention as a distribution-free framework for constructing prediction intervals with coverage guarantees. However, its coverage guarantees rely on data exchangeability, an assumption generally violated in time series. Active research has focused on developing conformal prediction methods for time series that overcome this limitation. While deep sequence models, such as recurrent neural networks and Transformers, have often been used in conformal prediction for time series, limited work has systematically studied how deep sequence models can be utilized in conformal prediction for time series. In this work, we systematically investigate the use of deep sequence models in conformal prediction for time series through three approaches: conditional quantile regression, conditional quantile function estimation, and localized conformal prediction. We provide a theoretical analysis establishing asymptotic conditional coverage guarantees for all three approaches under suitable assumptions. Through comprehensive experiments on…
PDF arXiv
arXiv cs.LGOct 1
Muon meets Tamed Langevin: Momentum Preconditioning beyond Convex and gradient-Lipschitz Potentials
Nikolaos Makras, Sotirios Sabanis
We consider the problem of sampling from Gibbs distributions on matrix spaces whose potential energies are neither convex nor globally gradient-Lipschitz. We introduce a family of non-quadratic kinetic energies that lead to a new underdamped Langevin system with momentum preconditioning, in which the gradient of the kinetic energy acts as a smooth spectral taming of the momentum. We prove that, under these relaxed assumptions on the potential, the resulting dynamics leaves the target Gibbs measure invariant, and we establish exponential convergence to equilibrium in a weighted total variation distance. Finally, we show that the corresponding Euler-Maruyama discretization admits moment bounds that are uniform in time, without any modification of the potential gradient, which ensures the stability of the resulting sampling algorithm.
PDF arXiv
arXiv cs.LGOct 1
$Ψ$-Resilience: Model-Free Feature Importance from 1D Topological Signals
Fabian Galis, Darian Onchis, Pedro Real Jurado
We introduce $Ψ$-Resilience, a model-free feature importance method that derives explanations directly from the data itself via 1D topological signals. Our method constructs a class-disagreement landscape by estimating class-conditional densities and taking their pointwise absolute difference along the feature axis. Then, the 0-dimensional persistence of this 1D signal defines a resilience functional that aggregates only those topological features that survive perturbations up to a robustness scale which is set by the user. This gives us a context-robust importance score that is inherently auditable via the underlying 1D landscapes and their persistence. We evaluate our method on both synthetic and real datasets. On synthetic generators with specified ground-truth importance, $Ψ$-Resilience recovers the ranking of features with high fidelity, achieving Spearman rank correlations up to 0.8 and performing competitively with multiple feature importance methods, including SHAP and mutual information. On real datasets with no known ground truth, our technique agrees with these methods, with correlations up to 0.9. These results show that $Ψ$-Resilience is a stable explanation method…
PDF arXiv
arXiv math.STOct 1
Sample complexity bounds for categorical Markov random fields via Discrete Diffusions
Shivam Kumar, Nabarun Deb
Many applications in statistics, economics, and physics require sampling from high-dimensional categorical distributions with local dependence structures. Examples include finite memory language models, Ising and Potts systems in statistical physics and protein folding, etc. In modern machine learning, discrete diffusions have emerged as a flexible approach for sampling such data, with strong empirical performance. Motivated by this, we develop learning methods with end-to-end sample complexity bounds for discrete diffusion with uniform noising under local dependence, which we model through low order Markov random fields (MRFs). Our main technical insight is a new \emph{pinning decomposition} of the discrete score. It shows that unlike in continuous diffusions, the score decomposes into components where the dependence on time separates multiplicatively from the dependence on the target. Building on this decomposition, we propose a \emph{weight-sharing neural score learner} and combine it with $τ$-leaping to obtain an end-to-end sampling procedure. Rather than treating score-learning error as a black-box input, as is common in existing sampling analyses, we study the score…
PDF arXiv