Papers

Field

Top tools here

Open problems in Flow matching

Scalable and stable training for flow matching1 paper raises it · 1 addresses it
Large-scale flow models struggle with training stability and integration error due to stochastic source-target pairing.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Efficient few-step generation without fixed discretization1 paper raises it · 1 addresses it
Few-step generation often requires heuristic timestep discretization, limiting flexibility and performance.
- Few-Step Generation via Data-Space Iterationraises · addresses
Data-free and simulation-free distillation for diffusion/flow models1 paper raises it · 1 addresses it
Distillation often relies on curated datasets or costly teacher rollouts, complicating training and scaling.
- Consistent Distribution Matching for Data-Free Diffusion Distillationraises · addresses
Dynamics-aligned latent representations for PDEs1 paper raises it · 1 addresses it
Reconstruction-based learning for PDEs often fails to capture the evolution of physical fields accurately.
- PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEsraises · addresses
Detecting memorization in diffusion models1 paper raises it · 1 addresses it
Memorization in diffusion models can lead to failure in proper distributional learning.
- Detection of Memorization in Diffusion Models Through Thermodynamic Singularityraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

flow matching · flow · generative modeling292 papers31 this week
physics informed neural · neural operators · informed neural networks291 papers26 this week
diffusion priors · posterior sampling · diffusion posterior73 papers14 this week

Topics in this field

Active contributors · 90 days

Anima AnandkumarNeural Operator Splitting for Fluid Dynamics in Complex Geometry5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Jing XuSame Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation4
Jun ZhangEqual Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv math.NAOct 6
FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations
Shun Zhang
We construct neural approximation classes native to the graph spaces H(div) and H(curl), in two and three dimensions and, for H(div), in any dimension. Every realization lies in the space for all parameter values, and with kinked potentials, such as ReLU networks, the admissible jumps appear at finite width. The classes are images of scalar and componentwise networks under fixed operators of the de Rham complex, and do not involve a mesh or finite element emulation. For H(div) in R^n two native classes are given on an equal footing, with a skew-symmetric potential $A$: $\mathrm{Div}\,A+R_nq+\mathbf{h}$, with the divergence $q$ as an explicit unknown, and $\mathrm{Div}\,A+\mathbf{z}$ with an $H^1$ field $\mathbf{z}$; for H(curl) the analogous classes are $\mathrm{grad}\,φ+Sr+\mathbf{h}$ in two dimensions and $\mathrm{grad}\,φ+\mathbf{z}$ in two and three dimensions. In all of them every interface jump of the field is carried by the potential term, $\mathrm{Div}\,A$ or $\mathrm{grad}\,φ$, while the remaining part has no interface jump (it is an $H^1$ field in the regular-decomposition classes); the classes with $\mathbf{z}$ are the componentwise approach enriched by this term.…
PDF arXiv
arXiv cs.LGOct 6
Generalized Matheron Variational Implicit Processes
Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen
Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression,…
PDF arXiv
arXiv math.NAOct 6
Learned Adaptive Multiresolution Diffusion Imaging
Christian Tantardini, Stig Rune Jensen, Roberto Di Remigio Eikås, Joakim Henrik Beck
Adaptive multiresolution methods reduce representation cost by concentrating fine-scale degrees of freedom where needed, but their tree updates are usually governed by fixed local criteria. We introduce Learned Adaptive Multiresolution Diffusion Imaging (Learned AMDI), which preserves the AMDI fixed-tree propagator and hierarchy constraints while replacing the post-propagation selector with a shared local policy trained by proximal policy optimization. Regression tests reproduce deterministic AMDI trajectories to machine precision when identical trees are used. In the Haar implementation studied here, the deterministic one-step selector accepts no refinements in 54 decisions. Across nine held-out cases, Learned AMDI executes 393 refinements and reduces the mean terminal reference discrepancy from $0.17496$ to $0.13657$, while occupancy rises from $0.13737$ to $0.26660$. Step-resolved diagnostics reveal occasional small adaptation-energy increases; fixed-tree energy stability therefore does not guarantee monotonicity of the learned outer iteration. At comparable occupancy, a validation-tuned observed-detail threshold reaches a discrepancy of $0.13792$ with slightly better RMSE and…
PDF arXiv
arXiv cs.LGOct 6
Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
Martin Eppert, Krishna Balasubramanian, Subhro Ghosh, Jason Klusowski, Yan Shuo Tan
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the property of statistical adaptivity, that is, they perform nearly as well as a method tailored to the true data-generating model for a heterogeneous set of models, while not being told which model the data comes from. We study how such adaptivity is learned in a controlled location-estimation problem. Each task is an unlabeled sample whose family is hidden: Gaussian data call for averaging, with error of order $n^{-1}$, whereas uniform data are best estimated from their extremes, at the faster rate $n^{-2}$. We also provide the example of a symmetric Gaussian mixture, for which a rate of $σ^2_n/n$ can be attained. On scalar inputs, softmax attention computes the derivative of the empirical cumulant-generating function. A single primitive therefore both supplies features that distinguish the families and forms estimators interpolating between the sample mean and the mid-range. We combine attention experts through either a softmax mixture of experts or a gated linear unit (GLU), and analyze stagewise gradient…
PDF arXiv
arXiv cs.LGOct 6
Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation
Donghan He, Luhuan Wu
Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete variables, however, the pathwise identity cannot generally be exact for every differentiable function. We propose a general framework to construct finite-order exact pathwise gradient estimators for a range of common discrete variables such as Poisson. The estimator is the least-norm solution among all solutions that are unbiased for polynomials of degree at most. The resulting estimators preserve the hard forward sample, require no temperature tuning, and can be implemented in a few lines of codes. Against other admissible solutions, our estimator is unique and minimizes weight variance; in contrast, prior works use categorical variables or augmented representations to approximate non-categorical variables that induces excess variance and computations. To understand approximation bias for functions beyond the prescribed class, we also derive a non-asymptotic bias bound. In experiments our low order methods match or improve tuned baselines across linear, nonlinear and hierarchical latent-variable models, while out-speeding…
PDF arXiv
arXiv math.NAOct 6
A Neural JKO Scheme for Hellinger-Kantorovich Gradient Flows via Monge-Growth Pairs
Geuntaek Seo, Cheolhyeong Kim, Hwijae Son, Hyung Ju Hwang
We develop a mesh-free neural JKO scheme for advection-reaction-diffusion equations with a gradient-flow structure in the Hellinger-Kantorovich (HK) geometry of unbalanced optimal transport. Each update is parametrized by a spatial map and a mass-changing factor, allowing spatial redistribution and local mass creation or loss to be treated jointly within a single variational step. Their cone action bounds the squared HK distance from above, yielding a sufficient condition for discrete energy dissipation through comparison with the identity pair. Minimizing the pair objective over all admissible pairs recovers the exact JKO minimum when the source and a minimizer have positive densities. We establish existence and mass bounds for JKO minimizers and, under additional assumptions, obtain positivity and regularity together with a discrete Euler-Lagrange equation and a metric-dissipation identity. The self-consistent chemical potential is then nonincreasing along an optimal map. There exist parametric pairs whose endpoint densities and objective values converge to those of an exact JKO minimizer, provided a regular-pair approximation hypothesis holds. Finally, we show that a…
PDF arXiv

Papers

Topics in this field

flow matching · flow · generative modeling292 papers31 this week
physics informed neural · neural operators · informed neural networks291 papers26 this week
diffusion priors · posterior sampling · diffusion posterior73 papers14 this week

Active contributors · 90 days

Anima AnandkumarNeural Operator Splitting for Fluid Dynamics in Complex Geometry5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Jing XuSame Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation4
Jun ZhangEqual Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4

Activity · 12 weeks

Coming with more sources: repos

Open problems in Flow matching

Scalable and stable training for flow matching1 paper raises it · 1 addresses it
Large-scale flow models struggle with training stability and integration error due to stochastic source-target pairing.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Efficient few-step generation without fixed discretization1 paper raises it · 1 addresses it
Few-step generation often requires heuristic timestep discretization, limiting flexibility and performance.
- Few-Step Generation via Data-Space Iterationraises · addresses
Data-free and simulation-free distillation for diffusion/flow models1 paper raises it · 1 addresses it
Distillation often relies on curated datasets or costly teacher rollouts, complicating training and scaling.
- Consistent Distribution Matching for Data-Free Diffusion Distillationraises · addresses
Dynamics-aligned latent representations for PDEs1 paper raises it · 1 addresses it
Reconstruction-based learning for PDEs often fails to capture the evolution of physical fields accurately.
- PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEsraises · addresses
Detecting memorization in diffusion models1 paper raises it · 1 addresses it
Memorization in diffusion models can lead to failure in proper distributional learning.
- Detection of Memorization in Diffusion Models Through Thermodynamic Singularityraises · addresses

Named by an AI from these papers' abstracts.

Recent papers

All in Research →

arXiv math.NAOct 6
FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations
Shun Zhang
We construct neural approximation classes native to the graph spaces H(div) and H(curl), in two and three dimensions and, for H(div), in any dimension. Every realization lies in the space for all parameter values, and with kinked potentials, such as ReLU networks, the admissible jumps appear at finite width. The classes are images of scalar and componentwise networks under fixed operators of the de Rham complex, and do not involve a mesh or finite element emulation. For H(div) in R^n two native classes are given on an equal footing, with a skew-symmetric potential $A$: $\mathrm{Div}\,A+R_nq+\mathbf{h}$, with the divergence $q$ as an explicit unknown, and $\mathrm{Div}\,A+\mathbf{z}$ with an $H^1$ field $\mathbf{z}$; for H(curl) the analogous classes are $\mathrm{grad}\,φ+Sr+\mathbf{h}$ in two dimensions and $\mathrm{grad}\,φ+\mathbf{z}$ in two and three dimensions. In all of them every interface jump of the field is carried by the potential term, $\mathrm{Div}\,A$ or $\mathrm{grad}\,φ$, while the remaining part has no interface jump (it is an $H^1$ field in the regular-decomposition classes); the classes with $\mathbf{z}$ are the componentwise approach enriched by this term.…
PDF arXiv
arXiv cs.LGOct 6
Generalized Matheron Variational Implicit Processes
Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen
Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression,…
PDF arXiv
arXiv math.NAOct 6
Learned Adaptive Multiresolution Diffusion Imaging
Christian Tantardini, Stig Rune Jensen, Roberto Di Remigio Eikås, Joakim Henrik Beck
Adaptive multiresolution methods reduce representation cost by concentrating fine-scale degrees of freedom where needed, but their tree updates are usually governed by fixed local criteria. We introduce Learned Adaptive Multiresolution Diffusion Imaging (Learned AMDI), which preserves the AMDI fixed-tree propagator and hierarchy constraints while replacing the post-propagation selector with a shared local policy trained by proximal policy optimization. Regression tests reproduce deterministic AMDI trajectories to machine precision when identical trees are used. In the Haar implementation studied here, the deterministic one-step selector accepts no refinements in 54 decisions. Across nine held-out cases, Learned AMDI executes 393 refinements and reduces the mean terminal reference discrepancy from $0.17496$ to $0.13657$, while occupancy rises from $0.13737$ to $0.26660$. Step-resolved diagnostics reveal occasional small adaptation-energy increases; fixed-tree energy stability therefore does not guarantee monotonicity of the learned outer iteration. At comparable occupancy, a validation-tuned observed-detail threshold reaches a discrepancy of $0.13792$ with slightly better RMSE and…
PDF arXiv
arXiv cs.LGOct 6
Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
Martin Eppert, Krishna Balasubramanian, Subhro Ghosh, Jason Klusowski, Yan Shuo Tan
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the property of statistical adaptivity, that is, they perform nearly as well as a method tailored to the true data-generating model for a heterogeneous set of models, while not being told which model the data comes from. We study how such adaptivity is learned in a controlled location-estimation problem. Each task is an unlabeled sample whose family is hidden: Gaussian data call for averaging, with error of order $n^{-1}$, whereas uniform data are best estimated from their extremes, at the faster rate $n^{-2}$. We also provide the example of a symmetric Gaussian mixture, for which a rate of $σ^2_n/n$ can be attained. On scalar inputs, softmax attention computes the derivative of the empirical cumulant-generating function. A single primitive therefore both supplies features that distinguish the families and forms estimators interpolating between the sample mean and the mid-range. We combine attention experts through either a softmax mixture of experts or a gated linear unit (GLU), and analyze stagewise gradient…
PDF arXiv
arXiv cs.LGOct 6
Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation
Donghan He, Luhuan Wu
Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete variables, however, the pathwise identity cannot generally be exact for every differentiable function. We propose a general framework to construct finite-order exact pathwise gradient estimators for a range of common discrete variables such as Poisson. The estimator is the least-norm solution among all solutions that are unbiased for polynomials of degree at most. The resulting estimators preserve the hard forward sample, require no temperature tuning, and can be implemented in a few lines of codes. Against other admissible solutions, our estimator is unique and minimizes weight variance; in contrast, prior works use categorical variables or augmented representations to approximate non-categorical variables that induces excess variance and computations. To understand approximation bias for functions beyond the prescribed class, we also derive a non-asymptotic bias bound. In experiments our low order methods match or improve tuned baselines across linear, nonlinear and hierarchical latent-variable models, while out-speeding…
PDF arXiv
arXiv math.NAOct 6
A Neural JKO Scheme for Hellinger-Kantorovich Gradient Flows via Monge-Growth Pairs
Geuntaek Seo, Cheolhyeong Kim, Hwijae Son, Hyung Ju Hwang
We develop a mesh-free neural JKO scheme for advection-reaction-diffusion equations with a gradient-flow structure in the Hellinger-Kantorovich (HK) geometry of unbalanced optimal transport. Each update is parametrized by a spatial map and a mass-changing factor, allowing spatial redistribution and local mass creation or loss to be treated jointly within a single variational step. Their cone action bounds the squared HK distance from above, yielding a sufficient condition for discrete energy dissipation through comparison with the identity pair. Minimizing the pair objective over all admissible pairs recovers the exact JKO minimum when the source and a minimizer have positive densities. We establish existence and mass bounds for JKO minimizers and, under additional assumptions, obtain positivity and regularity together with a discrete Euler-Lagrange equation and a metric-dissipation identity. The self-consistent chemical potential is then nonincreasing along an optimal map. There exist parametric pairs whose endpoint densities and objective values converge to those of an exact JKO minimizer, provided a regular-pair approximation hypothesis holds. Finally, we show that a…
PDF arXiv

Fields · AI · updated 21m ago

Flow matching

Found by the topic model: flow matching, physics informed neural, diffusion priors.

71papers this week−54.8% vs last week

All papers · 656 With code · 3

Newest Most relevant

621 papers this month