Papers

Field

Top tools here

Open problems in Flow matching

Stable and scalable source-target pairing for flow matching1 paper raises it · 1 addresses it
Unstable or random pairing degrades training and integration, limiting model performance at scale.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Efficient few-step generation without fixed discretization1 paper raises it · 1 addresses it
Fixed timestep discretization in flow matching is heuristic, expensive to tune, and restrictive for varying sample difficulty.
- Few-Step Generation via Data-Space Iterationraises · addresses
Data-free and simulation-free distillation for flow/diffusion models1 paper raises it · 1 addresses it
Existing distillation methods rely on curated datasets, costly teacher rollouts, or auxiliary networks, complicating training and scaling.
- Consistent Distribution Matching for Data-Free Diffusion Distillationraises · addresses
Accelerating numerical integration for neuro-spectral PDE solvers1 paper raises it · 1 addresses it
Sequential time-stepping in neuro-spectral architectures limits temporal parallelism and slows training/inference.
- PaPi: Parallel Picard-Duhamel Integration for Fast Neuro-Spectral Architecturesraises · addresses
Dynamics-aligned latent spaces for PDE representation learning1 paper raises it · 1 addresses it
Reconstruction-based objectives fail to capture evolution geometry, limiting accurate field prediction.
- PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

flow matching · flow · generative modeling299 papers33 this week
physics informed neural · neural operators · informed neural networks295 papers27 this week
diffusion priors · posterior sampling · diffusion posterior74 papers15 this week

Topics in this field

Active contributors · 90 days

Anima AnandkumarNeural Operator Splitting for Fluid Dynamics in Complex Geometry5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Jing XuSame Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation4
Jun ZhangEqual Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGSep 28
FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models
Yinuo Ren, Haoxuan Chen, Grant M. Rotskoff, Jiequn Han, Lexing Ying
Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the pretrained distribution. Feynman-Kac sequential Monte Carlo (SMC) makes this correction exact in principle, but its prescribed weights routinely degenerate when the proposal dynamics are misaligned with the tilt, capping the practical gains from additional particles. We introduce FluxLite, a lightweight, training-free proposal-control framework for discrete diffusion. On the sparse directed graph of pretrained reverse rates, any sparse jump-rate perturbation can be exactly compensated by a $q_t$-weighted graph-divergence term in the Feynman-Kac potential; the target path is therefore preserved while the residual reweighting variance becomes a local convex objective. We instantiate this principle as two practical samplers: a one-hop local reallocation rule (HEU) and a small nonnegative quadratic program over pretrained-rate bases (D-VCG). We further prove population stability under the standard score-entropy training loss, identifying a tilted-path coverage factor that governs robustness to score error, together with…
PDF arXiv
arXiv cs.LGSep 28
Simplex Diffusion Models
Justin Deschenaux, Alexandre Galashov, Andrew Campbell, Li Kevin Wenliang, James Thornton +2
Diffusion models have revolutionized generative modeling for continuous data through the gradual refinement of a belief state. This iterative refinement has not yet carried over to discrete diffusion models, which discard uncertainty at intermediate steps through categorical sampling (information collapse). We propose Simplex Diffusion Models (SDMs), a framework that lifts the diffusion process to the probability simplex to represent beliefs over categories. SDMs admit probability paths with closed-form reverse transitions and can be trained with a simple cross-entropy loss. Contrary to earlier proposals such as Dirichlet Flow Matching which requires integrating an ordinary differential equation, we introduce a DDIM-like sampler with a tunable level of stochasticity. Because SDMs operate on samples on the simplex, they can carry uncertainty across denoising steps, which mitigates information collapse. On OpenWebText, SDMs are competitive with strong Discrete Diffusion baselines, achieving $17.0$ GenPPL at $5.46$ unigram entropy in 64 sampling steps, close to real validation data. Even without Self-Conditioning (SC), SDMs outperform masked and uniform diffusion (with SC or…
PDF arXiv
arXiv cs.LGSep 28
One Proposal for Every Margin: Zero-Shot Amortized Sequential Importance Sampling for Binary Matrices
Ruishuo Chen, Weijia Li, Xun Wang, Yu Chen, Leheng Cai +1
In ecology, psychometrics, and the analysis of social and financial networks, binary matrices are often analyzed conditional on their observed row and column sums, which restricts the problem to a finite sample space of matrices with the same margins. Two fundamental problems are to count this space and to sample uniformly from it. Sequential importance sampling (SIS) addresses both with independent weighted samples and an unbiased count estimator, but its efficiency depends critically on the proposal distribution. Existing proposals are analytically designed, and their accuracy can vary substantially with the margins. We show that the ideal SIS proposal, under which every weight equals the count and the variance vanishes, is exactly the policy of a generative flow network (GFlowNet) with unit reward on every matrix that has the given margins. We therefore propose MarginFlow, a framework that turns the design of the proposal into a learning problem and amortizes it across margins by exploiting their self-similarity. Every partial matrix is itself an instance with reduced margins, so one set transformer that reads the remaining margins serves every margin. We train MarginFlow on a…
PDF arXiv
arXiv cs.LGSep 28
Physics-Guided Conditional Diffusion Model for Rare Event Synthesis and Diagnosis for the Water-Gas Shift Reaction
Md Abrar Rafid Siddique, Bibek Aryal, Qiugang Lu
As the world moves towards sustainable energy sources, hydrogen (H2) can be treated as an eco-friendly alternative to fossil fuels due to its high energy density and zero carbon emissions. The water-gas shift (WGS) reaction is a widely used industrial process for hydrogen production by converting carbon monoxide and steam into hydrogen and carbon dioxide. However, occurrences like severe fouling, catalyst deterioration, and thermal runaway can hamper the reaction kinetics/process safety and decrease the yield of H2. These incidents are rare, and gathering process data under such abnormal conditions is challenging. In this work, we propose a physics-guided conditional diffusion model to generate realistic rare-event trajectories for the WGS reaction. The proposed model integrates a conditional denoising diffusion probabilistic model (CDDPM) with governing laws of the reaction to generate physically consistent process trajectories. The conditioning features allow the model to produce high-quality synthetic profiles for rare-event domains that are typically beyond the training regimes. The generated rare-event trajectories then augment the raw dataset for a balanced distribution…
PDF arXiv
arXiv cs.LGSep 28
Universal Approximation of Measure-to-Measure Operators by Pushforwards
Takashi Furuya, Nicholas H. Nelsen, Frank Cole
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite $p$-th moment…
PDF arXiv
arXiv cs.LGSep 28
Manifold-Stable Flow Matching
Amirhossein Nazerian, Ali Pezeshki, Jianguo Zhao
Flow matching (FM) learns generative dynamics through velocity regression. Geometric FM variants commonly assume a prior supported on the data manifold, requiring geometric knowledge that is often unavailable. Without such knowledge, low regression error alone does not guarantee manifold adherence. Adherence keeps generated samples within valid configurations and is empirically associated with better task performance. We introduce manifold-stable flow matching (MSFM), which can start from an arbitrary ambient prior, not necessarily supported on the manifold. Using tools from nonlinear dynamics, namely contraction theory, MSFM combines learned tangential transport with prescribed normal contraction. The construction uses analytical projectors for known manifolds and local affine proxies estimated by principal component analysis for unknown data geometry. By implementing contraction theory in both cases of known and unknown manifolds, we guarantee manifold invariance and transverse convergence to the manifold within a desired time window (e.g., one second). We derive a family of compatible probability paths and decompose the training loss into a learnable tangential term and a…
PDF arXiv

Papers

Topics in this field

flow matching · flow · generative modeling299 papers33 this week
physics informed neural · neural operators · informed neural networks295 papers27 this week
diffusion priors · posterior sampling · diffusion posterior74 papers15 this week

Active contributors · 90 days

Anima AnandkumarNeural Operator Splitting for Fluid Dynamics in Complex Geometry5
Stefan T. RadevConditional Mean Flows: Few-Step Densities for Simulation-Based Inference5
Taiji SuzukiFinite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows5
Jing XuSame Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation4
Jun ZhangEqual Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models4
Alexander KorotinDiscrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs4

Recent papers

All papers in Research

arXiv cs.LGSep 28
FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models
Yinuo Ren, Haoxuan Chen, Grant M. Rotskoff, Jiequn Han, Lexing Ying
Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the pretrained distribution. Feynman-Kac sequential Monte Carlo (SMC) makes this correction exact in principle, but its prescribed weights routinely degenerate when the proposal dynamics are misaligned with the tilt, capping the practical gains from additional particles. We introduce FluxLite, a lightweight, training-free proposal-control framework for discrete diffusion. On the sparse directed graph of pretrained reverse rates, any sparse jump-rate perturbation can be exactly compensated by a $q_t$-weighted graph-divergence term in the Feynman-Kac potential; the target path is therefore preserved while the residual reweighting variance becomes a local convex objective. We instantiate this principle as two practical samplers: a one-hop local reallocation rule (HEU) and a small nonnegative quadratic program over pretrained-rate bases (D-VCG). We further prove population stability under the standard score-entropy training loss, identifying a tilted-path coverage factor that governs robustness to score error, together with…
PDF arXiv
arXiv cs.LGSep 28
Simplex Diffusion Models
Justin Deschenaux, Alexandre Galashov, Andrew Campbell, Li Kevin Wenliang, James Thornton +2
Diffusion models have revolutionized generative modeling for continuous data through the gradual refinement of a belief state. This iterative refinement has not yet carried over to discrete diffusion models, which discard uncertainty at intermediate steps through categorical sampling (information collapse). We propose Simplex Diffusion Models (SDMs), a framework that lifts the diffusion process to the probability simplex to represent beliefs over categories. SDMs admit probability paths with closed-form reverse transitions and can be trained with a simple cross-entropy loss. Contrary to earlier proposals such as Dirichlet Flow Matching which requires integrating an ordinary differential equation, we introduce a DDIM-like sampler with a tunable level of stochasticity. Because SDMs operate on samples on the simplex, they can carry uncertainty across denoising steps, which mitigates information collapse. On OpenWebText, SDMs are competitive with strong Discrete Diffusion baselines, achieving $17.0$ GenPPL at $5.46$ unigram entropy in 64 sampling steps, close to real validation data. Even without Self-Conditioning (SC), SDMs outperform masked and uniform diffusion (with SC or…
PDF arXiv
arXiv cs.LGSep 28
One Proposal for Every Margin: Zero-Shot Amortized Sequential Importance Sampling for Binary Matrices
Ruishuo Chen, Weijia Li, Xun Wang, Yu Chen, Leheng Cai +1
In ecology, psychometrics, and the analysis of social and financial networks, binary matrices are often analyzed conditional on their observed row and column sums, which restricts the problem to a finite sample space of matrices with the same margins. Two fundamental problems are to count this space and to sample uniformly from it. Sequential importance sampling (SIS) addresses both with independent weighted samples and an unbiased count estimator, but its efficiency depends critically on the proposal distribution. Existing proposals are analytically designed, and their accuracy can vary substantially with the margins. We show that the ideal SIS proposal, under which every weight equals the count and the variance vanishes, is exactly the policy of a generative flow network (GFlowNet) with unit reward on every matrix that has the given margins. We therefore propose MarginFlow, a framework that turns the design of the proposal into a learning problem and amortizes it across margins by exploiting their self-similarity. Every partial matrix is itself an instance with reduced margins, so one set transformer that reads the remaining margins serves every margin. We train MarginFlow on a…
PDF arXiv
arXiv cs.LGSep 28
Physics-Guided Conditional Diffusion Model for Rare Event Synthesis and Diagnosis for the Water-Gas Shift Reaction
Md Abrar Rafid Siddique, Bibek Aryal, Qiugang Lu
As the world moves towards sustainable energy sources, hydrogen (H2) can be treated as an eco-friendly alternative to fossil fuels due to its high energy density and zero carbon emissions. The water-gas shift (WGS) reaction is a widely used industrial process for hydrogen production by converting carbon monoxide and steam into hydrogen and carbon dioxide. However, occurrences like severe fouling, catalyst deterioration, and thermal runaway can hamper the reaction kinetics/process safety and decrease the yield of H2. These incidents are rare, and gathering process data under such abnormal conditions is challenging. In this work, we propose a physics-guided conditional diffusion model to generate realistic rare-event trajectories for the WGS reaction. The proposed model integrates a conditional denoising diffusion probabilistic model (CDDPM) with governing laws of the reaction to generate physically consistent process trajectories. The conditioning features allow the model to produce high-quality synthetic profiles for rare-event domains that are typically beyond the training regimes. The generated rare-event trajectories then augment the raw dataset for a balanced distribution…
PDF arXiv
arXiv cs.LGSep 28
Universal Approximation of Measure-to-Measure Operators by Pushforwards
Takashi Furuya, Nicholas H. Nelsen, Frank Cole
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite $p$-th moment…
PDF arXiv
arXiv cs.LGSep 28
Manifold-Stable Flow Matching
Amirhossein Nazerian, Ali Pezeshki, Jianguo Zhao
Flow matching (FM) learns generative dynamics through velocity regression. Geometric FM variants commonly assume a prior supported on the data manifold, requiring geometric knowledge that is often unavailable. Without such knowledge, low regression error alone does not guarantee manifold adherence. Adherence keeps generated samples within valid configurations and is empirically associated with better task performance. We introduce manifold-stable flow matching (MSFM), which can start from an arbitrary ambient prior, not necessarily supported on the manifold. Using tools from nonlinear dynamics, namely contraction theory, MSFM combines learned tangential transport with prescribed normal contraction. The construction uses analytical projectors for known manifolds and local affine proxies estimated by principal component analysis for unknown data geometry. By implementing contraction theory in both cases of known and unknown manifolds, we guarantee manifold invariance and transverse convergence to the manifold within a desired time window (e.g., one second). We derive a family of compatible probability paths and decompose the training loss into a learnable tangential term and a…
PDF arXiv

Activity · 12 weeks

Open problems in Flow matching

Stable and scalable source-target pairing for flow matching1 paper raises it · 1 addresses it
Unstable or random pairing degrades training and integration, limiting model performance at scale.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Efficient few-step generation without fixed discretization1 paper raises it · 1 addresses it
Fixed timestep discretization in flow matching is heuristic, expensive to tune, and restrictive for varying sample difficulty.
- Few-Step Generation via Data-Space Iterationraises · addresses
Data-free and simulation-free distillation for flow/diffusion models1 paper raises it · 1 addresses it
Existing distillation methods rely on curated datasets, costly teacher rollouts, or auxiliary networks, complicating training and scaling.
- Consistent Distribution Matching for Data-Free Diffusion Distillationraises · addresses
Accelerating numerical integration for neuro-spectral PDE solvers1 paper raises it · 1 addresses it
Sequential time-stepping in neuro-spectral architectures limits temporal parallelism and slows training/inference.
- PaPi: Parallel Picard-Duhamel Integration for Fast Neuro-Spectral Architecturesraises · addresses
Dynamics-aligned latent spaces for PDE representation learning1 paper raises it · 1 addresses it
Reconstruction-based objectives fail to capture evolution geometry, limiting accurate field prediction.
- PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEsraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 43m ago

Flow matching

Found by the topic model: flow matching, physics informed neural, diffusion priors.

75papers this week−51.9% vs last week

All papers · 668 With code · 3

Newest Most relevant

630 papers this month