Papers

Field

Top tools here

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

neural operators · physics informed neural · informed neural networks375 papers34 this week
graph neural · graph learning · graph neural networks312 papers30 this week
relu networks · gradient descent · deep neural161 papers24 this week
deep neural · neural networks · neural network163 papers22 this week
flow matching · normalizing flows · flow205 papers18 this week
bayesian neural · bayesian networks · amortized bayesian106 papers16 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGOct 28
Learning a Prior-Data Fitted Network for Graph Foundation Models via Structural Causal Model Priors
Ziwei Zhang, Tianyin Liao, Chunyu Hu, Ge Lan, Nuo Xu +5
Graph Foundation Models (GFMs) have recently emerged as a promising paradigm for learning transferable representations across different graphs and domains. However, existing GFMs face notable limitations, including limited pre-training scale or reliance on aligning with textual information in Large Language Models (LLMs). To overcome these limitations, recent studies have drawn inspiration from prior-data fitted networks (PFNs), which learn from large synthetic datasets and support multi-domain prediction backed by causal learning. However, existing graph PFNs encounter severe difficulties in modeling causal relationships within and across node and edge sets, generating diverse graph structures, and designing suitable architectures that can learn from complex synthetic datasets. To address these challenges, we propose **GSCM-PFN** (Graph SCM-based Prior-Data Fitted Network), a graph PFN pre-trained on synthetic data generated from an SCM-based prior that jointly models nodes and graph topology. Specifically, we (1) design a joint feature-edge-label structural causal model (SCM) to guide synthetic graph generation and capture causal relationships within and across node and edge…
PDF arXiv
arXiv cs.CCOct 28
A Witnessing Duality for ReLU Network Depth
Dimitrios Myrisiotis
A depth lower bound for ReLU networks asserts that every shallower network errs on some input; we ask when a weak theory can prove such a bound so that an error can be computed. For the exact computation of $\max(x_1,\ldots,x_n)$, recent work shows two hidden layers suffice with unrestricted rational weights. By contrast, Haase, Hertrich and Loho (ICLR 2023) proved that integer weights require depth logarithmic in $n$, and Averkov, Hojny and Merkert (ICLR 2025) extended the bound to nonbias weights whose denominators are powers of a fixed integer not divisible by a prime $q$. With integer weights the bound is sharp: the classical balanced construction has depth $\lceil\log_2 n\rceil$. For each fixed $q$ and depth cap $K$, we prove this known bound in $\mathbf{VTC}^0$, the bounded-arithmetic theory of uniform constant-depth threshold computation, together with a refuter that is itself such a computation and whose correctness the theory proves. Given a network with at most $K$ hidden layers, nonbias weights whose reduced denominators avoid $q$, arbitrary rational biases and $n>q^K$ inputs, the refuter returns a rational input of polynomial length on which the network errs. The…
PDF arXiv
arXiv cs.LGOct 28
NAGO: Noise-Aware Generative Operator via Flow Matching
Guanyu Chen, Shengze Xu, Pengwei Liu, Xingyu Ren, Pengkai Wang +3
We introduce NAGO, a noise-aware generative operator for efficient sampling from invariant measures of stochastic partial differential equations (SPDEs) in function space. Our central idea is to treat the source measure as a design variable rather than a fixed Gaussian default. We establish admissibility conditions for well-defined infinite-dimensional flows and show that the finite-band source--target covariance ratio controls the spectral stiffness of the marginal velocity field. This analysis yields an admissibility-constrained spectral-matching rule: choose a source that is sufficiently rough for well-posedness, then minimize covariance mismatch over the resolved Fourier modes. On three representative stochastic-field benchmarks, this source design improves few-step spectral accuracy and several complementary distributional and physical statistics. In a controlled Matern-field ablation, it reduces the accurate RK4 budget from about 30 steps for Gaussian functional flow matching to 3--5 steps, a $6$--$10\times$ reduction in neural ODE evaluations. These gains are strongest for stationary multiscale fields and depend on the metric, resolution, and guidance protocol.
PDF arXiv
arXiv cs.LGOct 28
A Hybrid Physics-informed holomorphic neural networks Method
Zhijie Su, Qiaolin He, Zhiping Mao, Shiquan Zhang
Physics-informed holomorphic neural networks (PIHNNs) exploit complex-analytic representations to satisfy homogeneous PDEs by construction, but their extension to nonhomogeneous problems is nontrivial. We propose a hybrid physics-informed holomorphic neural network (HPIHNN) framework based on a decomposition of the solution into particular and homogeneous components. The particular component is computed by a spectral method on a simple extended domain, while the homogeneous component is approximated by a PIHNN trained only on the boundary with modified boundary data. The resulting formulation avoids meshing complex geometries and retains the dimensionality-reduction advantage of holomorphic representations. The method is applied to nonhomogeneous Poisson, Stokes, biharmonic, linear elasticity, and Helmholtz equations. Numerical experiments covering complex geometries, corner singularities, and nearly incompressible regimes demonstrate that the proposed method accurately resolves the corresponding solutions and, in the considered tests, generally achieves lower errors and shorter training times than standard PINNs. The results indicate that coupling spectral particular solutions…
PDF arXiv
arXiv cs.LGOct 28
HotPOT: Operator Transformer with Heterogeneous Physics Experts for Large-Scale PDE Pre-training
Jiayi Li, Guancheng Zhou, Yisi Luo, Ruihua Chen, Penghao Jiang +3
Large-scale PDE pre-training trains a single model on the data from multiple partial differential equations (PDEs). However, adding heterogeneous PDEs with different governing equations, physical variables, and variable interactions does not always improve predictive performance across tasks. Existing PDE mixture-of-experts models mitigate cross-task interference through learned routing, but typically use homogeneous expert architectures, making it difficult to directly reflect differences in state evolution across equations. We introduce HotPOT, an operator transformer with heterogeneous physics experts for large-scale PDE pre-training. HotPOT uses a shared encoder to capture information common across PDEs, while each PDE family uses a neural-numerical expert that pairs a learnable predictor with a fixed numerical update containing no trainable parameters. Each predictor outputs the quantities required by its paired numerical update, which then computes the next state. This design allows different PDEs to share a common encoder while retaining their own learnable predictors and numerical updates. Experiments across five PDE families and six settings show that HotPOT outperforms…
PDF arXiv
arXiv cs.LGOct 28
Averaged Annealed Markov Chains Achieve the Algorithmic Threshold for Tensor PCA, Part II: Langevin Dynamics
Aniket Das, Tselil Schramm
We consider the tensor principal components analysis (tensor PCA) problem on the sphere, in which we are asked to recover an unknown vector in $\mathcal{S}_N=\{x\in\mathbb{R}^N:\lVert x\rVert^2=N\}$ from a noisy observation of its $k$-th tensor power, for a fixed $k\ge3$. Tensor PCA exhibits a well-studied information-computation gap: global algorithms such as spectral methods and sum-of-squares work above an algorithmic threshold, below which we know matching lower bounds (against these and some other restricted models of computation). In contrast, local algorithms such as Markov chains were only known to succeed from a random initialization at a polynomially larger signal-to-noise ratio, and were believed to be inferior to global algorithms for tensor PCA. Continuing the program initiated in Part I [arXiv:2610.16705], we show that spherical Langevin dynamics, when supplemented with annealing and ergodic averaging, achieves polynomial-time strong recovery above the algorithmic threshold from a uniformly random initialization. Together with Part I, our results demonstrate that local Markov chains can succeed all the way down to the algorithmic threshold from an uninformative cold…
PDF arXiv

Papers

Topics in this field

neural operators · physics informed neural · informed neural networks375 papers34 this week
graph neural · graph learning · graph neural networks312 papers30 this week
relu networks · gradient descent · deep neural161 papers24 this week
deep neural · neural networks · neural network163 papers22 this week
flow matching · normalizing flows · flow205 papers18 this week
bayesian neural · bayesian networks · amortized bayesian106 papers16 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.LGOct 28
Learning a Prior-Data Fitted Network for Graph Foundation Models via Structural Causal Model Priors
Ziwei Zhang, Tianyin Liao, Chunyu Hu, Ge Lan, Nuo Xu +5
Graph Foundation Models (GFMs) have recently emerged as a promising paradigm for learning transferable representations across different graphs and domains. However, existing GFMs face notable limitations, including limited pre-training scale or reliance on aligning with textual information in Large Language Models (LLMs). To overcome these limitations, recent studies have drawn inspiration from prior-data fitted networks (PFNs), which learn from large synthetic datasets and support multi-domain prediction backed by causal learning. However, existing graph PFNs encounter severe difficulties in modeling causal relationships within and across node and edge sets, generating diverse graph structures, and designing suitable architectures that can learn from complex synthetic datasets. To address these challenges, we propose **GSCM-PFN** (Graph SCM-based Prior-Data Fitted Network), a graph PFN pre-trained on synthetic data generated from an SCM-based prior that jointly models nodes and graph topology. Specifically, we (1) design a joint feature-edge-label structural causal model (SCM) to guide synthetic graph generation and capture causal relationships within and across node and edge…
PDF arXiv
arXiv cs.CCOct 28
A Witnessing Duality for ReLU Network Depth
Dimitrios Myrisiotis
A depth lower bound for ReLU networks asserts that every shallower network errs on some input; we ask when a weak theory can prove such a bound so that an error can be computed. For the exact computation of $\max(x_1,\ldots,x_n)$, recent work shows two hidden layers suffice with unrestricted rational weights. By contrast, Haase, Hertrich and Loho (ICLR 2023) proved that integer weights require depth logarithmic in $n$, and Averkov, Hojny and Merkert (ICLR 2025) extended the bound to nonbias weights whose denominators are powers of a fixed integer not divisible by a prime $q$. With integer weights the bound is sharp: the classical balanced construction has depth $\lceil\log_2 n\rceil$. For each fixed $q$ and depth cap $K$, we prove this known bound in $\mathbf{VTC}^0$, the bounded-arithmetic theory of uniform constant-depth threshold computation, together with a refuter that is itself such a computation and whose correctness the theory proves. Given a network with at most $K$ hidden layers, nonbias weights whose reduced denominators avoid $q$, arbitrary rational biases and $n>q^K$ inputs, the refuter returns a rational input of polynomial length on which the network errs. The…
PDF arXiv
arXiv cs.LGOct 28
NAGO: Noise-Aware Generative Operator via Flow Matching
Guanyu Chen, Shengze Xu, Pengwei Liu, Xingyu Ren, Pengkai Wang +3
We introduce NAGO, a noise-aware generative operator for efficient sampling from invariant measures of stochastic partial differential equations (SPDEs) in function space. Our central idea is to treat the source measure as a design variable rather than a fixed Gaussian default. We establish admissibility conditions for well-defined infinite-dimensional flows and show that the finite-band source--target covariance ratio controls the spectral stiffness of the marginal velocity field. This analysis yields an admissibility-constrained spectral-matching rule: choose a source that is sufficiently rough for well-posedness, then minimize covariance mismatch over the resolved Fourier modes. On three representative stochastic-field benchmarks, this source design improves few-step spectral accuracy and several complementary distributional and physical statistics. In a controlled Matern-field ablation, it reduces the accurate RK4 budget from about 30 steps for Gaussian functional flow matching to 3--5 steps, a $6$--$10\times$ reduction in neural ODE evaluations. These gains are strongest for stationary multiscale fields and depend on the metric, resolution, and guidance protocol.
PDF arXiv
arXiv cs.LGOct 28
A Hybrid Physics-informed holomorphic neural networks Method
Zhijie Su, Qiaolin He, Zhiping Mao, Shiquan Zhang
Physics-informed holomorphic neural networks (PIHNNs) exploit complex-analytic representations to satisfy homogeneous PDEs by construction, but their extension to nonhomogeneous problems is nontrivial. We propose a hybrid physics-informed holomorphic neural network (HPIHNN) framework based on a decomposition of the solution into particular and homogeneous components. The particular component is computed by a spectral method on a simple extended domain, while the homogeneous component is approximated by a PIHNN trained only on the boundary with modified boundary data. The resulting formulation avoids meshing complex geometries and retains the dimensionality-reduction advantage of holomorphic representations. The method is applied to nonhomogeneous Poisson, Stokes, biharmonic, linear elasticity, and Helmholtz equations. Numerical experiments covering complex geometries, corner singularities, and nearly incompressible regimes demonstrate that the proposed method accurately resolves the corresponding solutions and, in the considered tests, generally achieves lower errors and shorter training times than standard PINNs. The results indicate that coupling spectral particular solutions…
PDF arXiv
arXiv cs.LGOct 28
HotPOT: Operator Transformer with Heterogeneous Physics Experts for Large-Scale PDE Pre-training
Jiayi Li, Guancheng Zhou, Yisi Luo, Ruihua Chen, Penghao Jiang +3
Large-scale PDE pre-training trains a single model on the data from multiple partial differential equations (PDEs). However, adding heterogeneous PDEs with different governing equations, physical variables, and variable interactions does not always improve predictive performance across tasks. Existing PDE mixture-of-experts models mitigate cross-task interference through learned routing, but typically use homogeneous expert architectures, making it difficult to directly reflect differences in state evolution across equations. We introduce HotPOT, an operator transformer with heterogeneous physics experts for large-scale PDE pre-training. HotPOT uses a shared encoder to capture information common across PDEs, while each PDE family uses a neural-numerical expert that pairs a learnable predictor with a fixed numerical update containing no trainable parameters. Each predictor outputs the quantities required by its paired numerical update, which then computes the next state. This design allows different PDEs to share a common encoder while retaining their own learnable predictors and numerical updates. Experiments across five PDE families and six settings show that HotPOT outperforms…
PDF arXiv
arXiv cs.LGOct 28
Averaged Annealed Markov Chains Achieve the Algorithmic Threshold for Tensor PCA, Part II: Langevin Dynamics
Aniket Das, Tselil Schramm
We consider the tensor principal components analysis (tensor PCA) problem on the sphere, in which we are asked to recover an unknown vector in $\mathcal{S}_N=\{x\in\mathbb{R}^N:\lVert x\rVert^2=N\}$ from a noisy observation of its $k$-th tensor power, for a fixed $k\ge3$. Tensor PCA exhibits a well-studied information-computation gap: global algorithms such as spectral methods and sum-of-squares work above an algorithmic threshold, below which we know matching lower bounds (against these and some other restricted models of computation). In contrast, local algorithms such as Markov chains were only known to succeed from a random initialization at a polynomially larger signal-to-noise ratio, and were believed to be inferior to global algorithms for tensor PCA. Continuing the program initiated in Part I [arXiv:2610.16705], we show that spherical Langevin dynamics, when supplemented with annealing and ergodic averaging, achieves polynomial-time strong recovery above the algorithmic threshold from a uniformly random initialization. Together with Part I, our results demonstrate that local Markov chains can succeed all the way down to the algorithmic threshold from an uninformative cold…
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 40m ago

Neural operators

Found by the topic model: neural operators, graph neural, flow matching.

184papers this week−21.7% vs last week

All papers · 1.8k With code · 11

Newest Most relevant

1.1k papers this month

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.