Papers

Field

Top tools here

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

graph neural · graph learning · graph neural networks310 papers28 this week
neural operators · physics informed neural · informed neural networks368 papers27 this week
relu networks · gradient descent · deep neural159 papers22 this week
deep neural · neural networks · neural network160 papers19 this week
bayesian neural · bayesian networks · amortized bayesian106 papers16 this week
flow matching · normalizing flows · flow202 papers15 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGOct 29
From Local Recovery to Unified Generation: Discovering Triple Set in Knowledge Graphs via Observation-Anchored Latent Diffusion
Jiaqi Wang, Yisen Gao, Huayang Shi, Junhao Hou, Wengen Li +5
Triple set prediction (TSP) is to recover missing relational triples in a knowledge graph by predicting the triple set instead of isolated links. Existing approaches predominantly rely on the subgraph-based completion paradigm, where the original graph is first partitioned into local subgraphs which are then independently completed. Such a paradigm suffers from fragmented structural reasoning, limited cross subgraph dependency modeling, and missing predictions across subgraphs. This paper proposes LatentDiff, a novel unified latent generative framework for TSP, which reformulates knowledge graph completion as latent space graph generation. LatentDiff first learns a compact latent representation using a relation-aware residual-quantized knowledge graph autoencoder, which encodes the whole graph into relation-aware latent embeddings, quantizes them with chunked residual codebooks, and reconstructs triples through a factorized decoder. On top of this latent space, we introduce an observation-anchored latent graph diffusion model, where the target latent graph representation is perturbed toward the observed latent. The reverse process recovers missing triples through progressive…
PDF arXiv
arXiv stat.MLOct 29
The Geometry of Multitask Learning: Sharp Rates for Shared Subspace Estimation
Tony Cai, Abhinav Chakraborty, Samory Kpotufe
Learning a shared subspace in multitask learning is an important statistical problem, yet exact rates remain poorly understood in many of the formal settings considered in the literature. To address this gap, which arises from limitations of existing upper-bound techniques, we propose a general multitask lemma. The lemma provides sufficient conditions for fast subspace estimation in generic multitask settings, in the form of geometric conditions on the task regression functions and their estimators. We subsequently show that the multitask lemma yields sharp minimax rates---previously unknown---for linear regression and classification in the regimes considered, and rates matching up to logarithmic factors for additive-noise regression.
PDF arXiv
arXiv stat.MLOct 28
Modular Bayesian Foundation Models: Bayesian time series encoders for OOD detection, routing and forecasting
Mame Diarra Toure, Quang Hung Pham, Mohamed Ryad Zemouri, David Stephens, Luc Vouligny +1
Industrial systems continuously generate time series that vary in sampling frequency, scale, dynamics, and operating regime. Consequently a single time-series foundation model may not be able to represent all regimes effectively and retraining a large model for every new dataset or regime shift is computationally expensive. Modular foundation models address this issue by reusing specialised encoders trained on diverse datasets and by allowing the expansion of capabilities through the addition of new experts. This approach requires compact experts to minimize storage cost and a proper selection mechanism of experts for unseen data. We propose a modular foundation model framework that combines end-to-end Bayesian encoder training with uncertainty based routing. Using Singular Bayesian Neural Networks (SBNNs), we build a collection of compact Bayesian encoders whose epistemic uncertainty guides label-free selection. At the selected ranks, our Bayesian collection uses 63.8\% less storage than its deterministic counterpart. We evaluate this framework on time-series data through distribution shift, encoder selection, and probabilistic forecasting. Across seven incoming datasets,…
PDF arXiv
arXiv cs.LGOct 28
Exponential-Nemytskii Neural Operators: Learning Field Evolution Across Step Sizes
Jingye Li, Jinglai Li
In variable-step operator learning, agreement at observed step sizes does not by itself determine predictions at other temporal resolutions. We propose an operator whose step-size dependence is prescribed by a first-order Duhamel approximation of semilinear dynamics. It combines exact linear propagation with a local nonlinear function of the field and its derivatives, allowing evaluation below the observation interval without supervision densely sampled across step sizes. Under suitable conditions and at fixed spatial resolution, we prove that agreement between models at two step sizes controls their prediction differences throughout a prescribed range. We also give conditions for local identifiability of both components from two step sizes, supporting interpretation of the learned structure. A latent extension applies the same construction to learned fields. We evaluate the base model on two synthetic problems and the extension on two real datasets against several baselines.
PDF arXiv
arXiv cs.LGOct 28
Representation Complexity Along Learning Trajectories: Trading Width for Precision and Time
Ryan Singh, Miguel De Llanza, Christopher L. Buckley
What computational role does overparameterisation play during learning? Information-plane analyses suggest that neural networks can traverse representations substantially more complex than those required by their final predictors. We explore when this transient complexity serves a computational purpose by comparing the representation complexity needed for prediction with that needed to make an unknown target statistically visible through gradient feedback. We introduce an oracle model of neural network learning and use Kolmogorov \(n\)-widths to quantify effective representation complexity. Our main technical result transfers complexity bounds from the forward representation to the corresponding gradient-query features. Combined with statistical-query lower bounds, this gives a task-agnostic lower bound on the initial effective width required for target-dependent gradient information to become detectable. For \(k\)-sparse parity in \(D\) dimensions, a known target admits an \(O(k)\)-width representation, while the general bound already implies a polynomial separation between representation and discovery. We then exploit parity's invariance under coordinatewise sign reflections to…
PDF arXiv
arXiv cs.LGOct 28
Rethinking GFlowNet Training: Negative Reinforce Dynamics
Pedro Dall'Antonia, Daniel Csillag, Diego Mesquita
Generative Flow Networks (GFlowNets) are models that learn to sample discrete objects proportional to a target reward. They are trained by minimizing balance errors over sampled trajectories, which inherently ties correctness to exploration: if a region is not visited, balance cannot be enforced there. Adaptive exploration is therefore crucial, yet standard training relies on a semi-gradient update that treats the sampling distribution as fixed at each optimization step. On the other hand, the full gradient reveals a second mechanism that reduces the observed errors in an anti-exploratory manner. In this paper, we establish a connection between the full gradient and min-min dynamics between the GFlowNet and a decoupled sampler, showing that the sampler reduces observed violations by avoiding high-error regions. This insight motivates a bridge between two-player games and single-policy dynamics, which underpins the design of our proposed method, Negative Reinforce Dynamics (NRD). By tying the parameters of an adversarial game together, we recover a single-policy dynamic that extends exploration while maintaining correctness. In standard benchmarks, our method outperforms existing…
PDF arXiv

Papers

Topics in this field

graph neural · graph learning · graph neural networks310 papers28 this week
neural operators · physics informed neural · informed neural networks368 papers27 this week
relu networks · gradient descent · deep neural159 papers22 this week
deep neural · neural networks · neural network160 papers19 this week
bayesian neural · bayesian networks · amortized bayesian106 papers16 this week
flow matching · normalizing flows · flow202 papers15 this week

Active contributors · 90 days

Recent papers

All papers in Research

arXiv cs.LGOct 29
From Local Recovery to Unified Generation: Discovering Triple Set in Knowledge Graphs via Observation-Anchored Latent Diffusion
Jiaqi Wang, Yisen Gao, Huayang Shi, Junhao Hou, Wengen Li +5
Triple set prediction (TSP) is to recover missing relational triples in a knowledge graph by predicting the triple set instead of isolated links. Existing approaches predominantly rely on the subgraph-based completion paradigm, where the original graph is first partitioned into local subgraphs which are then independently completed. Such a paradigm suffers from fragmented structural reasoning, limited cross subgraph dependency modeling, and missing predictions across subgraphs. This paper proposes LatentDiff, a novel unified latent generative framework for TSP, which reformulates knowledge graph completion as latent space graph generation. LatentDiff first learns a compact latent representation using a relation-aware residual-quantized knowledge graph autoencoder, which encodes the whole graph into relation-aware latent embeddings, quantizes them with chunked residual codebooks, and reconstructs triples through a factorized decoder. On top of this latent space, we introduce an observation-anchored latent graph diffusion model, where the target latent graph representation is perturbed toward the observed latent. The reverse process recovers missing triples through progressive…
PDF arXiv
arXiv stat.MLOct 29
The Geometry of Multitask Learning: Sharp Rates for Shared Subspace Estimation
Tony Cai, Abhinav Chakraborty, Samory Kpotufe
Learning a shared subspace in multitask learning is an important statistical problem, yet exact rates remain poorly understood in many of the formal settings considered in the literature. To address this gap, which arises from limitations of existing upper-bound techniques, we propose a general multitask lemma. The lemma provides sufficient conditions for fast subspace estimation in generic multitask settings, in the form of geometric conditions on the task regression functions and their estimators. We subsequently show that the multitask lemma yields sharp minimax rates---previously unknown---for linear regression and classification in the regimes considered, and rates matching up to logarithmic factors for additive-noise regression.
PDF arXiv
arXiv stat.MLOct 28
Modular Bayesian Foundation Models: Bayesian time series encoders for OOD detection, routing and forecasting
Mame Diarra Toure, Quang Hung Pham, Mohamed Ryad Zemouri, David Stephens, Luc Vouligny +1
Industrial systems continuously generate time series that vary in sampling frequency, scale, dynamics, and operating regime. Consequently a single time-series foundation model may not be able to represent all regimes effectively and retraining a large model for every new dataset or regime shift is computationally expensive. Modular foundation models address this issue by reusing specialised encoders trained on diverse datasets and by allowing the expansion of capabilities through the addition of new experts. This approach requires compact experts to minimize storage cost and a proper selection mechanism of experts for unseen data. We propose a modular foundation model framework that combines end-to-end Bayesian encoder training with uncertainty based routing. Using Singular Bayesian Neural Networks (SBNNs), we build a collection of compact Bayesian encoders whose epistemic uncertainty guides label-free selection. At the selected ranks, our Bayesian collection uses 63.8\% less storage than its deterministic counterpart. We evaluate this framework on time-series data through distribution shift, encoder selection, and probabilistic forecasting. Across seven incoming datasets,…
PDF arXiv
arXiv cs.LGOct 28
Exponential-Nemytskii Neural Operators: Learning Field Evolution Across Step Sizes
Jingye Li, Jinglai Li
In variable-step operator learning, agreement at observed step sizes does not by itself determine predictions at other temporal resolutions. We propose an operator whose step-size dependence is prescribed by a first-order Duhamel approximation of semilinear dynamics. It combines exact linear propagation with a local nonlinear function of the field and its derivatives, allowing evaluation below the observation interval without supervision densely sampled across step sizes. Under suitable conditions and at fixed spatial resolution, we prove that agreement between models at two step sizes controls their prediction differences throughout a prescribed range. We also give conditions for local identifiability of both components from two step sizes, supporting interpretation of the learned structure. A latent extension applies the same construction to learned fields. We evaluate the base model on two synthetic problems and the extension on two real datasets against several baselines.
PDF arXiv
arXiv cs.LGOct 28
Representation Complexity Along Learning Trajectories: Trading Width for Precision and Time
Ryan Singh, Miguel De Llanza, Christopher L. Buckley
What computational role does overparameterisation play during learning? Information-plane analyses suggest that neural networks can traverse representations substantially more complex than those required by their final predictors. We explore when this transient complexity serves a computational purpose by comparing the representation complexity needed for prediction with that needed to make an unknown target statistically visible through gradient feedback. We introduce an oracle model of neural network learning and use Kolmogorov \(n\)-widths to quantify effective representation complexity. Our main technical result transfers complexity bounds from the forward representation to the corresponding gradient-query features. Combined with statistical-query lower bounds, this gives a task-agnostic lower bound on the initial effective width required for target-dependent gradient information to become detectable. For \(k\)-sparse parity in \(D\) dimensions, a known target admits an \(O(k)\)-width representation, while the general bound already implies a polynomial separation between representation and discovery. We then exploit parity's invariance under coordinatewise sign reflections to…
PDF arXiv
arXiv cs.LGOct 28
Rethinking GFlowNet Training: Negative Reinforce Dynamics
Pedro Dall'Antonia, Daniel Csillag, Diego Mesquita
Generative Flow Networks (GFlowNets) are models that learn to sample discrete objects proportional to a target reward. They are trained by minimizing balance errors over sampled trajectories, which inherently ties correctness to exploration: if a region is not visited, balance cannot be enforced there. Adaptive exploration is therefore crucial, yet standard training relies on a semi-gradient update that treats the sampling distribution as fixed at each optimization step. On the other hand, the full gradient reveals a second mechanism that reduces the observed errors in an anti-exploratory manner. In this paper, we establish a connection between the full gradient and min-min dynamics between the GFlowNet and a decoupled sampler, showing that the sampler reduces observed violations by avoiding high-error regions. This insight motivates a bridge between two-player games and single-policy dynamics, which underpins the design of our proposed method, Negative Reinforce Dynamics (NRD). By tying the parameters of an adversarial game together, we recover a single-policy dynamic that extends exploration while maintaining correctness. In standard benchmarks, our method outperforms existing…
PDF arXiv

Activity · 12 weeks

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 2h ago

Neural operators

Found by the topic model: neural operators, graph neural, flow matching.

159papers this week−32.6% vs last week

All papers · 1.7k With code · 11

Newest Most relevant

1.1k papers this month