Papers

Field

Top tools here

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

neural operators · physics informed neural · informed neural networks375 papers31 this week
graph neural · graph learning · graph neural networks312 papers27 this week
deep neural · neural networks · neural network163 papers21 this week
relu networks · gradient descent · deep neural161 papers15 this week
flow matching · normalizing flows · flow205 papers14 this week
bayesian neural · bayesian networks · amortized bayesian106 papers14 this week

Topics in this field

Active contributors · 90 days

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGOct 26
CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning
Bo-Young Lim, Seokhwan Kim, Kisung Lee, Hyuk-Yoon Kwon
Continual graph learning (CGL) maintains models as graphs evolve without retraining from scratch, while mitigating catastrophic forgetting. Existing methods typically target either transductive or inductive settings and often store past subgraphs or require task identifiers at inference. We propose Community-aware Prototype Extraction for Continual Graph Learning (CoPE-CGL), a replay-free framework that uses a parameter-free structural prior, precomputed by multi-hop Laplacian smoothing, to anchor learning, memory, and inference. CoPE-CGL combines topology-infused bottleneck adaptation, community-level prototypes, and dynamic routing by majority vote over node-to-prototype matches. The default protocol assumes each test graph comes from one learned task; a node-level variant handles mixed-task graphs. Each task adds at most a constant amount of storage, with no past graph data retained. At equal storage and without task identifiers, CoPE-CGL matches the strongest replay baseline even when it receives true task identifiers: average performance (AP) is 95.17% and 95.72% versus 94.28% and 95.82% on Arxiv and CoraFull, respectively, with 5.9-14.0 times faster runtime and zero…
PDF arXiv
arXiv physics.flu-dynOct 26
Generative AI in Turbulence: Foundations, Applications, Validations and Benchmarks
Luca Biferale, Michele Buzzicotti
Generative AI offers new approaches to turbulence problems in which incomplete observations or unresolved d.o.f. admit multiple physically plausible solutions. Potential applications include stochastic closures, super-resolution, data assimilation, uncertainty quantification, and the generation of Eulerian and Lagrangian flow ensembles. However, turbulence is not a conventional image-generation problem. Its multiscale dynamics, intermittency, extreme events, rough velocity fields and intrinsically stochastic transport make visual realism, pointwise error, and low-order spectral agreement insufficient measures of physical fidelity. We examine where generative AI provide novel capabilities with emphasis on diffusion models, score-based methods, and related probability-transport formulations. We discuss their use in unconditional generation, probabilistic reconstruction, reduced-order modeling, temporal evolution, and mesh-free representations. We argue that validation must address turbulence properties: energy transfer, structure functions, anomalous scaling, pdf tails, coherent structures, conservation laws, temporal consistency, ensemble diversity and generalization. Finally, we…
PDF arXiv
arXiv cs.LGOct 26
Subspace Crystallization in Grokking: Rank Compression Tracks the Phase Transition
Shlok Mehendale, Paras Chopra
Grokking describes understanding something deeply and intuitively. In neural networks, it is the phenomenon where a network generalizes long after memorizing its training data. This delay has been linked to a fall in the effective rank of the network's weight matrices, but whether this rank compression drives grokking or follows it is still unresolved. We study a mean-field MLP and a one-layer transformer trained on modular arithmetic and observe that: (i) Compression follows the onset, not memorization. Every transformer memorizes its training data within 250 steps, while the grokking onset (the first step at which test accuracy reaches 95%) varies from 6,500 to 21,000 steps. The steepest fall in effective rank comes about 900 steps after the onset, however late the onset occurs, and networks that memorize without grokking compress far less. (ii) Subspace crystallization. We call the span of the fewest top singular directions of a weight matrix that preserve 99% of test accuracy its sufficient subspace. Its size can shrink as the margin between the correct and the best wrong output grows, so we also count the directions needed to keep the outputs within 10%. In the transformer,…
PDF arXiv
arXiv cs.LGOct 26
FlowGenesis: Representation-Driven Self-Organization of Neural Architectures
Maksim A Kazanskii
Deep neural networks typically rely on architectures specified before training, despite the continuous evolution of their internal representations. We introduce FlowGenesis, a framework in which neural architectures adapt through neuron birth and pruning guided by Representation Flux, a local statistic capturing changes in minibatch mean activations. Neurons exhibiting persistently high Flux undergo splitting designed to approximately preserve the network function, while neurons with low Flux and low activity are pruned. This mechanism allows network capacity to evolve without prescribing a target architecture or performing a separate architecture search. Experiments on image and text classi- fication show that networks initialized with different sizes approach similar final neuron-count ranges within each dataset while maintaining comparable predictive performance. We further extend FlowGenesis to continual learning, where representation-guided structural adaptation reduces the neurons and parameters allocated to successive tasks while maintaining accuracy comparable to fixed-width Progressive Neural Networks in a task-incremental learning setting. These findings demonstrate…
PDF arXiv
arXiv cs.AIOct 26
A Diagnostic Protocol for GNNs in Electricity Load Forecasting: Separating Parameter Sharing from Neighborhood Access
Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude, Itai Zehavi, Argyris Kalogeratos
A graph neural network (GNN) forecasting a collection of load series combines two mechanisms, sharing parameters across series and reading neighboring series, and a gain over a non-graph model may come from either; telling them apart decides whether a graph is worth maintaining. We propose a model diagnostic protocol that compares the GNN with three reference models: a per-node network, a network shared across nodes without neighbors, and the same GNN without edges. Degree-preserving rewiring tests whether neighbor identity matters. After a tuned benchmark on regional load in France and on low-voltage feeders in the UK, we apply the protocol to deliberately short node histories, as when non-stationary demand leaves only recent observations relevant: the feeders, within one week, and a public dataset of client consumption series. On the feeders, sharing parameters recovers most of the error gap between node-specific models and GNNs, about two thirds against a regularized one. Adding the edges to an otherwise identical backbone brings a smaller gain, half or more of which disappears under rewiring. On the client series, the neighbor mean degrades the per-node model on the shorter…
PDF arXiv
arXiv cs.LGOct 26
Mapping the Solution-Set Structure of Grokking on Latin Squares
Zihan Gu, Lulu Zhao, Tianyi Ma, Hua Zhang, Wenbo Su +2
Grokking is a delayed transition from memorisation to generalisation. A network first fits observed examples while remaining near chance on held-out inputs, then generalises abruptly. Prior work has focused mainly on this delay, finding that the network's main loss becomes weight decay after the fitting phase. We name the solution to this optimization problem as the minimum-norm endpoint, and study which tasks make all minimum-norm endpoints generalise from partial observations. Specifically, we model pairwise input tasks as edge colourings of $K_{q,q}$, with Latin colourings as our structured family, and classify each observed instance as all-good, mixed, or no-good by its endpoint set. Furthermore, we prove two contrasting results: (1) Finite-abelian-group families are asymptotically all-good from partial observations; (2) Under a uniform random Latin square, all-good tasks become asymptotically rare for observation-based learners, leaving mixed or no-good behaviour. Moreover, experiments show that relabellings of addition generalise despite their dense polynomial representations. Finally, on tasks of composite order, finite-training results suggest that predictors learn a…
PDF arXiv

Papers

Topics in this field

neural operators · physics informed neural · informed neural networks375 papers31 this week
graph neural · graph learning · graph neural networks312 papers27 this week
deep neural · neural networks · neural network163 papers21 this week
relu networks · gradient descent · deep neural161 papers15 this week
flow matching · normalizing flows · flow205 papers14 this week
bayesian neural · bayesian networks · amortized bayesian106 papers14 this week

Active contributors · 90 days

Activity · 12 weeks

Recent papers

All papers in Research

arXiv cs.LGOct 26
CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning
Bo-Young Lim, Seokhwan Kim, Kisung Lee, Hyuk-Yoon Kwon
Continual graph learning (CGL) maintains models as graphs evolve without retraining from scratch, while mitigating catastrophic forgetting. Existing methods typically target either transductive or inductive settings and often store past subgraphs or require task identifiers at inference. We propose Community-aware Prototype Extraction for Continual Graph Learning (CoPE-CGL), a replay-free framework that uses a parameter-free structural prior, precomputed by multi-hop Laplacian smoothing, to anchor learning, memory, and inference. CoPE-CGL combines topology-infused bottleneck adaptation, community-level prototypes, and dynamic routing by majority vote over node-to-prototype matches. The default protocol assumes each test graph comes from one learned task; a node-level variant handles mixed-task graphs. Each task adds at most a constant amount of storage, with no past graph data retained. At equal storage and without task identifiers, CoPE-CGL matches the strongest replay baseline even when it receives true task identifiers: average performance (AP) is 95.17% and 95.72% versus 94.28% and 95.82% on Arxiv and CoraFull, respectively, with 5.9-14.0 times faster runtime and zero…
PDF arXiv
arXiv physics.flu-dynOct 26
Generative AI in Turbulence: Foundations, Applications, Validations and Benchmarks
Luca Biferale, Michele Buzzicotti
Generative AI offers new approaches to turbulence problems in which incomplete observations or unresolved d.o.f. admit multiple physically plausible solutions. Potential applications include stochastic closures, super-resolution, data assimilation, uncertainty quantification, and the generation of Eulerian and Lagrangian flow ensembles. However, turbulence is not a conventional image-generation problem. Its multiscale dynamics, intermittency, extreme events, rough velocity fields and intrinsically stochastic transport make visual realism, pointwise error, and low-order spectral agreement insufficient measures of physical fidelity. We examine where generative AI provide novel capabilities with emphasis on diffusion models, score-based methods, and related probability-transport formulations. We discuss their use in unconditional generation, probabilistic reconstruction, reduced-order modeling, temporal evolution, and mesh-free representations. We argue that validation must address turbulence properties: energy transfer, structure functions, anomalous scaling, pdf tails, coherent structures, conservation laws, temporal consistency, ensemble diversity and generalization. Finally, we…
PDF arXiv
arXiv cs.LGOct 26
Subspace Crystallization in Grokking: Rank Compression Tracks the Phase Transition
Shlok Mehendale, Paras Chopra
Grokking describes understanding something deeply and intuitively. In neural networks, it is the phenomenon where a network generalizes long after memorizing its training data. This delay has been linked to a fall in the effective rank of the network's weight matrices, but whether this rank compression drives grokking or follows it is still unresolved. We study a mean-field MLP and a one-layer transformer trained on modular arithmetic and observe that: (i) Compression follows the onset, not memorization. Every transformer memorizes its training data within 250 steps, while the grokking onset (the first step at which test accuracy reaches 95%) varies from 6,500 to 21,000 steps. The steepest fall in effective rank comes about 900 steps after the onset, however late the onset occurs, and networks that memorize without grokking compress far less. (ii) Subspace crystallization. We call the span of the fewest top singular directions of a weight matrix that preserve 99% of test accuracy its sufficient subspace. Its size can shrink as the margin between the correct and the best wrong output grows, so we also count the directions needed to keep the outputs within 10%. In the transformer,…
PDF arXiv
arXiv cs.LGOct 26
FlowGenesis: Representation-Driven Self-Organization of Neural Architectures
Maksim A Kazanskii
Deep neural networks typically rely on architectures specified before training, despite the continuous evolution of their internal representations. We introduce FlowGenesis, a framework in which neural architectures adapt through neuron birth and pruning guided by Representation Flux, a local statistic capturing changes in minibatch mean activations. Neurons exhibiting persistently high Flux undergo splitting designed to approximately preserve the network function, while neurons with low Flux and low activity are pruned. This mechanism allows network capacity to evolve without prescribing a target architecture or performing a separate architecture search. Experiments on image and text classi- fication show that networks initialized with different sizes approach similar final neuron-count ranges within each dataset while maintaining comparable predictive performance. We further extend FlowGenesis to continual learning, where representation-guided structural adaptation reduces the neurons and parameters allocated to successive tasks while maintaining accuracy comparable to fixed-width Progressive Neural Networks in a task-incremental learning setting. These findings demonstrate…
PDF arXiv
arXiv cs.AIOct 26
A Diagnostic Protocol for GNNs in Electricity Load Forecasting: Separating Parameter Sharing from Neighborhood Access
Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude, Itai Zehavi, Argyris Kalogeratos
A graph neural network (GNN) forecasting a collection of load series combines two mechanisms, sharing parameters across series and reading neighboring series, and a gain over a non-graph model may come from either; telling them apart decides whether a graph is worth maintaining. We propose a model diagnostic protocol that compares the GNN with three reference models: a per-node network, a network shared across nodes without neighbors, and the same GNN without edges. Degree-preserving rewiring tests whether neighbor identity matters. After a tuned benchmark on regional load in France and on low-voltage feeders in the UK, we apply the protocol to deliberately short node histories, as when non-stationary demand leaves only recent observations relevant: the feeders, within one week, and a public dataset of client consumption series. On the feeders, sharing parameters recovers most of the error gap between node-specific models and GNNs, about two thirds against a regularized one. Adding the edges to an otherwise identical backbone brings a smaller gain, half or more of which disappears under rewiring. On the client series, the neighbor mean degrades the per-node model on the shorter…
PDF arXiv
arXiv cs.LGOct 26
Mapping the Solution-Set Structure of Grokking on Latin Squares
Zihan Gu, Lulu Zhao, Tianyi Ma, Hua Zhang, Wenbo Su +2
Grokking is a delayed transition from memorisation to generalisation. A network first fits observed examples while remaining near chance on held-out inputs, then generalises abruptly. Prior work has focused mainly on this delay, finding that the network's main loss becomes weight decay after the fitting phase. We name the solution to this optimization problem as the minimum-norm endpoint, and study which tasks make all minimum-norm endpoints generalise from partial observations. Specifically, we model pairwise input tasks as edge colourings of $K_{q,q}$, with Latin colourings as our structured family, and classify each observed instance as all-good, mixed, or no-good by its endpoint set. Furthermore, we prove two contrasting results: (1) Finite-abelian-group families are asymptotically all-good from partial observations; (2) Under a uniform random Latin square, all-good tasks become asymptotically rare for observation-based learners, leaving mixed or no-good behaviour. Moreover, experiments show that relabellings of addition generalise despite their dense polynomial representations. Finally, on tasks of composite order, finite-training results suggest that predictors learn a…
PDF arXiv

Open problems in Neural operators

Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it
Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
- Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
- MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it
Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.
- Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it
Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.
- GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it
Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 32m ago

Neural operators

Found by the topic model: neural operators, graph neural, flow matching.

156papers this week−35.3% vs last week

All papers · 1.8k With code · 11

Newest Most relevant

1.1k papers this month