Papers

Field

Top tools here

Open problems in Physics informed neural

Effective fusion of graph and text signals3 papers raise it · 1 addresses it
Combining structural and semantic information is critical for tasks like link prediction but lacks a clear optimal strategy.
Robust numerical solver diagnosis and improvement1 paper raises it · 1 addresses it
Current solvers often fail without clear explanations, and improving them is ad-hoc rather than systematic.
- Training Numerical Intelligence via Auto-Diagnosis and Skill Discoveryraises · addresses
Stable graph-structure injection into LLMs1 paper raises it · 0 address it
Graph soft prompting often collapses tokens, losing relational information and failing to improve performance.
- On the Token Collapse Phenomenon in Textual Graph Soft Promptingraises
Scalable and adaptive graph sparsification for GNNs1 paper raises it · 1 addresses it
Sparse training is costly or inflexible, and dynamic sparsification lacks efficient, reusable methods.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses
Convergence analysis for probabilistic meshless methods on manifolds1 paper raises it · 1 addresses it
Extending probabilistic meshless PDE solvers to manifolds lacks theoretical guarantees.
- Probabilistic Meshless Methods on Manifoldsraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

graph neural · graph learning · graph neural networks297 papers27 this week
physics informed neural · informed neural networks · neural operators337 papers24 this week
relu networks · deep neural · gradient descent186 papers20 this week
kernel · svm · gaussian143 papers10 this week
arnold networks · symmetries · parameter symmetries100 papers10 this week
muon optimizer · practical muon · gradient descent55 papers6 this week

Topics in this field

Active contributors · 90 days

Pietro LiòNon-Zero Radius: The Missing Component of Neural Networks10
Anima AnandkumarFG-ConvNO: A Geometry-Aware Neural Operator for Propeller CFD Prediction6
Rong-Hua LiToward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach5
Xunkai LiToward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach5
Souvik ChakrabortyFlow Mesh: A Velocity-Regularized Flow Matching Framework for Stable Distributional Modeling of Physical Fields on Unstructured Meshes4
Alessio BorgiLocaLeJEPA: Maximum Entropy Representations under Geometric Priors4

Active contributors

Activity · 12 weeks

Activity

Recent papers

All papers in Research

arXiv cs.LGSep 22
DeepFEAv2: Deep Learning for Transient Finite Element Analysis Beyond Structured Meshes
Georgios Triantafyllou, Panagiotis G. Kalozoumis, Dimitris K. Iakovidis
Finite Element Analysis (FEA) is widely used for transient mechanical simulations, but its high computational cost limits real-time and high-resolution applications. Deep learning surrogate models can reduce this cost; however, many existing approaches are restricted to steady-state prediction or cannot jointly predict Node- and Element-based Outputs (NEO) over time. The state-of-the-art DeepFEA framework has addressed these issues but remains limited to structured finite element (FE) meshes. To overcome this limitation, this study proposes DeepFEAv2, a deep learning surrogate framework that enables prediction of transient FEA simulations across different FE mesh topologies and element types. The main contributions of DeepFEAv2 are: (a) a module that uses the FE connectivity matrix to organize input features by element and arrange them into an input sequence guided by the mesh topology; (b) a novel neural network architecture designed to process the input sequence and jointly predict NEO over time; and (c) a FEA-informed optimization strategy for regularizing these NEO predictions. DeepFEAv2 was evaluated on structured and unstructured 3D linear elastic datasets, as well as on a…
PDF arXiv
arXiv stat.MLSep 22
SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa, Taejun Park
Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in…
PDF arXiv
arXiv cs.LGSep 22
Double Descent and Malign Overfitting in Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli, Marc Mézard
Conventional wisdom in deep learning holds that overparameterization---having more parameters $p$ than training samples $n$---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might expect the same benign overfitting for diffusion models, whose training reduces to regression, i.e. to minimizing a quadratic score-matching loss. Yet the opposite is observed: overfitting here is catastrophic, driving the model into a memorization regime. We resolve this paradox by combining experiments on U-Nets trained on CelebA with a random-features model for which we derive closed-form learning curves. We show that with a fixed number $m$ of noise realizations per training sample, an interpolation peak does occur, but at $p\sim nm$ rather than at $p\sim n$ as in standard regression. The rise of the test loss, however, sets in much earlier, at $p\sim n$, independently of $m$. This overfitting is malign because, although the implicit regularization of training is fully at work, it drives the model toward the empirical score, which memorizes the training set, rather than toward the…
PDF arXiv
arXiv cs.LGSep 22
PreGS: A Parameter-Transfer-Based Multi-Expert Graph Neural Network for Node Classification
Zhicong Cai, Yinglong Zhang, Xiaoying Hong, Xuewen Xia, Xing Xu
Graph neural networks have achieved strong performance in node classification by aggregating information from graph neighborhoods. However, a single aggregation mechanism may be insufficient to capture diverse structural patterns across graph datasets. Moreover, independently training multiple structural branches can introduce substantial overhead without necessarily producing stable node representations. To address these issues, this paper proposes PreGS, a parameter-transfer-based multi-expert graph neural network framework. PreGS first pretrains a multi-head graph attention network (GAT) and transfers the linear transformation weights of its first-layer attention heads to multiple GraphSAGE experts. The transferred experts are frozen and used as complementary structural branches. The fused raw node features, GAT head representations, and GraphSAGE expert representations are fed into a multilayer perceptron (MLP), whose output is further fused with the pretrained GAT logits. Based on PreGS, we further develop PreGSv2, which introduces source-level weighting and a structural gating mechanism for adaptive multi-source feature integration. Experiments on eight public graph…
PDF arXiv
arXiv cs.AISep 22
Canonical locks that encode part-whole hierarchies
Rajat Modi, Yogesh Singh Rawat
One of the challenges in representational learning is how to encode part-whole hierarchies in a neural net. Prior works rely on flattening tree-like structures into string-like sequences and training a sequence-to-sequence model via autoregression. While such a representation works for parse-trees in NLP, it is not entirely clear how to make it work for images. Thus, we propose a geometric primitive called canonical locks. The key idea is that parts/wholes can be modelled as higher-dimensional vectors ($d \geq 4$), and information can be encoded in their relative phase differences. Inductively, the net consists of positionally-bound bottom-up and top-down neural fields, which drive each other to achieve a state of thermal equilibrium. Additionally, we show the existence of a few symmetrical configurations in the net. The computational iterations taken to break these symmetries depend on the angle between parts/wholes arranged on a disk (or more precisely a ring) in higher dimensions. It also appears to have connections to the psychological phenomenon of mental rotation.
PDF arXiv
arXiv cs.LGSep 22
The Dynamics of Quasiregular Neural Learning
Matthia Sabatelli
Many learning problems combine a dominant regularity with systematic exceptions. Motivated by U-shaped learning in language acquisition, we study this interaction in controlled quasiregular regression problems where regular and exceptional solutions are explicitly known. Neural networks can partially acquire exceptions, subsequently regress toward the dominant regularity, and finally recover. This overregularization becomes substantially stronger when exceptions are rare, despite their early acquisition, but does not emerge equally across all regularities considered. Our results isolate a simple form of competition between regularities and exceptions during neural learning.
PDF arXiv

Papers

Topics in this field

graph neural · graph learning · graph neural networks297 papers27 this week
physics informed neural · informed neural networks · neural operators337 papers24 this week
relu networks · deep neural · gradient descent186 papers20 this week
kernel · svm · gaussian143 papers10 this week
arnold networks · symmetries · parameter symmetries100 papers10 this week
muon optimizer · practical muon · gradient descent55 papers6 this week

Active contributors · 90 days

Pietro LiòNon-Zero Radius: The Missing Component of Neural Networks10
Anima AnandkumarFG-ConvNO: A Geometry-Aware Neural Operator for Propeller CFD Prediction6
Rong-Hua LiToward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach5
Xunkai LiToward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach5
Souvik ChakrabortyFlow Mesh: A Velocity-Regularized Flow Matching Framework for Stable Distributional Modeling of Physical Fields on Unstructured Meshes4
Alessio BorgiLocaLeJEPA: Maximum Entropy Representations under Geometric Priors4

Recent papers

All papers in Research

arXiv cs.LGSep 22
DeepFEAv2: Deep Learning for Transient Finite Element Analysis Beyond Structured Meshes
Georgios Triantafyllou, Panagiotis G. Kalozoumis, Dimitris K. Iakovidis
Finite Element Analysis (FEA) is widely used for transient mechanical simulations, but its high computational cost limits real-time and high-resolution applications. Deep learning surrogate models can reduce this cost; however, many existing approaches are restricted to steady-state prediction or cannot jointly predict Node- and Element-based Outputs (NEO) over time. The state-of-the-art DeepFEA framework has addressed these issues but remains limited to structured finite element (FE) meshes. To overcome this limitation, this study proposes DeepFEAv2, a deep learning surrogate framework that enables prediction of transient FEA simulations across different FE mesh topologies and element types. The main contributions of DeepFEAv2 are: (a) a module that uses the FE connectivity matrix to organize input features by element and arrange them into an input sequence guided by the mesh topology; (b) a novel neural network architecture designed to process the input sequence and jointly predict NEO over time; and (c) a FEA-informed optimization strategy for regularizing these NEO predictions. DeepFEAv2 was evaluated on structured and unstructured 3D linear elastic datasets, as well as on a…
PDF arXiv
arXiv stat.MLSep 22
SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa, Taejun Park
Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in…
PDF arXiv
arXiv cs.LGSep 22
Double Descent and Malign Overfitting in Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli, Marc Mézard
Conventional wisdom in deep learning holds that overparameterization---having more parameters $p$ than training samples $n$---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might expect the same benign overfitting for diffusion models, whose training reduces to regression, i.e. to minimizing a quadratic score-matching loss. Yet the opposite is observed: overfitting here is catastrophic, driving the model into a memorization regime. We resolve this paradox by combining experiments on U-Nets trained on CelebA with a random-features model for which we derive closed-form learning curves. We show that with a fixed number $m$ of noise realizations per training sample, an interpolation peak does occur, but at $p\sim nm$ rather than at $p\sim n$ as in standard regression. The rise of the test loss, however, sets in much earlier, at $p\sim n$, independently of $m$. This overfitting is malign because, although the implicit regularization of training is fully at work, it drives the model toward the empirical score, which memorizes the training set, rather than toward the…
PDF arXiv
arXiv cs.LGSep 22
PreGS: A Parameter-Transfer-Based Multi-Expert Graph Neural Network for Node Classification
Zhicong Cai, Yinglong Zhang, Xiaoying Hong, Xuewen Xia, Xing Xu
Graph neural networks have achieved strong performance in node classification by aggregating information from graph neighborhoods. However, a single aggregation mechanism may be insufficient to capture diverse structural patterns across graph datasets. Moreover, independently training multiple structural branches can introduce substantial overhead without necessarily producing stable node representations. To address these issues, this paper proposes PreGS, a parameter-transfer-based multi-expert graph neural network framework. PreGS first pretrains a multi-head graph attention network (GAT) and transfers the linear transformation weights of its first-layer attention heads to multiple GraphSAGE experts. The transferred experts are frozen and used as complementary structural branches. The fused raw node features, GAT head representations, and GraphSAGE expert representations are fed into a multilayer perceptron (MLP), whose output is further fused with the pretrained GAT logits. Based on PreGS, we further develop PreGSv2, which introduces source-level weighting and a structural gating mechanism for adaptive multi-source feature integration. Experiments on eight public graph…
PDF arXiv
arXiv cs.AISep 22
Canonical locks that encode part-whole hierarchies
Rajat Modi, Yogesh Singh Rawat
One of the challenges in representational learning is how to encode part-whole hierarchies in a neural net. Prior works rely on flattening tree-like structures into string-like sequences and training a sequence-to-sequence model via autoregression. While such a representation works for parse-trees in NLP, it is not entirely clear how to make it work for images. Thus, we propose a geometric primitive called canonical locks. The key idea is that parts/wholes can be modelled as higher-dimensional vectors ($d \geq 4$), and information can be encoded in their relative phase differences. Inductively, the net consists of positionally-bound bottom-up and top-down neural fields, which drive each other to achieve a state of thermal equilibrium. Additionally, we show the existence of a few symmetrical configurations in the net. The computational iterations taken to break these symmetries depend on the angle between parts/wholes arranged on a disk (or more precisely a ring) in higher dimensions. It also appears to have connections to the psychological phenomenon of mental rotation.
PDF arXiv
arXiv cs.LGSep 22
The Dynamics of Quasiregular Neural Learning
Matthia Sabatelli
Many learning problems combine a dominant regularity with systematic exceptions. Motivated by U-shaped learning in language acquisition, we study this interaction in controlled quasiregular regression problems where regular and exceptional solutions are explicitly known. Neural networks can partially acquire exceptions, subsequently regress toward the dominant regularity, and finally recover. This overregularization becomes substantially stronger when exceptions are rare, despite their early acquisition, but does not emerge equally across all regularities considered. Our results isolate a simple form of competition between regularities and exceptions during neural learning.
PDF arXiv

Activity · 12 weeks

Fields · AI · updated 1h ago

Physics informed neural

Found by the topic model: physics informed neural, graph neural, relu networks.

97papers this week−38.2% vs last week

All papers · 1.1k With code · 5

Newest Most relevant

696 papers this month

Open problems in Physics informed neural

Effective fusion of graph and text signals3 papers raise it · 1 addresses it
Combining structural and semantic information is critical for tasks like link prediction but lacks a clear optimal strategy.
Robust numerical solver diagnosis and improvement1 paper raises it · 1 addresses it
Current solvers often fail without clear explanations, and improving them is ad-hoc rather than systematic.
- Training Numerical Intelligence via Auto-Diagnosis and Skill Discoveryraises · addresses
Stable graph-structure injection into LLMs1 paper raises it · 0 address it
Graph soft prompting often collapses tokens, losing relational information and failing to improve performance.
- On the Token Collapse Phenomenon in Textual Graph Soft Promptingraises
Scalable and adaptive graph sparsification for GNNs1 paper raises it · 1 addresses it
Sparse training is costly or inflexible, and dynamic sparsification lacks efficient, reusable methods.
- EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses
Convergence analysis for probabilistic meshless methods on manifolds1 paper raises it · 1 addresses it
Extending probabilistic meshless PDE solvers to manifolds lacks theoretical guarantees.
- Probabilistic Meshless Methods on Manifoldsraises · addresses

Named by an AI from these papers' abstracts.