TodayNewsResearchFieldsStacksMarketAnalytics
⌘K

Research

New papers ranked by attention, with their code, models and discussion linked. Pick a field on the left.

Research · AI field

Neural operators

Found by the topic model: neural operators, graph neural, flow matching.

Open the field page
150papers this week−36.2% vs last week1050 this month

Papers

Open problems

Rising research topics

Ask the research

Sample
ByteTrail · AI and developer-tools intelligence
NewsResearchStacksAPI
Fields · this week
All fields1916Multimodal319Frontier LLMs236Recursive self improvement226Agents & indirect prompt injection175Inference & serving172Neural operators150Clinical reasoning138Robotics122Forecasting time75Linear bandits50AI safety46Voice AI41Character recognition OCR30Molecular property prediction27Federated learning18All 23 fields
WeekMonthQuarterAll
NewestMost relevant

Papers

Open problems in Neural operators

  • Stable and efficient online reinforcement learning for diffusion models3 papers raise it · 3 address it

    Current methods are either inefficient due to poor credit assignment or unstable without careful tuning, limiting practical deployment.

    • DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Modelsraises · addresses
    • Routing Policy Optimization for Online Diffusion Reinforcement Learningraises · addresses
    • MEND: RL For Flow Models via Proximal Velocity Matchingraises · addresses
  • Scalable minibatch optimal transport for flow matching1 paper raises it · 1 addresses it

    Small minibatches limit benefits of optimal transport, while large minibatches are computationally prohibitive, and theoretical gaps remain in gradient and integration stability.

    • Scaling Online Optimal Transport Minibatches for Flow Matchingraises · addresses
  • Benchmarking System One models on graph tasks1 paper raises it · 1 addresses it

    Capabilities of fast, choice-based models on graph-related tasks are unclear, and existing evaluations lack standardized, diagnostic benchmarks.

    • GraphDecisionBench: Benchmarking System One Models on Graph Tasksraises · addresses
  • Efficient edge sparsification for graph neural networks1 paper raises it · 1 addresses it

    Reusing a fixed sparse graph locks training to a fixed topology, while dynamic sparsification is computationally expensive.

    • EDiS: Edge Disjoint Subgraph Sparsification Framework for Graph Neural Networksraises · addresses

Named by an AI from these papers' abstracts.

Rising research topics

  • Algorithmic fairness10+42.9%
  • Spiking neural networks7+40.0%
  • Diffusion priors16+14.3%
  • Channel state information8−20.0%
  • Quantum machine learning14−22.2%
  • Character recognition OCR30−28.6%
  • Linear bandits50−31.5%
  • Molecular property prediction27−32.5%

Papers

  • arXiv eess.SYOct 29

    Adaptive Scenario Generation for KL-DRO Around a Learned Density Model

    Aron Brenner, Saurabh Amin

    Distributionally robust optimization (DRO) hedges data-driven two-stage stochastic programs against error in the distribution estimated from a finite record, and standard ambiguity sets differ in how they treat realizations absent from that record. A divergence ball around the empirical distribution contains only reweightings of the record, whereas a transport ball also contains distributions supported off the record, including on realizations the data-generating process cannot produce, against which the decision is then hedged as well. We instead propose a Kullback-Leibler (KL) ball centered at a density model fitted to the record, the learned nominal, which lets the adversary up-weight costly unobserved realizations in proportion to the probability the model assigns them. Solving the resulting problem poses a second difficulty, because it can accommodate only a few scenarios, and an independent sample of that size is likely to omit the rare realizations on which the worst case concentrates. We therefore build the scenario set adaptively, alternating between solving the finite problem and drawing new realizations from the learned nominal, steered toward the worst case at the…

    PDF
  • arXiv cs.LGOct 29

    An ODE-based Sampler for Discrete Diffusion Models

    Samson Gourevitch, Yazid Janati, Eric Moulines, Eric P. Xing, Alain Durmus +1

    The marginals of a continuous diffusion model can be reproduced by a deterministic ordinary differential equation (ODE), known as the probability flow ODE, which provides a foundation for accelerated sampling and consistency distillation. In this paper we address the question whether an analogous formulation exists for discrete diffusion models and answer it affirmatively by introducing exponential residual flows (ERFs). Our framework augments the discrete state with a continuous residual variable for each token and position. These residuals evolve deterministically according to piecewise-defined ODEs whose vector fields depend on the current discrete state and the forward and backward rate matrices of the prescribed diffusion model. The discrete state changes whenever a residual reaches zero. We prove that this construction preserves the marginals of the original diffusion. We derive a practical discretization of our sampler and specialize it to masked and uniform diffusion. The deterministic trajectories of ERFs support consistency distillation and yield empirical improvements over previous approaches across a variety of benchmarks including code and text generation tasks.

    PDF
  • arXiv math.NAOct 29

    NEOPIC: A Neural Operator Framework for Particle-based Kinetic Plasma Simulations

    Sriramkrishnan Muralikrishnan, Chelsea Maria John, Jeffrey Kelling, Michael Bussmann, Robert Speck +1

    We introduce NEOPIC, a framework for learning the field operator in particle-based methods applied to kinetic plasma simulations. Its key novelty is a neural operator that acts directly on the empirical measure of the particles, rather than on a grid, and is evaluated at the particle positions. No mesh or tree is needed and, unlike grid-based neural operators, no deposition onto a mesh takes place, so the particle noise that dominates the inputs of particle simulations is not aliased. Since it replaces only the field solve, it serves as a plug-and-play field solver for particle-based methods: it can be trained on data from particle-in-cell, particle-in-Fourier or tree codes, and combined with any time integrator, including implicit schemes. We prove that the discrete evaluation coincides with the continuous operator at any number of particles (representation equivalence) and derive an error analysis that separates sampling, learning and time-stepping errors: up to the learning error, NEOPIC has the error structure of standard Monte Carlo particle methods. We show that the particle noise in the training data acts as an implicit regularizer, which gives guidance on the number of…

    PDF
  • arXiv cs.LGOct 29

    Numerical Integration on Bounded Domains via Learned Reflected Diffusions

    Kirill Korolev, Artur Goldman, Timofei Gritsaev, Tigran Ramazyan, Nikita Morozov

    We study high-dimensional integration on bounded convex domains. Given a non-negative integrand $r$, the goal is to estimate $Z=\int_{D}r(x)\,\mathrm dx$ on $D\subset\mathbb R^d$. Standard Quasi-Monte Carlo does not adapt its evaluation points to the integrand, while effective importance sampling requires a proposal that captures its concentration. Learned diffusion samplers can supply such a proposal, but existing methods are formulated on $\mathbb R^d$ and do not respect the domain constraint $x\in D$. We instead build the proposal from reflected dynamics that stay in $D$ by construction. A reflected diffusion progressively transforms the target distribution toward the uniform distribution on $D$, and a neural score model is trained to reverse this process. A change-of-measure formula for reflected diffusions yields a training objective without additional boundary terms and an unbiased importance-sampling estimator of $Z$. For non-smooth integrands, where reparameterization gradients may be undefined or uninformative, we derive a REINFORCE estimator of the training gradient using Gaussian proposal densities before reflection, without evaluating reflected transition densities or…

    PDF
  • arXiv cs.LGOct 29

    Reinforcement learning for control of partial differential equations

    Nicolò Botteghi, Matteo Tomasetto, Andrea Manzoni

    Controlling dynamical systems governed by partial differential equations (PDEs) remains a challenging task in scientific computing, owing to high-dimensional state spaces, distributed actuation, nonlinear dynamics, and stringent real-time constraints. Reinforcement learning (RL) has emerged as a promising paradigm for synthesizing feedback controllers that adapt to varying operating conditions and can be deployed efficiently in real-time. This review provides a unified perspective on RL and PDE-constrained optimization, and surveys model-free, model-based, and multi-agent RL approaches to PDE control, with particular emphasis on sample efficiency, generalization across parametric variations, and scalability to high-dimensional problems. Two demonstrative examples, the stabilization of a chaotic system and a flow control problem, illustrate the performance and pitfalls of RL agents in practice. Finally, we discuss evaluation and benchmarking practices, current limitations, and open research directions toward reliable, data-efficient, and scalable RL for scientific computing.

    PDF
  • arXiv cs.LGOct 29

    A Pretrained Model for Bayesian Filtering in Heterogeneous Dynamical Systems

    Ke Fan, Wen Wu, Haitao Su, Yilin Wang, Yuan Zhou +1

    Filtering has not yet had its foundation-model moment: state estimation is still dominated either by online Bayesian filtering algorithms that require known dy- namics as inputs, or by learning-based neural filters trained from scratch for a single system or parametric family. The core challenge is structural heterogene- ity: dynamical systems differ in their governing dynamics, physical scales, and noise levels, precluding a shared representation space for a reusable filter. To bridge this gap, we introduce FilterFM, the first pretrained model for Bayesian filtering that estimates latent states across heterogeneous dynamical systems with a single checkpoint. FilterFM employs a hierarchical two-level observation se- quence encoder and a prior-to-posterior state-reconstruction module that together handle varying observation and state dimensions across heterogeneous systems, while modeling state uncertainty via quantile prediction. Trained on a mixture of heterogeneous filtering tasks, FilterFM exhibits superior in-domain performance, delivering significant gains in estimation accuracy, runtime efficiency, and noise robustness compared to existing baselines. Beyond in-domain…

    PDF
  • arXiv cs.LGOct 29

    Composing and Steering Diffusion Models with Generalized Feynman-Kac Correctors

    Wael Mattar, Daniel Freedman

    Diffusion models have become the leading paradigm in generative modeling, yet many applications require sampling from distributions that are not those of any single pretrained model, but are instead defined through combinations of several models, possibly tilted toward high values of a downstream reward. Naively combining the scores of the models yields incorrect samples, while existing corrections are restricted to specific families of targets. In this work, we introduce the generalized Feynman-Kac correctors (GFKC), a principled framework for sampling from an arbitrary target function of several pretrained diffusion models. We show that the geometric interpolation between an arbitrary initial function and the target function of the models satisfies a Feynman--Kac partial differential equation, and derive its potential in closed form. To evaluate this potential during generation, we further generalize the log-probability ODE to track the density of each model along trajectories driven by a different velocity field. The resulting algorithm operates entirely in the log domain, incorporates reward-based steering at no additional conceptual cost, and corrects the generation on the…

    PDF
  • arXiv cs.AIOct 29

    Bridging Semantics and Structure: A Theoretically Grounded Framework for Link Prediction on TAGs

    Chen Shao, Ruirui Wang, Tobias Kafer, Danai Koutra

    Link prediction on text-attributed graphs (TAGs) is a fundamental problem with applications in knowledge discovery, recommendation and beyond. Recent progress has highlighted the promise of combining Graph Neural Networks (GNNs) with Language Models (LMs) in order to leverage the complementary structural and semantic signals. However, due to the distinct inductive biases of GNNs and LMs, the design of an optimal fusion strategy remains an open research question. In this work, we present Bridge-LP, an intuitive, simple, efficient and theoretically grounded framework that unifies graph and language representations. It is built around three theoretically-justified design components: (D1) jumping connection of textual proximity, which enables robust LM training; (D2) compressed embedding, which encodes neighborhood information long exploited by classical link prediction heuristics, thereby enriching structural signals in a learnable way; and (D3) cache embedding training strategy, which reduces computational complexity. Together, these mechanisms establish an end-to-end cross-modal framework that scales to large graphs while capturing richer graph-text interactions. We conduct an…

    PDF
  • arXiv cs.LGOct 29

    Mode Dynamics: A Framework for Neural Network Learning

    Hossein Honarvar

    We introduce Mode Dynamics, a framework that connects coordinated parameter motion during neural network learning to changes in model outputs. Starting from a recorded parameter trajectory, we decompose its updates into collective modes and map each mode to its state-dependent output effect. The framework organizes this link through three mechanisms. Decomposition identifies collective modes in parameter motion and their time-varying amplitudes. Reconstruction tests whether the effects induced by those modes recover local and accumulated output change. Concentration quantifies how behavioral contribution is distributed across modes. Controlled scalar regression and next-token experiments directly test these mechanisms. They show that small sets of leading modes reconstruct behavior, induced effects depend on the training path, and modal complexity varies with task and data. We then evaluate the framework on held-out image and language-model outputs using CIFAR-100 autoencoding and a public Pythia-70M pretraining trajectory. Across these settings, the results connect compact modal subspaces of recorded parameter motion to evolving behavior.

    PDF
  • arXiv cs.LGOct 29

    FIDE: Fiber-Informed Differentiable Experimental Design for Reference-Relative Law Dynamics

    Pietro Zanotta, Ján Drgoňa

    Aggregate measurements generally leave an evolving population law underdetermined, so different measurement systems can be equally adequate for a scientific task while constraining different aspects of the law. We study how to resolve this residual design freedom in baseline-preserving settings, where an existing dynamical model remains the working baseline outside quantities explicitly valued by the task. \textbf{F}iber-\textbf{I}nformed \textbf{D}ifferentiable \textbf{E}xperimental Design (\textbf{FIDE}) completes each candidate's aggregate constraints by information projection relative to a common frozen reference, then ranks scientifically adequate designs by the dynamical revision required to realize the resulting completed paths. The resulting \emph{Full action} is the minimum mean-squared velocity correction needed to realize a completed path; equivalently, its square root is the sharp reference-relative expectation-rate discrepancy over a gradient-normalized Sobolev class. This defines a conditional minimum-revision principle for baseline-preserving design rather than a generic truth-recovery objective. Experiments show that this preference changes sensor selection, can…

    PDF
  • arXiv cs.AIOct 29

    Efficient Learning of Mechanism Designs and Compositionality

    João Pedro Gandarela, Thiago Rios, Stefan Menzel, André Freitas

    Learning mechanism behaviour requires a representation of how connected parts jointly constrain motion. We study whether exposing reusable constraint subassemblies improves prediction from limited data. Assur-KB represents planar mechanisms through irreducible modules, boundary interfaces, and solve dependencies while retaining source geometry. Our theoretical results characterize transfer through module interfaces and establish physical limits arising from missing interface information and accumulated sensitivity. To test the representation claim directly, the experimental design compares matched mechanism-curve pairs across training budgets within the same multilayer-perceptron family. This controlled comparison establishes the utility of the complete representation in the evaluated learning setting. Assur-derived features provide a geometric-mean $1.8\times$ advantage in sample efficiency over flattened adjacency and position inputs across budgets of 50-10,000 mechanisms. Separate experiments establish invariance under tested joint relabelings, higher valid-generation rates under dyadic composition, and a coarser structural index over 10.02 million mechanisms. In summary, the…

    PDF
  • arXiv cs.LGOct 29

    Data-driven Random-feature Approximation of Koopman Generators of Nonlinear Continuous-time Systems

    Ana Čukarska, Maximilian Gollwitzer, Viola Muning Li, Vladyslav Fediukov, Zahra Monfared +1

    Learning continuous-time dynamical systems from data is an active area of research. The Koopman formalism is often used here, because it is a flexible approach that is informative about the system's fundamental properties. The formalism represents dynamical systems through linear operators acting on a suitable function space. A practical difficulty arising in the numerical approximation of Koopman operators is the choice of a ``dictionary'', meaning an effective finite-dimensional representation of the function space. In this paper, we focus on learning dynamical systems in continuous-time, and so we consider numerical approximations of the Koopman generator instead of the Koopman operator. Our main contribution is a data-driven random-feature method to obtain a parsimonious and effective function representation for Galerkin approximation methods such as gEDMD. The representation we construct is data-informed, which allows us to improve upon data-agnostic methods. We empirically demonstrate the suitability of our method for forecasting tasks and system analysis. The numerical experiments show that our approach is accurate, and faster than iterative optimization. Further, we prove…

    PDF
  • arXiv cs.CVOct 29

    sDMD: Sinkhorn Distribution Matching Distillation

    Jaihoon Kim, Lingxiao Li, Soshi Shimada, Roshni Cooper, Minhyuk Sung +1

    Distribution Matching Distillation (DMD) distills multi-step diffusion models into few-step generators by minimizing reverse KL divergence. Despite its strong performance, DMD guides samples by local density alone, which introduces two limitations: teacher scores can be unreliable in regions of low teacher density, and the mode-seeking behavior of reverse KL can collapse the student onto a subset of the teacher modes. We first connect DMD to Wasserstein gradient flow, showing that its gradient follows from a fixed-point formulation with integral KL (IKL) as the energy functional. This perspective motivates augmenting IKL with an optimal transport term that addresses these limitations by guiding outlier samples toward the target and promoting mode coverage. We realize this term using Sinkhorn divergence estimated from finite batches. In practice, however, errors in the estimated Sinkhorn velocity can perturb in-domain samples for which IKL guidance is already accurate. To this end, we introduce sDMD, which adaptively activates the Sinkhorn correction based on discriminator outputs, focusing on outlier samples where IKL may be unreliable. With 2--4 sampling steps on FLUX.1-dev,…

    PDF
  • arXiv stat.MLOct 29

    Probabilistic Meshless Methods on Manifolds

    Federico Pavesi, Noémie Jaquier, Antonio Candelieri

    Probabilistic meshless methods solve partial differential equations (PDEs) by placing a Gaussian process prior on the solution and conditioning it on the PDE at a finite set of collocation nodes. The posterior provides both an approximation of the solution and a Bayesian quantification of its error. While these methods are well established on Euclidean domains, their extension to manifolds lacks, to our knowledge, any convergence analysis. We extend them to compact manifolds and study their convergence in two settings. First, for known manifolds whose Laplace-Beltrami eigenpairs are available, the prior can be evaluated exactly, but the PDE operator generally cannot be applied to it in closed form. We replace this operator by a standard generalized finite difference discretization on a point cloud sampled from the manifold, and show that the resulting method is consistent at the points of the cloud, at a rate governed by their fill distance. Second, for implicit manifolds, known only through a point cloud, both the prior and the operator must be approximated. We propose a new generalized finite difference scheme with a free stencil radius, and show that it retains the consistency…

    PDF
  • arXiv cs.LGOct 29

    Possibility-based Inference: A Deterministic Framework for Robust and Reliable Uncertainty Quantification in Decision-Making

    Seyedfakhredin Musavishavazi, Maryam Alipour

    Decision-making under uncertainty remains a key issue in Machine Learning (ML), particularly in the deployment of safety-critical systems, where reliable quantification of system behavior is essential for evaluating behavioral impacts and strengthening confidence in ML models. This contribution, influenced by fuzzy set theory and grounded in Bayesian statistics, addresses challenges related to data precision and the performance-accuracy trade-off in Bayesian inference (BI). To this end, we introduce a possibility-based contrastive grade of prediction as an alternative quantification method to existing latent posterior-based approaches for parameter estimation (approximation) and model selection. Unlike conventional Bayesian inference, which represents uncertainty within the probabilistic framework, the proposed model adopts a possibility-based representation that allows uncertain and potentially conflicting information to be characterized more flexibly. Rather than focusing on isolated aspects of a problem, the resulting mathematical framework provides a systematic approach to data mining while mitigating overly confident, yet implausible predictions generated by highly…

    PDF
  • arXiv cs.LGOct 29

    Going deep, reaching further: What can Boltzmann machines efficiently represent?

    Tobias Reinhart, Gemma De les Coves

    While universal approximation theorems theoretically underpin the representation capabilities of machine learning models, they have two shortcomings: They do not focus on efficient representations and they tend to be insensitive to depth and other heuristically important properties. In this work we show that these two shortcomings are intertwined. We prove that machine learning models which are deeper or have higher-order interactions can efficiently represent a larger set of functions, i.e. have larger efficient reach. It follows that deep Boltzmann machines (DBMs) have maximal efficient reach, restricted Boltzmann machines (RBMs) sub-maximal, and k-body generalizations of RBMs form a hierarchy between the two, even though all are universal. More generally, (i) we provide a full characterization of machine learning models with maximal efficient reach; (ii) we derive separations between deep and shallow models for three representation tasks; and (iii) we prove that the efficient reach of shallow models constitutes an infinite hierarchy indexed by the interaction order. The proofs draw on different methodologies, namely on (i) a recent characterization of universal spin models;…

    PDF
  • arXiv cs.LGOct 29

    Grounding Predictive Reliability through Masked Observability

    Shang Gao, Bernhard Sick, Franz Götz-Hahn

    An uncertainty estimate is meaningful only if its semantics can be tied to observable predictive behavior. Normal-Inverse-Gamma (NIG) evidential regression provides a structured decomposition of predictive variability into data-scale and predictive-mean components, but marginal likelihood does not identify their allocation. We address this semantic ambiguity in masked representation learning. MORSE (Masked Observability for Reliable Structured Encoding) predicts the same hidden target under controlled changes in visible context and uses normalized cross-context variation to ground predictive-mean uncertainty, while a Gaussian objective learns the conditional mean and data scale. The normalized variation plays the role of the NIG factor $1/ν$, which the marginal leaves unidentified; this yields a policy-relative observability variance that measures dependence on visible evidence, distinct from variation induced by resampling the training set. On a Gaussian-process benchmark with a contextual reference built from analytic Bayes conditional means, MORSE has the lowest log-variance error among compared evidential, ensemble, and dropout estimators in all nine tested settings, while…

    PDF
  • arXiv cs.LGOct 29

    Implicit Bias of (Stochastic) Gradient Descent on Deep Linear Diagonal Networks

    Wiebke Bartolomaeus, Holger Rauhut

    Modern machine learning models are often overparameterized, i.e., have more parameters than training data. This typically implies that the loss function admits multiple minimizers so that training methods (and their hyperparameter choices) have significant influence on the computed solution. We study this so-called implicit bias phenomenon in the context of training deep linear diagonal networks. Previous work has shown that the gradient flow for learning such networks converges to (almost) sparse solutions when initialized small. For an arbitrary number of layers, we extend this analysis to the practically more realistic (stochastic) gradient descent iterations. Our approach identifies (stochastic) gradient descent as a perturbed (stochastic) mirror gradient descent. This allows us to establish convergence and characterize the limiting solution, relating it to weighted $\ell_1$-minimization. Moreover, we provide sufficient conditions on the step sizes and quantify the influence of the step sizes on the implicit bias.

    PDF
  • arXiv math.OCOct 29

    Carbon-Aware Electric Vehicle Routing Problem with Decision-Focused Learning

    Jaume Reixach, Wanru Chen, Jiaming Wu, Balázs Kulcsár

    Electric vehicles produce no tailpipe emissions, but charging them produces emissions that depend on the time-varying generation mix of the electricity consumed. We study an Electric Vehicle Routing Problem with a carbon-aware objective that accounts for charging-related emissions alongside conventional routing costs. The problem determines routing, charging, and waiting decisions, allowing charging demand to be shifted toward periods with lower grid carbon intensity. Because these decisions depend on forecasts of future carbon intensity, we propose a decision-focused learning approach so that the training of the model employed for predicting the carbon intensity of the grid over time is solver-aware. In particular, we train a linear predictor using approximate SPO+ directions and employ a time-limited Large Neighborhood Search followed by exact postprocessing as the underlying solver. Experiments on realistic Germany-based instances show that explicitly considering charging-related emissions reduces realized emissions by 28.76% on average relative to carbon-blind optimization, although at the expense of higher non-carbon operating cost. Moreover, decision-focused training…

    PDF
  • arXiv math.DSOct 29

    Learning Kolmogorov Backward Operators for Spectral Analysis and Forecasting: A Data-Driven Variational Galerkin Framework

    Yuanchao Xu, Claire Valva, Ben Gao, Isao Ishikawa

    For a time-homogeneous Markov process, the fixed-lag solution operator \( (T_τf)(x)=\mathbb E[f(X_{t+τ})\mid X_t=x] \) of the Kolmogorov backward equation evolves observables linearly and deterministically, even when the dynamics are stochastic. We approximate this operator using a two-stage method where we first approximate this operator directly from data, without knowledge of the governing equation (e.g., Fourier Neural Operator), and then compress the Galerkin projection onto a finite-dimensional observable space that is fixed in advance or selected variationally. The error analysis traces the learning error of the estimator, the sampling error of the Galerkin inner products, and the invariance and reconstruction properties of the observable space to the computed spectra and forecasts. The sampling error enters only through a multiplicative factor. Near an isolated spectral cluster, the invariance defect of a subspace controls its distance from the target invariant subspace. The forecast error is controlled by the one-step error of the reduced model and the projection error of the observable; when the measure is invariant and the reduced model preserves constants and is…

    PDF
Load more