Papers

Field

Top tools here

Open problems in Classifier & causal discovery

Benchmarking agentic black-box optimization1 paper raises it · 1 addresses it
Lack of standardized cross-domain benchmarks makes it hard to compare methods and isolate design choices in agentic BBO.
- A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimizationraises · addresses
Identifying individual world latents without reconstruction1 paper raises it · 1 addresses it
Existing methods fail to recover individual latents without reconstruction, decoders, or labels, limiting interpretability and causal analysis.
- DSReg: Provably Recovering Individual World Latents without Reconstructionraises · addresses
Long-tailed recognition with instance-wise confusion1 paper raises it · 2 address it
Class-wise logit adjustments ignore instance-level confusion, harming minority class performance in imbalanced datasets.
- CILA: Confusion-based Instance-wise Logit Adjustment for Long-Tailed Recognitionraises · addresses
- Using Set Learning to Improve Set-Valued Classifiers on Long-tailed Dataaddresses
Data augmentation under label noise1 paper raises it · 1 addresses it
Theoretical understanding of when augmentation helps or harms generalization in noisy-label settings is missing.
- Understanding Data Augmentation under Label Noiseraises · addresses
Causal attribution with scarce incident data1 paper raises it · 1 addresses it
Exact attribution of alarm-score changes in causal systems is combinatorial and requires incident data, limiting practicality.
- What Moved the Alarm Score? Causal Attribution with Exact Computation and Scarce-Incident Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

classifier · supervised · machine learning183 papers42 this week
causal discovery · causal inference · causal models123 papers19 this week
conformal prediction · split conformal prediction · aware conformal71 papers14 this week
bayesian optimization · bayesian optimization bo · black box optimization93 papers12 this week

Topics in this field

Active contributors · 90 days

Yujia ZhengDSReg: Provably Recovering Individual World Latents without Reconstruction4
Gunwoong ParkLocal Fisher Information Enables Sparse Causal Discovery4
Aapo HyvärinenIdentifying Dependent Components from Multi-domain Linear Mixtures3
Sara MagliacaneIdentifying Dependent Components from Multi-domain Linear Mixtures3
Stefan FeuerriegelDebiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference3
Isabel ValeraDeDiCa: A Scalable Deconfounding Discrete Causal Generative Model3

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv stat.MLOct 6
ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding
Christophe Muller, Ayub Kharel, Alex Luedtke, Chan Park, Eric Tchetgen Tchetgen +4
Standard causal identification methods often assume no unmeasured confounding and can fail when relevant confounders are unobserved. Proximal causal inference instead uses proxy variables to identify effects under hidden confounding. However, nonparametric proximal estimation can be challenging in practice: recovering causal estimands such as the conditional average treatment effect (CATE) requires solving an ill-posed integral equation that is data-hungry, hyperparameter-sensitive, and optimization-unstable. Bayesian inference for such models provides a desirable alternative, mitigating these difficulties by regularizing through the prior. However, computing a posterior is itself challenging, as a typical likelihood function will include latent variables. Following the recent success of tabular foundation models in backdoor, instrumental variable, and frontdoor settings, we propose that prior-data fitted networks (PFNs) are uniquely suited to resolve this bottleneck. Indeed, by training on synthetic data sampled from compliant structural causal models with access to oracle counterfactuals, we simplify the task substantially, amortizing the implied Bayesian operator inversion…
PDF arXiv
arXiv cs.MSOct 6
Scen-Opt: A Scenario Optimization Toolbox for Data-Driven Convex Programming
Ben Wooding, Simone Garatti, Marco C. Campi, Abolfazl Lavaei
The scenario approach is a well-established statistical framework for data-driven decision-making. In particular, in data-driven optimization, the scenario approach unveils how the problem structure governs out-of-sample generalization, and offers a principled basis for assessing and certifying the reliability of the optimal solution as per constraint satisfaction. Despite its strong theoretical development and wide applicability, no software toolbox has been available to date that enables user-friendly, data-driven convex optimization within the scenario-approach framework. In this paper, we introduce Scen-Opt, an open-source software tool that integrates convex programming with data samples while providing statistical guarantees grounded in scenario theory. Scen-Opt is implemented in Python, supporting data-driven linear, quadratic, and semidefinite programming, and offers a Python-based web application with an intuitive and reactive graphical user interface (GUI) built using modern web technologies. Scen-Opt can be used directly through its online interface or installed locally, accommodating both manual input and data-file uploads (CSV, JSON, TXT, TSV, MAT, Excel, NPY, NPZ,…
PDF arXiv
arXiv cs.LGOct 6
Hub for Outliers, Spokes for Inliers: Uniform Latent Space Construction for Dual-Mismatched Semi-Supervised Learning
Li Yuan, Yaxin Hou, Jiawei Tang, Yongbiao Gao, Yuheng Jia
Semi-supervised learning typically assumes that labeled and unlabeled data share an identical class distribution and label space. However, this setting is often violated: unlabeled data may be imbalanced and contain unknown class samples, causing mismatches in both class distribution and label space. Such dual mismatch leads to majority classes dominating the latent space and unknown class samples being overconfidently misclassified, degrading feature discriminability and pseudo-label quality. To address this, we propose a hub-spoke latent geometry, where known classes are uniformly distributed around a central hub and each class forms compact clusters around its prototype, while the hub provides an anchor for a low-evidence region specifically designed for high-uncertainty unknown class samples. Integrated with an evidence-based classifier, this geometry ultimately enhances feature discriminability and uncertainty separation by mitigating majority-class domination through structured feature organization and guiding high-uncertainty unknown class samples toward the hub. Extensive experiments show that our method outperforms state-of-the-art methods, with a maximum improvement of…
PDF arXiv
arXiv cs.LGOct 5
Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures
Paweł Lorek, Rafał Nowak, Rafał Topolnicki, Tomasz Trzciński, Maciej Zięba
We study estimating rare-event probabilities $I = \mathbb{P}(g(\mathbf{X}) > γ)$ with $\mathbf{X} \sim \mathcal{N}(\boldsymbolμ, \boldsymbolΣ)$ and general $g : \mathbb{R}^d \to \mathbb{R}$. We address this problem through importance sampling, and propose a framework that substantially improves efficiency and robustness over baselines such as crude Monte Carlo, adaptive cross-entropy, variational-inference-based methods (including reverse- and forward-KL approaches), as well as Safe-ICE, Subset Simulation, and Sequential Monte Carlo, drawing on ideas from both rare-event estimation and cross-entropy optimization. The key contribution has two parts: first, we separate the problem into coverage, to overcome the cold-start barrier, and fitting, to refine proposals once a meaningful signal is available; second, we constrain the final GMM proposal so that it has finite importance-sampling variance (since coverage alone is not sufficient -- without safeguards, importance sampling may still suffer from infinite variance). Together, these ingredients yield expressive proposals; finite variance does not by itself guarantee practical stability at a fixed sampling budget. Extensive…
PDF arXiv
arXiv stat.MLOct 5
Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds
Davide Sartor, Meghan E. Huber, Donghyun Kim, Nathan Wycoff
Bayesian Optimization (BO) has become an established methodology for minimizing black-box functions of a vector input. Often, however, this parameter vector arises from the discretization of an inherently functional relationship. Several recent articles have considered the Functional Bayesian Optimization (FBO) setting, in which the variable to be optimized is not a member of a finite dimensional vector space, but rather an infinite dimensional function space. In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients. We discuss in detail the relationship between our method and existing ones, providing a unifying lens through which to view prior works. To assess our method against the state of the art, we conduct an extensive computational study, and along the way develop a novel set of benchmark test functions which port standard finite-dimensional ones to the infinite dimensional domain. Our experiments demonstrate that, on balance, the proposed…
PDF arXiv
arXiv cs.LGOct 5
Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models
Yuta Kobayashi, Divyam Madaan, Shalmali Joshi
Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates epistemic uncertainty (arising from lack of offline data) with aleatoric uncertainty (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.
PDF arXiv

Papers

Topics in this field

classifier · supervised · machine learning183 papers42 this week
causal discovery · causal inference · causal models123 papers19 this week
conformal prediction · split conformal prediction · aware conformal71 papers14 this week
bayesian optimization · bayesian optimization bo · black box optimization93 papers12 this week

Active contributors · 90 days

Yujia ZhengDSReg: Provably Recovering Individual World Latents without Reconstruction4
Gunwoong ParkLocal Fisher Information Enables Sparse Causal Discovery4
Aapo HyvärinenIdentifying Dependent Components from Multi-domain Linear Mixtures3
Sara MagliacaneIdentifying Dependent Components from Multi-domain Linear Mixtures3
Stefan FeuerriegelDebiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference3
Isabel ValeraDeDiCa: A Scalable Deconfounding Discrete Causal Generative Model3

Activity · 12 weeks

Coming with more sources: repos

Open problems in Classifier & causal discovery

Benchmarking agentic black-box optimization1 paper raises it · 1 addresses it
Lack of standardized cross-domain benchmarks makes it hard to compare methods and isolate design choices in agentic BBO.
- A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimizationraises · addresses
Identifying individual world latents without reconstruction1 paper raises it · 1 addresses it
Existing methods fail to recover individual latents without reconstruction, decoders, or labels, limiting interpretability and causal analysis.
- DSReg: Provably Recovering Individual World Latents without Reconstructionraises · addresses
Long-tailed recognition with instance-wise confusion1 paper raises it · 2 address it
Class-wise logit adjustments ignore instance-level confusion, harming minority class performance in imbalanced datasets.
- CILA: Confusion-based Instance-wise Logit Adjustment for Long-Tailed Recognitionraises · addresses
- Using Set Learning to Improve Set-Valued Classifiers on Long-tailed Dataaddresses
Data augmentation under label noise1 paper raises it · 1 addresses it
Theoretical understanding of when augmentation helps or harms generalization in noisy-label settings is missing.
- Understanding Data Augmentation under Label Noiseraises · addresses
Causal attribution with scarce incident data1 paper raises it · 1 addresses it
Exact attribution of alarm-score changes in causal systems is combinatorial and requires incident data, limiting practicality.
- What Moved the Alarm Score? Causal Attribution with Exact Computation and Scarce-Incident Estimationraises · addresses

Named by an AI from these papers' abstracts.

Recent papers

All in Research →

arXiv stat.MLOct 6
ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding
Christophe Muller, Ayub Kharel, Alex Luedtke, Chan Park, Eric Tchetgen Tchetgen +4
Standard causal identification methods often assume no unmeasured confounding and can fail when relevant confounders are unobserved. Proximal causal inference instead uses proxy variables to identify effects under hidden confounding. However, nonparametric proximal estimation can be challenging in practice: recovering causal estimands such as the conditional average treatment effect (CATE) requires solving an ill-posed integral equation that is data-hungry, hyperparameter-sensitive, and optimization-unstable. Bayesian inference for such models provides a desirable alternative, mitigating these difficulties by regularizing through the prior. However, computing a posterior is itself challenging, as a typical likelihood function will include latent variables. Following the recent success of tabular foundation models in backdoor, instrumental variable, and frontdoor settings, we propose that prior-data fitted networks (PFNs) are uniquely suited to resolve this bottleneck. Indeed, by training on synthetic data sampled from compliant structural causal models with access to oracle counterfactuals, we simplify the task substantially, amortizing the implied Bayesian operator inversion…
PDF arXiv
arXiv cs.MSOct 6
Scen-Opt: A Scenario Optimization Toolbox for Data-Driven Convex Programming
Ben Wooding, Simone Garatti, Marco C. Campi, Abolfazl Lavaei
The scenario approach is a well-established statistical framework for data-driven decision-making. In particular, in data-driven optimization, the scenario approach unveils how the problem structure governs out-of-sample generalization, and offers a principled basis for assessing and certifying the reliability of the optimal solution as per constraint satisfaction. Despite its strong theoretical development and wide applicability, no software toolbox has been available to date that enables user-friendly, data-driven convex optimization within the scenario-approach framework. In this paper, we introduce Scen-Opt, an open-source software tool that integrates convex programming with data samples while providing statistical guarantees grounded in scenario theory. Scen-Opt is implemented in Python, supporting data-driven linear, quadratic, and semidefinite programming, and offers a Python-based web application with an intuitive and reactive graphical user interface (GUI) built using modern web technologies. Scen-Opt can be used directly through its online interface or installed locally, accommodating both manual input and data-file uploads (CSV, JSON, TXT, TSV, MAT, Excel, NPY, NPZ,…
PDF arXiv
arXiv cs.LGOct 6
Hub for Outliers, Spokes for Inliers: Uniform Latent Space Construction for Dual-Mismatched Semi-Supervised Learning
Li Yuan, Yaxin Hou, Jiawei Tang, Yongbiao Gao, Yuheng Jia
Semi-supervised learning typically assumes that labeled and unlabeled data share an identical class distribution and label space. However, this setting is often violated: unlabeled data may be imbalanced and contain unknown class samples, causing mismatches in both class distribution and label space. Such dual mismatch leads to majority classes dominating the latent space and unknown class samples being overconfidently misclassified, degrading feature discriminability and pseudo-label quality. To address this, we propose a hub-spoke latent geometry, where known classes are uniformly distributed around a central hub and each class forms compact clusters around its prototype, while the hub provides an anchor for a low-evidence region specifically designed for high-uncertainty unknown class samples. Integrated with an evidence-based classifier, this geometry ultimately enhances feature discriminability and uncertainty separation by mitigating majority-class domination through structured feature organization and guiding high-uncertainty unknown class samples toward the hub. Extensive experiments show that our method outperforms state-of-the-art methods, with a maximum improvement of…
PDF arXiv
arXiv cs.LGOct 5
Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures
Paweł Lorek, Rafał Nowak, Rafał Topolnicki, Tomasz Trzciński, Maciej Zięba
We study estimating rare-event probabilities $I = \mathbb{P}(g(\mathbf{X}) > γ)$ with $\mathbf{X} \sim \mathcal{N}(\boldsymbolμ, \boldsymbolΣ)$ and general $g : \mathbb{R}^d \to \mathbb{R}$. We address this problem through importance sampling, and propose a framework that substantially improves efficiency and robustness over baselines such as crude Monte Carlo, adaptive cross-entropy, variational-inference-based methods (including reverse- and forward-KL approaches), as well as Safe-ICE, Subset Simulation, and Sequential Monte Carlo, drawing on ideas from both rare-event estimation and cross-entropy optimization. The key contribution has two parts: first, we separate the problem into coverage, to overcome the cold-start barrier, and fitting, to refine proposals once a meaningful signal is available; second, we constrain the final GMM proposal so that it has finite importance-sampling variance (since coverage alone is not sufficient -- without safeguards, importance sampling may still suffer from infinite variance). Together, these ingredients yield expressive proposals; finite variance does not by itself guarantee practical stability at a fixed sampling budget. Extensive…
PDF arXiv
arXiv stat.MLOct 5
Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds
Davide Sartor, Meghan E. Huber, Donghyun Kim, Nathan Wycoff
Bayesian Optimization (BO) has become an established methodology for minimizing black-box functions of a vector input. Often, however, this parameter vector arises from the discretization of an inherently functional relationship. Several recent articles have considered the Functional Bayesian Optimization (FBO) setting, in which the variable to be optimized is not a member of a finite dimensional vector space, but rather an infinite dimensional function space. In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients. We discuss in detail the relationship between our method and existing ones, providing a unifying lens through which to view prior works. To assess our method against the state of the art, we conduct an extensive computational study, and along the way develop a novel set of benchmark test functions which port standard finite-dimensional ones to the infinite dimensional domain. Our experiments demonstrate that, on balance, the proposed…
PDF arXiv
arXiv cs.LGOct 5
Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models
Yuta Kobayashi, Divyam Madaan, Shalmali Joshi
Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates epistemic uncertainty (arising from lack of offline data) with aleatoric uncertainty (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.
PDF arXiv

Fields · AI · updated 21m ago

Classifier & causal discovery

Found by the topic model: classifier, causal discovery, bayesian optimization.

87papers this week−13.0% vs last week

All papers · 470 With code · 2

Newest Most relevant

434 papers this month