Papers

Field

Top tools here

Open problems in Classifier & causal discovery

Benchmarking agentic black-box optimization1 paper raises it · 1 addresses it
Lack of standardized cross-domain benchmarks makes it hard to compare methods and isolate design choices in agentic BBO.
- A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimizationraises · addresses
Identifying individual world latents without reconstruction1 paper raises it · 1 addresses it
Existing methods fail to recover individual latents without reconstruction, decoders, or labels, limiting interpretability and causal analysis.
- DSReg: Provably Recovering Individual World Latents without Reconstructionraises · addresses
Long-tailed recognition with instance-wise confusion1 paper raises it · 2 address it
Class-wise logit adjustments ignore instance-level confusion, harming minority class performance in imbalanced datasets.
- CILA: Confusion-based Instance-wise Logit Adjustment for Long-Tailed Recognitionraises · addresses
- Using Set Learning to Improve Set-Valued Classifiers on Long-tailed Dataaddresses
Data augmentation under label noise1 paper raises it · 1 addresses it
Theoretical understanding of when augmentation helps or harms generalization in noisy-label settings is missing.
- Understanding Data Augmentation under Label Noiseraises · addresses
Causal attribution with scarce incident data1 paper raises it · 1 addresses it
Exact attribution of alarm-score changes in causal systems is combinatorial and requires incident data, limiting practicality.
- What Moved the Alarm Score? Causal Attribution with Exact Computation and Scarce-Incident Estimationraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

classifier · supervised · machine learning183 papers42 this week
causal discovery · causal inference · causal models123 papers18 this week
conformal prediction · split conformal prediction · aware conformal71 papers14 this week
bayesian optimization · bayesian optimization bo · black box optimization93 papers12 this week

Topics in this field

Active contributors · 90 days

Yujia ZhengDSReg: Provably Recovering Individual World Latents without Reconstruction4
Gunwoong ParkLocal Fisher Information Enables Sparse Causal Discovery4
Aapo HyvärinenIdentifying Dependent Components from Multi-domain Linear Mixtures3
Sara MagliacaneIdentifying Dependent Components from Multi-domain Linear Mixtures3
Stefan FeuerriegelDebiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference3
Isabel ValeraDeDiCa: A Scalable Deconfounding Discrete Causal Generative Model3

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv stat.MLOct 5
Assumption-lean logistic regression with missing covariates
Jyotishka Ray Choudhury, Kabir Aladin Verchand, Richard J. Samworth, Ashwin Pananjady
Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully chosen imputation schemes for missing data, or likelihood approximations that lead to nonconvex $M$-estimation problems. These methods and their relatives are suitable for scenarios in which the covariate distribution is known, and more broadly, have enjoyed tremendous success in linear models. But even in basic nonlinear problems such as logistic regression in moderate dimensions, such methods can experience drastic failure modes when the covariate distribution is unknown. Motivated by the need for reliable alternatives, we consider the problem of parameter estimation in logistic regression with missing covariates. Crucially, we operate in the assumption-lean setting where the covariate distribution is unknown (but bounded). We design a stochastic approximation method that is based on $Z$-estimation with a novel monotone operator, and establish that our algorithm is computationally efficient and achieves provable signal recovery at parametric rates under the hypothesis that covariates are missing completely at random. Our theory…
PDF arXiv
arXiv cs.AIOct 5
BiasFlow: Geometric Monitoring and Backbone Regularization for Spurious Feature Reliance
Haojin Deng, Zhiping Lin, Yimin Yang
Worst-group accuracy (WGA) evaluates a trained predictor but does not characterize how its frozen backbone behaves when a new head is learned. We introduce BiasFlow, a hook-based toolkit for monitoring class-attribute centroid alignment (IBMI), within-class centroid separation (W-IBMI), and feature-projection sensitivity. IBMI is confounded by class-attribute correlation and is not a measure of causal feature reliance. We pair these diagnostics with BiasFlow Regularization (BFR), a supervised, composable class-conditional centroid-alignment penalty. W-IBMI verifies the quantity BFR optimizes; it is scale dependent and does not independently establish attribute removal. Across the reported small-scale benchmarks, adding BFR improves or preserves mean WGA, with gains up to +26.0 pp on UrbanCars. The principal independent stress test freezes CelebA-Std backbones and trains fresh heads on biased data: BFR+GroupDRO improves WGA from 40.7% to 64.1%, while Male probe accuracy decreases from 92.5% to 72.2%. Attribute information remains recoverable, and cross-task results are mixed. A controlled synthetic-watermark ImageNet experiment additionally improves watermark-shift accuracy by…
PDF arXiv
arXiv cs.LGOct 5
Block Disentanglement in CRL: Bridging Identifiability and Visual State Estimation
Emre Acartürk, Pranamya Kulkarni, Puranjay Datta, Karthikeyan Shanmugam, Burak Varıcı +1
Causal representation learning (CRL) is the process of recovering causally-related latent variables from high-dimensional observations. As a label-free inference method, CRL is particularly attractive for applications where data labels are unavailable or impractical to obtain. While there has been significant progress in understanding the identifiability guarantees of CRL, such guarantees often hold under highly stylized assumptions, which temper the direct application to real-world problems. This paper has a two-fold objective for interventional CRL. First, it establishes identifiability guarantees for substantially weaker interventional assumptions, resulting in block disentanglement of the causal variables, where the block structure depends on the realistically available intervention mechanisms. Secondly, the block disentanglement framework is used for embodied visual state estimation, in which the objective is to recover the latent physical variables of a robotic system directly from visual data (images and videos) without labeled data. These two components are critically complementary. The block disentanglement theory delineates identifiability guarantees under weakened…
PDF arXiv
arXiv cs.LGOct 5
MatrixFormer: A Foundation Model for Matrix Completion
Dwaipayan Saha, Jacob Feitelberg, Kyuseong Choi, Raaz Dwivedi, Anish Agarwal
Matrix completion underlies problems from tabular imputation to causal inference, yet existing tabular foundation models treat it as entry-by-entry prediction, repeating context for every target and discarding the matrix's two-dimensional structure. We introduce MatrixFormer, a pre-trained matrix-native transformer that predicts a full distribution for every missing entry in a single forward pass. MatrixFormer is trained entirely on synthetic low-rank and latent-factor matrices under diverse missingness patterns. Applied zero-shot and with the same model weights, MatrixFormer achieves competitive performance on causal inference panel-data tasks, language-model benchmark-score completion, tabular imputation, and recommendation systems matrix completion. These results position MatrixFormer as a general-purpose foundation model for matrix completion.
PDF arXiv
arXiv cs.LGOct 5
HeuFouFT: Task-Guided Metaheuristic Coordinate Search for Fourier Fine-Tuning
Ruiheng Wang, Yubo Hou, Yakun Zhu, Tianle Shen, Tao Wan +1
We introduce Heuristic-Guided Fourier Fine-Tuning (HeuFouFT), a task-guided framework for selecting trainable frequency coordinates in Fourier fine-tuning. Existing uniform and Gaussian band-pass schemes allocate a limited spectral budget through fixed, task-agnostic rules. HeuFouFT instead searches for coordinates using downstream performance. A coarse intensity map from lightweight block-level probes initializes three metaheuristic optimizers: Genetic Algorithm with Simulated Annealing (GA-SA), Particle Swarm Optimization (PSO), and Cuckoo Search (CS). During search, a Random Forest filters each population so that only the top 30% of candidates proceed to proxy fine-tuning. On E2E with GPT-2-Medium, all three variants outperform random-uniform FourierFT, Gaussian band-pass FourierFT, and LoRA across five metrics. PSO further outperforms LoCA, the best-performing baseline, on four metrics while using 37.6% fewer trainable spectral coefficients. Once coordinates are selected, HeuFouFT requires only 15--18% FLOPs of Full FT. These results show that task-guided search allocates limited spectral capacity more effectively than fixed sampling. Our code is publicly available.
PDF arXiv
arXiv stat.MLOct 5
Valid Stopping in Adaptive Generator-Verifier Loops
Mahmoud Hegazy, Michael I. Jordan, Aymeric Dieuleveut
Numerous agentic workflows are based on a generator-verifier loop: a generator proposes candidates, a cheap verifier scores them, and the workflow terminates when a proposal is verified as good enough. The verifier typically proxies a more costly ground-truth oracle, and as the generator searches adaptively against it, false acceptances may accumulate. Proposals can pass the proxy but fail under the costlier ground-truth check. We study when to stop these loops while controlling the false discovery rate of the accepted proposals. Our construction introduces tools of independent interest in distribution-free statistical testing and conformal risk control, including analysis of $e$-values constructed through index betting and a novel conformal risk control procedure for non-monotone losses. We validate the approach in synthetic settings and on a protein-design benchmark.
PDF arXiv

Papers

Topics in this field

classifier · supervised · machine learning183 papers42 this week
causal discovery · causal inference · causal models123 papers18 this week
conformal prediction · split conformal prediction · aware conformal71 papers14 this week
bayesian optimization · bayesian optimization bo · black box optimization93 papers12 this week

Active contributors · 90 days

Yujia ZhengDSReg: Provably Recovering Individual World Latents without Reconstruction4
Gunwoong ParkLocal Fisher Information Enables Sparse Causal Discovery4
Aapo HyvärinenIdentifying Dependent Components from Multi-domain Linear Mixtures3
Sara MagliacaneIdentifying Dependent Components from Multi-domain Linear Mixtures3
Stefan FeuerriegelDebiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference3
Isabel ValeraDeDiCa: A Scalable Deconfounding Discrete Causal Generative Model3

Recent papers

All in Research →

arXiv stat.MLOct 5
Assumption-lean logistic regression with missing covariates
Jyotishka Ray Choudhury, Kabir Aladin Verchand, Richard J. Samworth, Ashwin Pananjady
Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully chosen imputation schemes for missing data, or likelihood approximations that lead to nonconvex $M$-estimation problems. These methods and their relatives are suitable for scenarios in which the covariate distribution is known, and more broadly, have enjoyed tremendous success in linear models. But even in basic nonlinear problems such as logistic regression in moderate dimensions, such methods can experience drastic failure modes when the covariate distribution is unknown. Motivated by the need for reliable alternatives, we consider the problem of parameter estimation in logistic regression with missing covariates. Crucially, we operate in the assumption-lean setting where the covariate distribution is unknown (but bounded). We design a stochastic approximation method that is based on $Z$-estimation with a novel monotone operator, and establish that our algorithm is computationally efficient and achieves provable signal recovery at parametric rates under the hypothesis that covariates are missing completely at random. Our theory…
PDF arXiv
arXiv cs.AIOct 5
BiasFlow: Geometric Monitoring and Backbone Regularization for Spurious Feature Reliance
Haojin Deng, Zhiping Lin, Yimin Yang
Worst-group accuracy (WGA) evaluates a trained predictor but does not characterize how its frozen backbone behaves when a new head is learned. We introduce BiasFlow, a hook-based toolkit for monitoring class-attribute centroid alignment (IBMI), within-class centroid separation (W-IBMI), and feature-projection sensitivity. IBMI is confounded by class-attribute correlation and is not a measure of causal feature reliance. We pair these diagnostics with BiasFlow Regularization (BFR), a supervised, composable class-conditional centroid-alignment penalty. W-IBMI verifies the quantity BFR optimizes; it is scale dependent and does not independently establish attribute removal. Across the reported small-scale benchmarks, adding BFR improves or preserves mean WGA, with gains up to +26.0 pp on UrbanCars. The principal independent stress test freezes CelebA-Std backbones and trains fresh heads on biased data: BFR+GroupDRO improves WGA from 40.7% to 64.1%, while Male probe accuracy decreases from 92.5% to 72.2%. Attribute information remains recoverable, and cross-task results are mixed. A controlled synthetic-watermark ImageNet experiment additionally improves watermark-shift accuracy by…
PDF arXiv
arXiv cs.LGOct 5
Block Disentanglement in CRL: Bridging Identifiability and Visual State Estimation
Emre Acartürk, Pranamya Kulkarni, Puranjay Datta, Karthikeyan Shanmugam, Burak Varıcı +1
Causal representation learning (CRL) is the process of recovering causally-related latent variables from high-dimensional observations. As a label-free inference method, CRL is particularly attractive for applications where data labels are unavailable or impractical to obtain. While there has been significant progress in understanding the identifiability guarantees of CRL, such guarantees often hold under highly stylized assumptions, which temper the direct application to real-world problems. This paper has a two-fold objective for interventional CRL. First, it establishes identifiability guarantees for substantially weaker interventional assumptions, resulting in block disentanglement of the causal variables, where the block structure depends on the realistically available intervention mechanisms. Secondly, the block disentanglement framework is used for embodied visual state estimation, in which the objective is to recover the latent physical variables of a robotic system directly from visual data (images and videos) without labeled data. These two components are critically complementary. The block disentanglement theory delineates identifiability guarantees under weakened…
PDF arXiv
arXiv cs.LGOct 5
MatrixFormer: A Foundation Model for Matrix Completion
Dwaipayan Saha, Jacob Feitelberg, Kyuseong Choi, Raaz Dwivedi, Anish Agarwal
Matrix completion underlies problems from tabular imputation to causal inference, yet existing tabular foundation models treat it as entry-by-entry prediction, repeating context for every target and discarding the matrix's two-dimensional structure. We introduce MatrixFormer, a pre-trained matrix-native transformer that predicts a full distribution for every missing entry in a single forward pass. MatrixFormer is trained entirely on synthetic low-rank and latent-factor matrices under diverse missingness patterns. Applied zero-shot and with the same model weights, MatrixFormer achieves competitive performance on causal inference panel-data tasks, language-model benchmark-score completion, tabular imputation, and recommendation systems matrix completion. These results position MatrixFormer as a general-purpose foundation model for matrix completion.
PDF arXiv
arXiv cs.LGOct 5
HeuFouFT: Task-Guided Metaheuristic Coordinate Search for Fourier Fine-Tuning
Ruiheng Wang, Yubo Hou, Yakun Zhu, Tianle Shen, Tao Wan +1
We introduce Heuristic-Guided Fourier Fine-Tuning (HeuFouFT), a task-guided framework for selecting trainable frequency coordinates in Fourier fine-tuning. Existing uniform and Gaussian band-pass schemes allocate a limited spectral budget through fixed, task-agnostic rules. HeuFouFT instead searches for coordinates using downstream performance. A coarse intensity map from lightweight block-level probes initializes three metaheuristic optimizers: Genetic Algorithm with Simulated Annealing (GA-SA), Particle Swarm Optimization (PSO), and Cuckoo Search (CS). During search, a Random Forest filters each population so that only the top 30% of candidates proceed to proxy fine-tuning. On E2E with GPT-2-Medium, all three variants outperform random-uniform FourierFT, Gaussian band-pass FourierFT, and LoRA across five metrics. PSO further outperforms LoCA, the best-performing baseline, on four metrics while using 37.6% fewer trainable spectral coefficients. Once coordinates are selected, HeuFouFT requires only 15--18% FLOPs of Full FT. These results show that task-guided search allocates limited spectral capacity more effectively than fixed sampling. Our code is publicly available.
PDF arXiv
arXiv stat.MLOct 5
Valid Stopping in Adaptive Generator-Verifier Loops
Mahmoud Hegazy, Michael I. Jordan, Aymeric Dieuleveut
Numerous agentic workflows are based on a generator-verifier loop: a generator proposes candidates, a cheap verifier scores them, and the workflow terminates when a proposal is verified as good enough. The verifier typically proxies a more costly ground-truth oracle, and as the generator searches adaptively against it, false acceptances may accumulate. Proposals can pass the proxy but fail under the costlier ground-truth check. We study when to stop these loops while controlling the false discovery rate of the accepted proposals. Our construction introduces tools of independent interest in distribution-free statistical testing and conformal risk control, including analysis of $e$-values constructed through index betting and a novel conformal risk control procedure for non-monotone losses. We validate the approach in synthetic settings and on a protein-design benchmark.
PDF arXiv

Activity · 12 weeks

Coming with more sources: repos

Open problems in Classifier & causal discovery

Benchmarking agentic black-box optimization1 paper raises it · 1 addresses it
Lack of standardized cross-domain benchmarks makes it hard to compare methods and isolate design choices in agentic BBO.
- A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimizationraises · addresses
Identifying individual world latents without reconstruction1 paper raises it · 1 addresses it
Existing methods fail to recover individual latents without reconstruction, decoders, or labels, limiting interpretability and causal analysis.
- DSReg: Provably Recovering Individual World Latents without Reconstructionraises · addresses
Long-tailed recognition with instance-wise confusion1 paper raises it · 2 address it
Class-wise logit adjustments ignore instance-level confusion, harming minority class performance in imbalanced datasets.
- CILA: Confusion-based Instance-wise Logit Adjustment for Long-Tailed Recognitionraises · addresses
- Using Set Learning to Improve Set-Valued Classifiers on Long-tailed Dataaddresses
Data augmentation under label noise1 paper raises it · 1 addresses it
Theoretical understanding of when augmentation helps or harms generalization in noisy-label settings is missing.
- Understanding Data Augmentation under Label Noiseraises · addresses
Causal attribution with scarce incident data1 paper raises it · 1 addresses it
Exact attribution of alarm-score changes in causal systems is combinatorial and requires incident data, limiting practicality.
- What Moved the Alarm Score? Causal Attribution with Exact Computation and Scarce-Incident Estimationraises · addresses

Named by an AI from these papers' abstracts.

Fields · AI · updated 39m ago

Classifier & causal discovery

Found by the topic model: classifier, causal discovery, bayesian optimization.

86papers this week−14.9% vs last week

All papers · 470 With code · 2

Newest Most relevant

434 papers this month