TodayNewsResearchFieldsStacksMarketAnalytics
⌘K

Research

New papers ranked by attention, with their code, models and discussion linked. Pick a field on the left.

Field

Papers

Open problems

Rising research topics

Ask the research

Sample
ByteTrail · AI and developer-tools intelligence
NewsResearchStacksFor AI agentsAPI
Fields · this week
All fields1917Multimodal327Frontier LLMs246Recursive self improvement207Inference & serving156Agents & indirect prompt injection153Neural operators131Robotics130Clinical reasoning120Forecasting time90Linear bandits66Voice AI59Character recognition OCR33AI safety32Molecular property prediction32Federated learning27Algorithmic fairness7All 23 fields
WeekMonthQuarterAll
NewestMost relevant

Papers

Rising research topics

  • Diffusion priors200.0%
  • Voice AI59−11.9%
  • Character recognition OCR33−17.5%
  • Quantum machine learning23−23.3%
  • Channel state information14−30.0%
  • Algorithmic fairness7−36.4%
  • Forecasting time90−37.9%
  • Recursive self improvement207−38.4%

Papers

  • arXiv cs.LGNov 11

    Robustness Within a Fairness Budget: Parity Frontiers and Equal-Opportunity Reversal

    Pablo Rivas, Jeremy Frank

    Adversarial training protects predictive performance under attack but does not determine how decisions are distributed across protected groups. We formulate binary classification as projected gradient descent (PGD) adversarial-risk minimization under a squared soft demographic-parity (DP) budget, optimized with projected multiplier and one-sided penalty updates. On Adult Income, using the binary sex field supplied by the benchmark for group auditing but excluding it from prediction, we test logistic regression and a one-hidden-layer multilayer perceptron (MLP) across three paired seeds, four PGD strengths, and five budgets. At $\varepsilon=0.3$, PGD training improves balanced accuracy over empirical-risk minimization (ERM) by $7.77$ and $11.45$ percentage points and removes $85.7\%$ and $89.5\%$ of attack-induced growth in hard demographic-parity difference (DPD). At the representative interior value $τ=0.015$, constrained training cuts DPD relative to PGD training by $52.2\%$ and $49.2\%$, while remaining $3.18$ and $8.34$ balanced-accuracy points above ERM. Across all eight seed-averaged model-attack paths, the sampled budget path strictly orders DPD and accuracy and reverses…

    PDF
  • arXiv cs.LGNov 11

    Characterizing Predictive Multiplicity in Ranking and Its Fairness Implications

    Ainhize Barrainkua, Mikel Malagón

    Prediction problems often admit competing models that achieve comparable performance yet produce conflicting outputs, a phenomenon known as predictive multiplicity. While this challenge has been extensively studied in classification tasks, its implications remain largely unexplored in ranking frameworks, which are critical for high-stakes societal decisions such as the allocation of welfare resources and social assistance. This paper investigates the phenomenon of predictive multiplicity in score-based ranking problems. We introduce formal definitions to capture multiplicity in ranking settings, supported by foundational theoretical guarantees. Furthermore, we examine the dual relationship between predictive multiplicity and algorithmic fairness in ranking problems: we analyze how model multiplicity can act as a potential source of unfairness, while demonstrating how exploring this decision space provides a principled avenue for discovering less discriminatory rankings. Finally, we complement our theoretical contributions with insightful empirical evaluations on both synthetic and real-world datasets, validating our analytical findings and illustrating their practical…

    PDF
  • arXiv cs.LGNov 11

    Overcoming the Resolution Limit: Significance-Aware Regularization for Intersectional Fairness

    Antonio Ferrara, André Panisson, Francesco Cozzi, Alan Perotti, Francesco Bonchi

    Intersectional fairness requires evaluating models across fine-grained subgroups. However, as group definitions become more granular, the sample size per subgroup shrinks, causing fairness estimates to become increasingly unstable. Existing in-processing methods are certainty-blind: they penalize observed disparities without distinguishing statistically certified unfairness from sampling noise. This leads to fairness overfitting, where models sacrifice utility to "fix" spurious disparities in sparse regimes. To address this, we introduce the significant-excess metric family, a continuous, uncertainty-aware fairness score that measures only the portion of a subgroup's disparity exceeding what sampling variation can explain at a specific confidence level. Building on this metric, we propose SAFER (Significance-Aware Fairness Regularizer), a differentiable in-processing method that gates fairness penalties using a soft-count likelihood-ratio evidence statistic. SAFER satisfies a provable safety property: when every subgroup disparity lies well below the gate threshold, the unconstrained optimum is nearly stationary for SAFER. At that optimum, the back-propagated contribution of its…

    PDF
  • arXiv stat.MLNov 11

    Synthetic Data Augmentation for Fairness under Data Imbalance: Theory and Adaptive Allocation

    Zhengchi Ma, Kaifeng Zhang, Anru R. Zhang

    Data imbalance across sensitive groups and class labels is common in classification problems and can contribute to substantial fairness disparities. We study how such imbalance affects Equal Opportunity and how synthetic data augmentation can mitigate the resulting unfairness. We first identify two mechanisms, prevalence imbalance and group-size imbalance, through which changes in the training distribution shift the learned decision boundary and induce disparities in group true positive rates. Motivated by this analysis, we study exact full-balance augmentation, which equalizes all group-label cells. We establish conditions under which full balancing strictly improves Equal Opportunity together with a prediction-error comparison under the oracle balanced distribution, and show that the Equal Opportunity gap of the synthetic-data classifier converges to that of the oracle balanced classifier as the sample size grows and the generator error vanishes. The finite-sample convergence rate agrees, up to logarithmic factors and generator-approximation error, with the information-theoretic lower-bound rate under the corresponding regularity regimes. Since exact balancing need not provide…

    PDF
  • arXiv stat.MLNov 10

    Robbing Peter to Pay Paul: Cross-Attribute Externalities of Fairness Interventions

    Fernandes Machado Agathe, Jamotton Charlotte, Charpentier Arthur

    Fairness interventions often targets disparities associated with a single sensitive attribute in binary classification tasks. However, when multiple sensitive attributes are associated with a data subject, improving fairness with respect to one attribute does not necessarily improve it across all attributes: it may exacerbate disparities associated with other sensitive attributes, including those that are unavailable to the practitioner. We refer to these unintended effects as cross-attribute externalities of fairness. We study how fairness with respect to one sensitive attribute varies when an initially unfair classification or regression model is corrected with respect to another attribute. We characterize these externalities using Wasserstein barycentric disparities and develop a distributional framework that captures changes in both marginal and joint fairness measures. This highlights a limitation of conventional fairness evaluation, in which the success of an intervention is assessed mainly for the attribute it targets. In a jointly Gaussian model, we derive explicit trajectories for marginal and joint disparities and identify conditions under which an intervention…

    PDF
  • arXiv stat.MLNov 9

    Hidden Tails: Certifying Tail-Risk Claims under Selective Labels

    El Mustapha Mansouri

    Historical decisions determine which outcomes are observed, potentially concealing a predictor's largest errors. Even unlimited data from the same observation process can therefore leave population tail risk unresolved. Here we derive sharp bounds on conditional value-at-risk, the average loss in a specified upper tail, for binary outcomes and bounded losses. These bounds span exactly the risks compatible with the observations and a stated limit on how outcome odds differ between observed and hidden cases at the same covariate values. Under stated regularity conditions, calibrated statistical certificates cover these ranges simultaneously across candidate predictors, supporting risk limits and model comparisons while separating statistical uncertainty from persistent ambiguity. For a fixed predictor and loss threshold, we derive a minimum-cost allocation for reducing excess-loss ambiguity by prioritizing hidden outcomes according to ambiguity removed per unit cost. This allocation treats audited population contributions as known and retains the original sensitivity bounds separately for audited and still-hidden outcomes. In controlled masking experiments with known loan…

    PDF
  • arXiv cs.LGNov 9

    Explanation-Aware Fairness Beyond Output Parity: Joint Output-Attribution Regularization for Medical Image Classification

    Gökhan Özbulak, Oscar Jimenez-del-Toro, Lilian Berton, André Anjos

    Output-level fairness does not constrain whether demographic groups rely on similar predictive contributions in representation space. For a nonlinear encoder with a linear head, we show that outcome-conditioned output score disparity equals the signed sum of feature attribution differences across subgroups by allowing small prediction gaps to arise through cancellation effect. We test whether an attribution disparity penalty complements output-level equalized odds based fairness surrogates. Across 960 DINOv2-LoRA models on HAM10000 and Harvard-GF datasets for gender and race fairness, attribution augmentation increases utility-fairness trade-off performance for hypervolume in six different settings with paired 95% confidence intervals and relative gains of 1.49-10.57%. Under a predefined utility constraint, augmented model families also achieve 9.0-38.7% lower equalized odds difference, lower attribution disparity, and reduced cancellation ratio while preserving predictive utility. These results support attribution fairness as a complementary explanation-aware constraint on output fairness.

    PDF
  • arXiv cs.LGNov 6

    A Taxonomy-Driven Synthetic Dataset for Retrieval Effectiveness and Fairness Evaluation in Hiring

    Warren Jouanneau, Morgane Hoffmann, Emma Jouffroy, Marc Palyart

    While machine learning hiring tools are ubiquitous, the need for privacy regulations and commercial sensitivity makes public benchmarking legally complex. We address this challenge by introducing a taxonomy-driven, fully synthetic HR dataset and benchmark to simultaneously evaluate retrieval effectiveness and algorithmic fairness. Grounded in 660K real freelancer profiles, our multilingual corpus features candidate profiles, project briefs, and counterfactual demographic twins. Auditing state-of-the-art embedding models reveals performance divergence from general MTEB scores, alongside fairness disparities in grammatically gendered languages. This demonstrates that general benchmarks are insufficient for HR, requiring tailored evaluations to identify Pareto-optimal models that balance semantic accuracy and fairness.

    PDF
  • arXiv cs.AINov 5

    The Matthews Correlation Coefficient: Fifty years

    Jesus S. Aguilar-Ruiz

    Fifty years after its introduction, the Matthews Correlation Coefficient (MCC) remains central -and debated- in binary classification. This survey offers a comprehensive retrospective and an in-depth analysis of MCC, covering its historical development, mathematical foundations, and implications for practice. We clarify long-standing misconceptions, formalize its equivalences (to Pearson's correlation and to Chi2 in 2x2 tables), and examine its behavior under class imbalance. Beyond MCC, we develop analytical (derivatives) and graphical (contour plots) frameworks that generalize to any confusion-matrix-based metric. These frameworks expose structural properties -bias, robustness, sensitivity- that scalar summaries conceal, and they provide actionable guidance for selecting and interpreting performance measures in applications where reliability and fairness matter. Together, our results position MCC within a broader toolkit while clarifying when it is informative -and when alternatives may be preferable.

    PDF
  • arXiv cs.LGAI safetyNov 5

    Reweighting is not Harmless: Understanding the Role of Dataset Characteristics

    Amalia Ionescu, Jose G. Hernandez, Emily F. Wong, Anil K. Saini, Jason H. Moore +1

    Reweighting is a widely used pre-processing method for promoting discrimination-free model behavior by assigning different weights to data points during model training. While it is often credited with improving fairness, particularly by mitigating systematic underestimation for underrepresented groups, it can also introduce unintended harms under certain data conditions. Using controlled simulations that vary group size and baseline event prevalence, we show how reweighting can yield misleading improvements in fairness by reducing underestimation for one group while introducing it for another. Across 3,000 simulated datasets with event prevalence of 7-14%, reweighting improved parity in positive classification and false negative rates (FNR) between groups, but did so partly by decreasing positive classifications and increasing FNR for the group with higher baseline prevalence. These effects were most pronounced when that group was a minority: its FNR rose by 6 percentage points, approximately 82 additional missed cases per 10,000 group members, and roughly three-quarters of the improvement in FNR parity came from this increase rather than from gains for the other group. The…

    AI70PDF
  • arXiv cs.LGAI safetyNov 4

    Practical Estimation of the Bayes Optimal Fairness-Accuracy Tradeoff with Soft Labels

    Mohit Sharma, Okan Koç, Amit Deshpande, Takashi Ishida, Gang Niu +2

    There is a fundamental limit to the best prediction error of any model for a given data distribution. In classification, the Bayes error quantifies this limit and serves as both a benchmark for trained models and a criterion to detect overfitting. The state-of-the-art Bayes error estimation techniques for binary classification use only soft labels (positive class probabilities) and require no input features or auxiliary training. We extend such notions to constrained classification problems, such as fairness. Fair classification is an important case of constrained or multi-objective classification. It requires a model not only to be accurate but also to yield fair outcomes (equal or near-equal true positive rates) across different demographic groups (e.g., race and gender). This leads to an inherent fairness-accuracy tradeoff or a Pareto frontier that is used to compare fair classifiers against each other. The focus of our work is to estimate the fundamental limit for this fairness-accuracy tradeoff. We derive an explicit expression for the fair Bayes error rate (i.e., best achievable error rate subject to given fairness constraints), and construct consistent, instance-free…

    AI40PDF
  • arXiv cs.LGAI safetyNov 3

    Fairness Metrics based on the AUC in Clinical Predictive AI

    João Matos, Ben Van Calster, Richard D. Riley, Paula Dhiman, Gary S. Collins

    Fairness evaluation of clinical prediction models for binary outcomes often involves fairness metrics that compare the area under the receiver operating characteristic curve (AUC, or c-statistic) across subgroups. However, the AUC has been shown to be a non-collapsible metric, meaning that its subgroup specific and overall values are not directly comparable as often implicitly assumed. This raises important questions about how existing AUC-based fairness metrics should be used and interpreted. We examine eight AUC-based fairness metrics proposed in the literature, including: within-group AUC parity; cross-group AUC (xAUC) parity; gap-to-overall AUC; equity-scaled AUC; minimum-based metrics; $\mathrm{xAUC}^1$; $\mathrm{xAUC}^0$; and $\mathrm{xAUC}^1$ or $\mathrm{xAUC}^0$ parity. We characterise their estimands, relate them to the decomposition of overall AUC (into within- and cross-group components), and critically appraise them. We then illustrate their behaviour using simulations based on the EuroSCORE II, a clinical prediction model for in-hospital mortality. We recommend reporting an AUC matrix, containing the overall AUC and all within- and cross-group AUC components,…

    AI60PDF
  • arXiv cs.AIAI safetyNov 3

    Fast, Cheap, and Fair? Evaluating Decision Models

    Muhammad Uzair Ul Haq, Davide Rigoni, Alessandro Sperduti, Flavia Fabris

    Industries are increasingly adopting AI-driven systems to automate a wide range of tasks, including high-stakes decisions in human resources, such as candidate screening and recruitment. This makes evaluating their fairness essential, particularly when decisions directly affect individuals' opportunities. Accordingly, the EU AI Act classifies recruitment AI systems as high-risk, imposing strict requirements on their deployment. In this context, compact ``System~1'' decision models offer a fast and cost-efficient alternative to general-purpose LLMs, yet their fairness remains largely unexplored. This article presents the first systematic bias and fairness audit of \jev{}, a typed-decision model targeting recruitment applications. The study evaluates how explicit demographic attributes and implicit candidate cues influence screening scores, rankings, and shortlisting outcomes. These effects are compared against a noise floor derived from name substitutions and typographical variations. The evaluation also examines output validity, latency, and cost. Counterfactual CV datasets and evaluation code are released for reproducibility.

    AIEvals80PDF
  • arXiv cs.LGAI safetyNov 2

    Rethinking Fairness Metrics for Multi-Class and Multi-Group Assessment: Not Everything is Binary

    Matteo Grazioso, Fabio Naccarella, Chiara Gallese, Marco S. Nobile

    Artificial Intelligence, and Machine Learning in general, is becoming pervasive across all domains and disciplines. By means of this data-driven approach, algorithms can learns complex task just by learning from samples. However, if the data-set is biased, they might lead to the development of unfair computer systems, that is, characterised by a prejudice towards specific individuals or groups, which might get an unmotivated output from the system. This circumstance lead to the development of multiple metrics designed to quantify the level of unfairness of an algorithm. However, a generalised assumption in the literature is that the sensitive features and the output labels are binary. When the data is not binary, it is forcefully binarized using a one-versus-the-rest simplification. In this work, we discuss the limitations of existing metrics and propose novel, mathematically sound, and generalized multi-group and multi-class formulations of the metrics. Ultimately, our results highlight the empirical assessment distortion caused by binarization strategies, which reveals the limitations of binary fairness metrics in the multi class setting.

    AI60PDF
  • arXiv cs.SEAgentsNov 2

    Specification-Driven Governance for Fairness-by-Design in Agentic Software Engineering: How Far Can We Go?

    Alfonso Cannavale, Alessandra Parziale, Valeria Pontillo, Patrizio Pelliccione, Andrea De Lucia +1

    Agentic AI is shifting developers' responsibilities from software implementation to software governance. While this paradigm improves productivity, it also raises challenges in ensuring that autonomous coding agents preserve developers' intentions. Among these, operationalizing fairness-by-design is largely unexplored, as existing approaches provide limited support for representing and propagating fairness requirements throughout agentic development workflows. In this paper, we investigate whether specification-driven development can operationalize fairness-by-design in the agentic development of machine learning systems. We study three coding agents on four machine learning applications, using four prompts that progressively operationalize fairness through nine Fairness Specification Elements, and analyze the traceability of fairness requirements across the generated artifacts, the fairness-accuracy trade-off of the resulting models, and the agents' design decisions and rationales. The results show that when fairness is operationalized, agents capture almost all fairness requirements in the specification (97.22%) and preserve most of them in the code (82.41%). However, the…

    AIAI safety70PDF
  • arXiv cs.AINov 1

    Fairness-Aware Design for Contextual Experiments: Guaranteeing Reliability and Equity in Heterogeneous Subgroups

    Guangyan Gan, Ling Zhang, Yanhua Cheng, Yongxiang Tang, Kaiyuan Li +2

    Experimental design is critical for evidence-based decision-making in healthcare, marketing, and public policy. However, designing efficient experiments across heterogeneous subgroups presents significant challenges. Existing methods often optimize for statistical power or overall sample efficiency, overlooking crucial fairness considerations across these different subgroups. To address this gap, we introduce a Fairness-Aware Contextual Track-and-Stop Design (F-CTSD) algorithm. The proposed F-CTSD algorithm provides statistical guarantees on subgroup fairness while minimizing required sample sizes. We quantify the fairness-efficiency trade-off and derive the exact sample complexity for the proposed F-CTSD algorithm under its fairness constraints. We further theoretically prove that the proposed F-CTSD algorithm consistently produces accurate treatment effect estimates even under fairness requirements, enhancing statistical reliability. Numerical experiments show that the proposed F-CTSD algorithm outperforms existing methods, achieving higher sample efficiency while reducing subgroup fairness violations by 4.95%.

    AI20PDF
  • arXiv cs.LGAI safetyNov 1

    Yielding Fairness: Coupled Learning and Fairness for Value-Weighted Ranking

    Ami Tavory, Yaniv Sheena, Shaked Bar, Yair Zagdansky

    Risk screening and recommendation systems pass or block accounts that generate both value and loss. Under a blocked-value budget, a cap on expected value forgone through blocking, the optimal ranking orders accounts by expected value per unit expected loss, a ratio of conditional expectations. This rule can conflict with accepted harm or net-harm criteria, blocking accounts that another criterion would pass at the same budget. We call these cases account-level objections and define group complaint disparity as the difference between groups' normalized realized objections. Weighted binary cross-entropy (W-BCE) implements the value-per-loss order with one score. Objection penalties modify the ranking, while group multipliers scale those penalties differently by group, tracing tradeoffs between loss capture and complaint disparity. This preserves the value-budgeted ranking structure and makes the cost of reducing contestable blocks explicit. Across public recommendation, insurance, credit, and mortgage datasets, we compare undamped W-BCE with ordinary BCE using yield Gini, an area summary of loss capture versus blocked value. Logarithmic weight damping is evaluated separately.…

    AI50PDF
  • arXiv stat.MLAI safetyNov 1

    Surrogate Losses for Fairness-Aware Classification: Fisher Consistency and Calibration

    Changcheng Song, Yi Yang, Xiangyu Chang, Xiaoling Lu, Yufeng Liu

    Fairness-aware classification seeks to balance predictive accuracy and fairness across demographic groups. Although surrogate losses facilitate the optimization of fairness-aware objectives, it remains unclear when minimizing these losses yields optimal decisions under the original fairness-aware risk. This paper investigates the statistical validity of surrogate-based fairness-aware classification under demographic parity and equal opportunity, considering both attribute-blind and attribute-aware settings. We introduce fairness-aware Fisher consistency and characterize it through conditions on the derivatives of the surrogate losses. The characterization shows that consistency depends not only on the individual loss functions but also on their interaction within the fairness-regularized objective. We further establish a calibration inequality that relates the excess fairness-aware risk to the excess surrogate risk, providing a quantitative connection between surrogate optimization and the original classification objective. Numerical experiments illustrate how deviations from the consistency conditions affect fairness-aware excess risk and how the regularization parameter…

    AI50PDF
  • arXiv cs.LGAI safetyOct 30

    A Framework for Task-Agnostic Counterfactual Fairness in Transformers through Activation Steering

    Joshua Wolff Anderson, Yanshan Wang, Shyam Visweswaran

    Bias in transformer-based models is a growing concern as these architectures become more prevalent in automated decision-making systems, such as prediction models for hiring, lending, and clinical treatment recommendation. Mechanistic interpretability (MI) is a growing body of work that measures meaningful causal effects of activations on model output by identifying human-interpretable features encoded in model activations. When such features correspond to sensitive attributes, they provide a direct way to quantify the bias those attributes introduce and to causally intervene to mitigate it. However, counterfactual fairness (CF) methods have not been systematically adapted to this setting of intervening on interpretable mechanisms in transformers. We propose a framework, Mechanistic Interpretability for Counterfactual Fairness (MICF), that utilizes MI tools to reformulate CF constraints in terms of transformer activations, unifying former approaches into a single pipeline for fairness analysis in transformers. MICF does not require a prior causal model or specific transformer architecture. We evaluate MICF on real-world tabular data and show that CF can be improved across…

    AI70PDF
  • arXiv cs.LGAI safetyOct 30

    KINDER: Kernel-based Independence for Fair Representation Learning via Prototype-space Erasure

    Abtin Mogharabin, Alp Toykan Kaplan, Jiaee Cheong, Sinan Kalkan

    Concept erasure is a prominent approach to achieving fairness in machine learning through the removal of sensitive attributes from learned representations. Prior concept erasure methods typically define debiasing indirectly through the failure of a chosen adversary, probe, or fairness penalty, which makes the target of erasure dependent on a particular decoder family or optimization setup. We introduce KINDER, which directly removes sensitive-attribute information from intermediate representations with respect to a chosen family of prototype functionals, and is compatible with any deep model containing an intermediate feature space, independent of the architecture, task type, or training paradigm. KINDER operates in a random Fourier feature space, where it estimates a sensitive subspace from protected group prototypes and projects representations onto its orthogonal complement. Thus, it forms an explicit representation cleaning mechanism distinct from prior fairness control studies. We conduct extensive experiments across diverse settings and show the effectiveness of KINDER on (i) supervised and self-supervised settings, (ii) classification, regression, and image segmentation…

    AI60PDF
Load more
Research · AI field

Algorithmic fairness

Found by the topic model: algorithmic fairness.

Open the field page
7papers this week−36.4% vs last week39 this month

Open problems in Algorithmic fairness

Open problems for Algorithmic fairness are being prepared; they appear within a day.