Papers

Field

Top tools here

Open problems in AI safety

Multilingual evaluation awareness gaps2 papers raise it · 0 address it
Safety evaluations in non-English languages may not reflect real-world behavior due to models acting differently when tested.
- Evaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?raises
- Language Is a Security Surface for Agent Guardsraises
Shortcut sensitivity in agent-safety evaluations1 paper raises it · 0 address it
Evaluation datasets can contain cues that allow guards to predict labels without measuring actual safety, leading to unreliable assessments.
- Shortcut Sensitivity in Agent-Safety Evaluationsraises
Interpretability of personalized model components1 paper raises it · 1 addresses it
Polysemantic neurons in dense LLMs hinder localization and understanding of personalization features, making control difficult.
- Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalizationraises · addresses
Faithful and scalable feature attribution1 paper raises it · 1 addresses it
Existing attribution methods either ignore feature dependencies or do not scale to high-dimensional inputs, limiting causal interpretability.
- Causal Integrated Gradients: Scalable Feature Attribution Along Causal Pathsraises · addresses
Confidence calibration and auditing1 paper raises it · 1 addresses it
Scalar recalibration alone cannot correct all confidence errors, and residual errors may remain undetected without proper auditing.
- Are You Sure Your LLM Is Sure? Repairing and Auditing Confidenceraises · addresses

Named by an AI from these papers' abstracts.

Open problems

Recently funded

Topics in this field

Active contributors · 90 days

Hasan DemirkiranLanguage Is a Security Surface for Agent Guards2
Jens ErnstbergerLanguage Is a Security Surface for Agent Guards2
Advait SinghEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Arya KaranjkarEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Dhruv KumarEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Jagat Sesh ChallaEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1

Active contributors

Activity · 12 weeks

Coming with more sources: repos

Activity

Recent papers

All in Research →

arXiv cs.LGOct 15
Time to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Kajaree Das, Oliver Böse, Subarnaduti Paul, Gabriel Storch, Martin Mundt
Deepfake detectors safeguard against misinformation, rendering their failures as much a societal as a technical concern. As generators constantly evolve, detectors trained on static datasets miss newly arising threats, while progressive training interferes catastrophically. Yet, mitigating forgetfulness in terms of accuracy, the prevalent focus of continual learning (CL), is insufficient to reflect real-world impact. We therefore scrutinize the effects of CL methods with respect to evolving fairness. To this end, we analyze four dimensions of fairness across diverse demographics for detectors trained progressively on a chronological sequence of 11 real-world deepfake generation methods. Tracking fairness metrics over time, we expose non-trivial interplays between performance and fairness that depend on both the choice of CL algorithm and demographic intersections. We find accuracy degradation to typically exacerbate unfairness and, conversely, fairness improvements to remain scarce over time. Dissecting the CL method families (rehearsal, prompts, regularization), we observe that inequitable forgetting is not explained by underrepresentation alone, but likely entangled with…
PDF arXiv
arXiv cs.CLOct 15
Behavioral Forgetting Is Not Privacy in Machine Unlearning
Tyler Lizzo, Larry Heck
Machine unlearning aims to remove targeted information from large language models, but existing evaluations often measure forgetting only on a designated benchmark distribution. We argue that such behavioral forgetting is an insufficient standard for privacy-oriented unlearning, and that evaluation should instead test whether forgetting persists across the access paths through which the same information may be recovered. We demonstrate this with a controlled case study of cross-lingual querying: across multiple multilingual model families, forgetting varies substantially when semantically equivalent information is queried in different languages and scripts. A single access path is therefore enough to expose a real gap between behavioral forgetting and information inaccessibility, motivating broader evaluation of access-path generalization and adversarial recoverability alongside in-distribution forgetting.

Papers

Activity · 12 weeks

Coming with more sources: repos

Fields · AI · updated 6m ago

AI safety

Alignment, interpretability, jailbreaks, red-teaming, robustness, privacy and fairness of AI systems.

47papers this week

All papers · 47 With code · 1

Newest Most relevant

47 papers this month

Active contributors · 90 days

Hasan DemirkiranLanguage Is a Security Surface for Agent Guards2
Jens ErnstbergerLanguage Is a Security Surface for Agent Guards2
Advait SinghEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Arya KaranjkarEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Dhruv KumarEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1
Jagat Sesh ChallaEvaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?1

Recent papers

All in Research →

arXiv cs.LGOct 15
Time to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Kajaree Das, Oliver Böse, Subarnaduti Paul, Gabriel Storch, Martin Mundt
Deepfake detectors safeguard against misinformation, rendering their failures as much a societal as a technical concern. As generators constantly evolve, detectors trained on static datasets miss newly arising threats, while progressive training interferes catastrophically. Yet, mitigating forgetfulness in terms of accuracy, the prevalent focus of continual learning (CL), is insufficient to reflect real-world impact. We therefore scrutinize the effects of CL methods with respect to evolving fairness. To this end, we analyze four dimensions of fairness across diverse demographics for detectors trained progressively on a chronological sequence of 11 real-world deepfake generation methods. Tracking fairness metrics over time, we expose non-trivial interplays between performance and fairness that depend on both the choice of CL algorithm and demographic intersections. We find accuracy degradation to typically exacerbate unfairness and, conversely, fairness improvements to remain scarce over time. Dissecting the CL method families (rehearsal, prompts, regularization), we observe that inequitable forgetting is not explained by underrepresentation alone, but likely entangled with…
PDF arXiv
arXiv cs.CLOct 15
Behavioral Forgetting Is Not Privacy in Machine Unlearning
Tyler Lizzo, Larry Heck
Machine unlearning aims to remove targeted information from large language models, but existing evaluations often measure forgetting only on a designated benchmark distribution. We argue that such behavioral forgetting is an insufficient standard for privacy-oriented unlearning, and that evaluation should instead test whether forgetting persists across the access paths through which the same information may be recovered. We demonstrate this with a controlled case study of cross-lingual querying: across multiple multilingual model families, forgetting varies substantially when semantically equivalent information is queried in different languages and scripts. A single access path is therefore enough to expose a real gap between behavioral forgetting and information inaccessibility, motivating broader evaluation of access-path generalization and adversarial recoverability alongside in-distribution forgetting.

PDF arXiv

arXiv cs.CLOct 15

In the Guise of Correction: How Framing and Cultural Adaptation Erode LLM Safety on Real-World Criminal Cases

Mohsinul Kabir, Md. Tariquzzaman, Karam S. R. Elradie, Polydoros Giannouris, Yuechen Jiang +2

Existing safety benchmarks rely largely on synthetic prompts with loosely grounded taxonomies and offer little focused analysis of severe crime categories. We introduce GUISE, a benchmark of criminal-assistance scenarios sourced from real-world cases and organised under a taxonomy from the UNODC International Classification of Crime for Statistical Purposes (ICCS). Each scenario is rendered under three linguistic framings and evaluated across five distinct cultures. Using our proposed two-tier evaluation framework, which judges both harmfulness and operational uplift, we find that: (i) framing alone drives unsafe compliance, with the mean unsafe compliance rate rising from 28.9% under direct requests to 66.4% under correction seeking (peaking at 97.3%; p < 0.05), as models correcting deliberately distorted details supply executable criminal steps; (ii) culturally adapted prompts elicit significantly more unsafe compliance than literal translations of the same requests in all four non-English languages (+7 to +18 points), a risk that translation-based benchmarks miss; and (iii) unsafe responses from the least-safe models carry strong Anglocentric registers, and the extent of this…

PDF arXiv

arXiv cs.CLOct 15

Bias Mitigation in Audio-Enabled LLMs through Adversarial Training and Distillation Methods

Adnan Al Ali, Jindřich Libovický, Anne Lauscher

Audio-enabled large language models (AuLLMs) allow users to interact with them simply by talking to them, without the need for an intermediate transcription. This expands the model's accessibility and capabilities by encoding non-verbal cues. However, using non-verbal cues may lead to biased behaviour based on the attributes inferred (or assumed) from them. In this work, we focus on the gender attribute and show that models' perception of it can be unlearned using adversarial methods without a loss in performance on other benchmarked tasks. Such models are useful in situations where gender inference is unwanted. However, in other practical settings, gender perception (without bias) may be desired.As a less radical measure, we distil the gender-agnostic model, creating a model that perceives gender but responds similarly to the gender-agnostic teacher, unless explicitly asked.We apply our newly proposed methods to three AuLLMs and obtain satisfactory results in the gender-unlearning part and partially satisfactory results in the distilling part.

PDF arXiv

arXiv cs.LGOct 15

Harmful Partial Interventions and Benign Leakage in Concept-based Models

Gregory Verghese, Christopher J. Soelistyo, Mateo Espinosa Zarlenga, Alessandro Barp, Christopher R. S. Banerji

Artificial intelligence systems deployed in high-risk settings increasingly require models that support meaningful human oversight. Concept-based models (CMs) offer a promising framework by enabling users to inspect and intervene on intermediate, human-understandable concepts. However, concept interventions can counterintuitively worsen downstream predictions, a failure mode we formalise as intervention harm. We distinguish harm after all concepts have been intervened on from harm under partial interventions, the more likely practical use case. Modern CMs often achieve high accuracy by encoding, or leaking, additional task-relevant information in their intermediate concept representations. Recent work argues that such leakage can be benign with respect to intervention harm when concept representations satisfy two conditions: sufficiency and localisation. We prove that these criteria are not sufficient to guarantee that concept interventions improve downstream performance. Across controlled and real-world settings, we find that concept supervision incompleteness is associated with increased intervention harm. Importantly, optimising the model when all concepts are intervened on, a…

PDF arXiv

arXiv cs.LGOct 15

FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning

Reeyansh Shah, Maram Hasan, Subhasis Chaudhuri, Biplab Banerjee

Federated LLM unlearning addresses targeted knowledge removal when forget and retain data remain distributed across private clients and are inaccessible to the server. The resulting locally trained adapters encode conflicting objectives, making their aggregation fundamentally different from conventional federated optimization. A key failure mode is magnitude explosion, whereby unnormalized cross-client aggregation increases the merged update magnitude and degrades retained utility under stronger unlearning. We propose FIRM (Fisher-Informed Retention Merging), a single-round framework for stable and retention-aware server-side aggregation. FIRM uses client-local Fisher statistics as a unified sensitivity signal to construct a Fisher-normalized base merge and impose explicit control over retain-sensitive parameters. Its second-order retention projection admits a unique minimum-distance solution, while the corresponding dual scales with the number of retain clients rather than the model dimension. Experiments across WMDP, TOFU, and MUSE-HP using Zephyr-7B and Llama-3-8B show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU.…

PDF arXiv

Open problems in AI safety

Multilingual evaluation awareness gaps2 papers raise it · 0 address it
Safety evaluations in non-English languages may not reflect real-world behavior due to models acting differently when tested.
- Evaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?raises
- Language Is a Security Surface for Agent Guardsraises
Shortcut sensitivity in agent-safety evaluations1 paper raises it · 0 address it
Evaluation datasets can contain cues that allow guards to predict labels without measuring actual safety, leading to unreliable assessments.
- Shortcut Sensitivity in Agent-Safety Evaluationsraises
Interpretability of personalized model components1 paper raises it · 1 addresses it
Polysemantic neurons in dense LLMs hinder localization and understanding of personalization features, making control difficult.
- Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalizationraises · addresses
Faithful and scalable feature attribution1 paper raises it · 1 addresses it
Existing attribution methods either ignore feature dependencies or do not scale to high-dimensional inputs, limiting causal interpretability.
- Causal Integrated Gradients: Scalable Feature Attribution Along Causal Pathsraises · addresses
Confidence calibration and auditing1 paper raises it · 1 addresses it
Scalar recalibration alone cannot correct all confidence errors, and residual errors may remain undetected without proper auditing.
- Are You Sure Your LLM Is Sure? Repairing and Auditing Confidenceraises · addresses

Named by an AI from these papers' abstracts.