A Theoretical Analysis of Environment-Aware Out-of-Distribution Detection for Vision-Language Models
Continual vision-language models (VLMs) encounter a stream of environments and must also reject inputs that do not belong to the environment at hand. Class information from earlier environments can aid out-of-distribution (OOD) detection, yet existing detectors score each environment in isolation, and when previous environments help or hurt is theoretically unclear. We develop a theory of environment-aware OOD detection built on contribution scores, pairwise cosine expectations measuring how the image and text features of one environment align with those of another. We propose a cross-environment maximum-concept-matching (MCM) score with two terms: a primary term matching the test image against the current class texts, and a secondary term matching it against the class texts of a previous environment, weighted by the intra-modal alignment between the two environments. We prove that, under a positive environment transition and a bounded class-matching margin for OOD inputs, this score attains a false-positive rate no larger than that of the standard MCM detector at an adjusted threshold. We then show that the score gap of an MCM detector between two environments equals the…
Theoretical analysis of OOD detection for vision-language models
- arXiv
- 2610.27211
- Authors
- Salimeh Sekeh, Mary Wisell
- Categories
- cs.LG