How Robust Are CNNs and Vision Transformers for Alzheimer's MRI Classification Across Datasets?

Hafija Aktar, Molly Croes, Frank S. Menniti, Abdeltawab Hendawi

Accurate assessment of Alzheimer's disease stages from MRI is important for understanding disease progression and supporting the development of computer-aided analysis tools. Deep learning models have shown promising results for Alzheimer's disease classification from MRI. However, when evaluated on single datasets, it is difficult to determine whether the observed performance reflects consistent architectural behavior or dataset-specific effects. In this work, we present a multi-dataset benchmarking framework with standardized preprocessing and evaluation. We compare five architectures: ResNet-50, ReXNet-150, Swin Transformer, ViT-Base, and ViT-Large. The models are evaluated across three MRI datasets representing severe class imbalance, train-test distribution mismatch, and large-scale augmentation. The results show broadly consistent performance patterns across datasets, with ViT-Large achieving the best performance on all three datasets and reaching up to 99.84% accuracy and 99.86% macro F1. Performance differences between architectures are most pronounced under distribution mismatch and considerably smaller in the augmented-data setting. Across datasets, lower class-level…