What they found
Göller and colleagues trained three image classifiers (VGG16, ResNet50 and ConvNeXt-Tiny) on 10 subsets of ImageNet-1K, each with 80 classes. For each model they saved 10 checkpoints, which are snapshots at rising accuracy. They ran two explanation methods on each checkpoint: Grad-CAM, which highlights the image areas that mattered, and CRAFT, which finds concepts the model uses. They scored the explanations on Faithfulness, Correctness, Sensitivity and Simplicity. As training went on, ResNet50 and ConvNeXt-Tiny showed better Faithfulness and lower Sensitivity, which the authors read as more robust explanations. VGG16 gave simpler explanations that were less faithful and less correct. The authors call this an initial step, and the abstract gives no detailed numbers.
Why it matters
Explanation tools are meant to build trust in areas like healthcare and autonomous systems. This study suggests an explanation's quality is not fixed. It can depend on how complex the model is and how well it has trained. Anyone who uses these tools should check them on the specific model, not assume they work the same everywhere.
How it changes the race
This is a small, incremental study of older-style image models and does not change the race for frontier SI. It adds evidence that explanation methods should be tested alongside model design choices.
Safety and control
The paper bears on transparency and accountability. Its results suggest that how reliable an explanation is can vary with model architecture and training stage, so explanations should not be trusted blindly in high-risk uses. The abstract only covers CNN image classifiers.
The abstract, by the authors
Deep learning models, particularly convolutional neural networks (CNNs), are widely adopted in computer vision, yet their opaque decision-making raises concerns in high-risk applications like healthcare and autonomous systems, where trust and accountability are critical. Therefore, robust explainable AI (XAI) methods are desirable to provide transparent insights into model decisions. To ensure these methods are reliable and effective across diverse scenarios, it is important to understand how model performance and architectural complexity influence explanation quality. Our study takes an initial step toward systematically investigating the impact of these two factors on key XAI evaluation metrics. We constructed a pipeline using 10 subsets of ImageNet-1K, each containing 80 unique classes, on each of which we trained three CNN architectures of increasing structural complexity: VGG16, ResNet50, and ConvNeXt-Tiny. For each individual model, we analyzed 10 training checkpoints representing increasing levels of model accuracy using two XAI methods: Grad-CAM (feature-attribution-based) and CRAFT (concept-based). We assessed these methods across four evaluation categories: Faithfulness, Correctness, Sensitivity, and Simplicity. Our findings reveal that as training progresses, the complex architectures (ResNet50 and ConvNeXt-Tiny) exhibit improved Faithfulness and lower Sensitivity, indicating more robust explanations. In contrast, VGG16's simpler architecture yields simpler but less faithful and correct explanations.
Quoted from arXiv:2611.12075. The research and its words belong to its authors.