---
title: "How a Model's Architecture and... | SI Paper Desk"
url: https://thesimind.si/papers/2611.12075/
description: "In this study of image models, explanation quality changed with both training progress and architecture: ResNet50 and ConvNeXt-Tiny gave more faithful..."
---

Number 5 of 12 · November 11, 2026

# How a Model's Architecture and Performance Influences Explainability: A Study of Grad-CAM and CRAFT in CNN-Based Image Classification

In this study of image models, explanation quality changed with both training progress and architecture: ResNet50 and ConvNeXt-Tiny gave more faithful, steadier explanations over training, while VGG16 gave simpler but less faithful ones.

Kilian Göller, Haadia Amjad, Steffen Seitz and 2 more · Submitted November 9, 2026 · arXiv 2611.12075v1 · cs.CV

[Read the paper on arXiv](https://arxiv.org/abs/2611.12075) · [The PDF](https://arxiv.org/pdf/2611.12075) · [The day's 12](https://thesimind.si/papers/day/2026-11-11/)

## What they found

Göller and colleagues trained three image classifiers (VGG16, ResNet50 and ConvNeXt-Tiny) on 10 subsets of ImageNet-1K, each with 80 classes. For each model they saved 10 checkpoints, which are snapshots at rising accuracy. They ran two explanation methods on each checkpoint: Grad-CAM, which highlights the image areas that mattered, and CRAFT, which finds concepts the model uses. They scored the explanations on Faithfulness, Correctness, Sensitivity and Simplicity. As training went on, ResNet50 and ConvNeXt-Tiny showed better Faithfulness and lower Sensitivity, which the authors read as more robust explanations. VGG16 gave simpler explanations that were less faithful and less correct. The authors call this an initial step, and the abstract gives no detailed numbers.

## Why it matters

Explanation tools are meant to build trust in areas like healthcare and autonomous systems. This study suggests an explanation's quality is not fixed. It can depend on how complex the model is and how well it has trained. Anyone who uses these tools should check them on the specific model, not assume they work the same everywhere.

## How it changes the race

This is a small, incremental study of older-style image models and does not change the race for frontier SI. It adds evidence that explanation methods should be tested alongside model design choices.

## Safety and control

The paper bears on transparency and accountability. Its results suggest that how reliable an explanation is can vary with model architecture and training stage, so explanations should not be trusted blindly in high-risk uses. The abstract only covers CNN image classifiers.

### The abstract, by the authors

Deep learning models, particularly convolutional neural networks (CNNs), are widely adopted in computer vision, yet their opaque decision-making raises concerns in high-risk applications like healthcare and autonomous systems, where trust and accountability are critical. Therefore, robust explainable AI (XAI) methods are desirable to provide transparent insights into model decisions. To ensure these methods are reliable and effective across diverse scenarios, it is important to understand how model performance and architectural complexity influence explanation quality. Our study takes an initial step toward systematically investigating the impact of these two factors on key XAI evaluation metrics. We constructed a pipeline using 10 subsets of ImageNet-1K, each containing 80 unique classes, on each of which we trained three CNN architectures of increasing structural complexity: VGG16, ResNet50, and ConvNeXt-Tiny. For each individual model, we analyzed 10 training checkpoints representing increasing levels of model accuracy using two XAI methods: Grad-CAM (feature-attribution-based) and CRAFT (concept-based). We assessed these methods across four evaluation categories: Faithfulness, Correctness, Sensitivity, and Simplicity. Our findings reveal that as training progresses, the complex architectures (ResNet50 and ConvNeXt-Tiny) exhibit improved Faithfulness and lower Sensitivity, indicating more robust explanations. In contrast, VGG16's simpler architecture yields simpler but less faithful and correct explanations.

Quoted from arXiv:2611.12075. The research and its words belong to its authors.

Desk score **82** out of 100

Why it was picked

- Frontier lab: xAI
- Interpretability (in the title)
- Agents
- Evals
- Policy
- Cross-listed in 2 fields

Topics

- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Vision](https://thesimind.si/papers/?tag=vision)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)

**arXiv ID:** 2611.12075v1
**Submitted:** November 9, 2026
**Fields:** cs.CV (Computer Vision), cs.AI (Artificial Intelligence)
**Level:** Intermediate
**Comments:** Accepted at the 4th World Conference on eXplainable Artificial Intelligence (xAI 2026), Fortaleza, Brazil, July 1-3, 2026. To appear in Springer CCIS. 12 pages, 5 figures

Fingerprint, SHA-256

`87392a47e81931d5c4510abfaefa4e212dabba8530f08eb515c1f6a65960fc76`

On Solana: [Proof of Mind seal, ipj4pc...z6te](https://solscan.io/tx/ipj4pcQsmFjnBmF5dFmUSA15Eex18bkxb83FjQuvHnS4DCz6yJoWcESM8bwWT8cMMh3f4GmDjbebLEAsauqz6te)

### The exact text fingerprinted

{"arxiv":"2611.12075","version":1,"title":"How a Model's Architecture and Performance Influences Explainability: A Study of Grad-CAM and CRAFT in CNN-Based Image Classification","authors":["Kilian Göller","Haadia Amjad","Steffen Seitz","Carsten Knoll","Ronald Tetzlaff"],"day":"2026-11-11","rank":5,"score":82,"status":"written","summary":{"takeaway":"In this study of image models, explanation quality changed with both training progress and architecture: ResNet50 and ConvNeXt-Tiny gave more faithful, steadier explanations over training, while VGG16 gave simpler but less faithful ones.","found":"Göller and colleagues trained three image classifiers (VGG16, ResNet50 and ConvNeXt-Tiny) on 10 subsets of ImageNet-1K, each with 80 classes. For each model they saved 10 checkpoints, which are snapshots at rising accuracy. They ran two explanation methods on each checkpoint: Grad-CAM, which highlights the image areas that mattered, and CRAFT, which finds concepts the model uses. They scored the explanations on Faithfulness, Correctness, Sensitivity and Simplicity. As training went on, ResNet50 and ConvNeXt-Tiny showed better Faithfulness and lower Sensitivity, which the authors read as more robust explanations. VGG16 gave simpler explanations that were less faithful and less correct. The authors call this an initial step, and the abstract gives no detailed numbers.","matters":"Explanation tools are meant to build trust in areas like healthcare and autonomous systems. This study suggests an explanation's quality is not fixed. It can depend on how complex the model is and how well it has trained. Anyone who uses these tools should check them on the specific model, not assume they work the same everywhere.","race":"This is a small, incremental study of older-style image models and does not change the race for frontier SI. It adds evidence that explanation methods should be tested alongside model design choices.","safety":"The paper bears on transparency and accountability. Its results suggest that how reliable an explanation is can vary with model architecture and training stage, so explanations should not be trusted blindly in high-risk uses. The abstract only covers CNN image classifiers.","level":"intermediate","tags":["interpretability","evals","vision","safety","benchmarks"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Kilian Göller, Haadia Amjad, Steffen Seitz, Carsten Knoll, Ronald Tetzlaff

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/) · [Nov 9, 2026 · Paper of the DayPhantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-MakingWhen SI models see ambiguous public signals from other models, they can drift to a group answer that no one privately held, and checking private judgments can catch this.Multi-agent systemsEvalsBenchmarks](https://thesimind.si/papers/2611.08971/) · [Nov 9, 2026 · Number 4HomeLoop: A Simulation Framework and Benchmark for Robot-IoT Coordination in Everyday Domestic Assistance and SafetyHomeLoop is a new simulated home benchmark where SI robots must work with smart devices and handle ambiguous alerts. The best of six models scored 61.9%, and models often over-reacted.RoboticsBenchmarksEvals](https://thesimind.si/papers/2611.08914/)

[Number 4](https://thesimind.si/papers/2611.12079/) · [All 12 for Nov 11, 2026](https://thesimind.si/papers/day/2026-11-11/) · [Number 6](https://thesimind.si/papers/2611.11714/)