Can I read Explaining the Impact of Training on Vision Models via Activation Clustering on EtoBox?

Explaining the Impact of Training on Vision Models via Activation Clustering by Boubekki, Ahcène; Fadel, Samuel G.; Mair, Sebastian is a scholarly article available to read on EtoBox.

What is Explaining the Impact of Training on Vision Models via Activation Clustering about?

This paper introduces Neuro-Activated Vision Explanations (NAVE), a method for extracting and visualizing the internal representations of vision model encoders. By clustering feature activations, NAVE provides insights into learned semantics without fine-tuning. Using object localization, we show that NAVE's concepts align with image semantics. Through extensive experiments, we analyze the impact of training strategies and architectures on encoder representation capabilities. Additionally, we apply NAVE to study training artifacts in vision transformers and reveal how weak training strategies and spurious correlations degrade model performance. Our findings establish NAVE as a valuable tool for post-hoc model inspection and improving transparency in vision models.

Author

Boubekki, Ahcène; Fadel, Samuel G.; Mair, Sebastian

Published

2024

Language

EN