Can I read Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks on EtoBox?

Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks by Haas, Jarrod; Yolland, William; Rabus, Bernhard is a scholarly article available to read on EtoBox.

What is Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks about?

We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.

Author

Haas, Jarrod; Yolland, William; Rabus, Bernhard

Published

2022

Language

EN