Can I read Latent Point Collapse on a Low Dimensional Embedding in Deep Neural Network Classifiers on EtoBox?

Latent Point Collapse on a Low Dimensional Embedding in Deep Neural Network Classifiers by Sbailò, Luigi; Ghiringhelli, Luca is a scholarly article available to read on EtoBox.

What is Latent Point Collapse on a Low Dimensional Embedding in Deep Neural Network Classifiers about?

The configuration of latent representations plays a critical role in determining the performance of deep neural network classifiers. In particular, the emergence of well-separated class embeddings in the latent space has been shown to improve both generalization and robustness. In this paper, we propose a method to induce the collapse of latent representations belonging to the same class into a single point, which enhances class separability in the latent space while enforcing Lipschitz continuity in the network. We demonstrate that this phenomenon, which we call \textit{latent point collapse}, is achieved by adding a strong $L_2$ penalty on the penultimate-layer representations and is the result of a push-pull tension developed with the cross-entropy loss function. In addition, we show the practical utility of applying this compressing loss term to the latent representations of a low-dimensional linear penultimate layer. The proposed approach is straightforward to implement and yields substantial improvements in discriminative feature embeddings, along with remarkable gains in robustness to input perturbations.

Author

Sbailò, Luigi; Ghiringhelli, Luca

Published

2023

Language

EN