About this document

Deep Neural Regression Collapse by arlanden is a document available to read on EtoBox.

This paper establishes the phenomenon of Neural Regression Collapse (NRC) in deep regression models, demonstrating that it occurs not only at the last layer but also in earlier layers. The authors propose conditions for NRC that include noise suppression, signal-target alignment, feature-weight alignment, and linear predictability, showing that models exhibiting NRC can learn the intrinsic dimension of low rank targets. Additionally, the study explores the role of weight decay in facilitating NRC, providing

Author

arlanden

Language

EN