Can I read NeurIPS 2024 How Jepa Avoids Noisy Features The Implicit Bias of Deep Linear Self Distillation Networks Paper Conference on EtoBox?

NeurIPS 2024 How Jepa Avoids Noisy Features The Implicit Bias of Deep Linear Self Distillation Networks Paper Conference by fanbyprinciple is a document available to read on EtoBox.

What is NeurIPS 2024 How Jepa Avoids Noisy Features The Implicit Bias of Deep Linear Self Distillation Networks Paper Conference about?

This document discusses the Joint Embedding Predictive Architecture (JEPA) and its advantages over the Masked AutoEncoder (MAE) in self-supervised learning. It reveals that JEPA has an implicit bias towards learning high-influence features, which allows it to focus on semantically rich subspaces rather than just reconstructing input data. The authors analyze the training dynamics of deep linear models to demonstrate these differences and their implications for representation learning.

Author

fanbyprinciple

Language

EN