Can I read Neural Network Overparameterization Effects on EtoBox?

Neural Network Overparameterization Effects by No12n533 is a document available to read on EtoBox.

What is Neural Network Overparameterization Effects about?

This document summarizes a research paper that studies how overparameterization, or increasing the number of parameters beyond the number of training examples, affects the convergence of stochastic gradient descent (SGD) when training neural networks. The paper introduces the concept of "gradient confusion," which is when gradients from different training examples are negatively correlated, slowing convergence. It finds that overparameterization through increasing the width of neural network layers leads to

Author

No12n533

Language

EN