About this document
Optimizer by TECH TUBE is a document available to read on EtoBox.
The document discusses various optimization techniques in machine learning, highlighting the limitations of Stochastic Gradient Descent (SGD) and introducing alternatives like Exponentially Weighted Momentum, Nesterov Momentum, and ADAM. It also addresses the challenges of first-order methods in high-dimensional spaces and the potential benefits of second-order optimization, despite its computational complexity. Additionally, it covers Xavier initialization as a method to prevent vanishing gradients in deep
- Author
- TECH TUBE
- Language
- EN