About this document

Adaptive SGD Learning Rate Optimization by kien.lt203474 is a document available to read on EtoBox.

This document introduces AdaS, an adaptive scheduling algorithm for stochastic gradient descent (SGD) optimization of deep neural networks. AdaS uses new metrics called "knowledge gain" and "mapping condition" to measure the quality of training in convolutional layers. Knowledge gain quantifies how useful gradients are for updates, and adapts the SGD learning rate accordingly. In experiments, AdaS exhibited faster convergence and better generalization than existing adaptive learning rate methods, without re

Author

kien.lt203474

Language

EN