Can I read Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units on EtoBox?

Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units by Xu, Yixi; Wang, Xiao is a scholarly article available to read on EtoBox.

What is Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units about?

This paper presents a general framework for norm-based capacity control for $L_{p,q}$ weight normalized deep neural networks. We establish the upper bound on the Rademacher complexities of this family. With an $L_{p,q}$ normalization where $q\le p^*$, and $1/p+1/p^{*}=1$, we discuss properties of a width-independent capacity control, which only depends on depth by a square root term. We further analyze the approximation properties of $L_{p,q}$ weight normalized deep neural networks. In particular, for an $L_{1,\infty}$ weight normalized network, the approximation error can be controlled by the $L_1$ norm of the output layer, and the corresponding generalization error only depends on the architecture by the square root of the depth.

Author

Xu, Yixi; Wang, Xiao

Published

2018

Language

EN