Can I read Profile-guided Memory Optimization for Deep Neural Networks on EtoBox?

Profile-guided Memory Optimization for Deep Neural Networks by Sekiyama, Taro; Imamichi, Takashi; Imai, Haruki; Raymond, Rudy is a scholarly article available to read on EtoBox.

What is Profile-guided Memory Optimization for Deep Neural Networks about?

Recent years have seen deep neural networks (DNNs) becoming wider and deeper to achieve better performance in many applications of AI. Such DNNs however require huge amounts of memory to store weights and intermediate results (e.g., activations, feature maps, etc.) in propagation. This requirement makes it difficult to run the DNNs on devices with limited, hard-to-extend memory, degrades the running time performance, and restricts the design of network models. We address this challenge by developing a novel profile-guided memory optimization to efficiently and quickly allocate memory blocks during the propagation in DNNs. The optimization utilizes a simple and fast heuristic algorithm based on the two-dimensional rectangle packing problem. Experimenting with well-known neural network models, we confirm that our method not only reduces the memory consumption by up to $49.5\%$ but also accelerates training and inference by up to a factor of four thanks to the rapidity of the memory allocation and the ability to use larger mini-batch sizes.

Author

Sekiyama, Taro; Imamichi, Takashi; Imai, Haruki; Raymond, Rudy

Published

2018

Language

EN