Can I read Performance characterization and optimization of pruning patterns for sparse DNN inference on EtoBox?

Performance characterization and optimization of pruning patterns for sparse DNN inference by Yunjie Liu; Jingwei Sun; Jiaqiang Liu; Guangzhong Sun is a Computer Science article available to read on EtoBox.

What is Performance characterization and optimization of pruning patterns for sparse DNN inference about?

Deep neural networks are suffering from over parameterized high storage and high consumption problems. Pruning can effectively reduce storage and computation costs of deep neural networks by eliminating their redundant parameters. In existing pruning methods, filter pruning achieves more efficient inference, while element-wise pruning maintains better accuracy. To make a trade-off between the two endpoints, a variety of pruning patterns has been proposed. This study analyzes the performance characteristics of sparse DNNs pruned by different patterns, including element-wise, vector-wise, block-wise, and group-wise. Based on the analysis, we propose an efficient implementation of group-wise sparse DNN inference, which can make better use of GPUs. Experimental results on VGG, ResNet, BERT and ViT show that our optimized group-wise pruning pattern achieves much lower inference latency on GPU than other sparse patterns and the existing group-wise pattern implementation.

Who reads Performance characterization and optimization of pruning patterns for sparse DNN inference?

Author

Yunjie Liu; Jingwei Sun; Jiaqiang Liu; Guangzhong Sun

Publisher

Elsevier BV

Published

2023

Language

EN

Field

Computer Science (Physical Sciences)