Can I read A Practical Mixed Precision Algorithm for Post-Training Quantization on EtoBox?

A Practical Mixed Precision Algorithm for Post-Training Quantization by Pandey, Nilesh Prasad; Nagel, Markus; van Baalen, Mart; Huang, Yin; Patel, Chirag; Blankevoort, Tijmen is a scholarly article available to read on EtoBox.

What is A Practical Mixed Precision Algorithm for Post-Training Quantization about?

Neural network quantization is frequently used to optimize model size, latency and power consumption for on-device deployment of neural networks. In many cases, a target bit-width is set for an entire network, meaning every layer get quantized to the same number of bits. However, for many networks some layers are significantly more robust to quantization noise than others, leaving an important axis of improvement unused. As many hardware solutions provide multiple different bit-width settings, mixed-precision quantization has emerged as a promising solution to find a better performance-efficiency trade-off than homogeneous quantization. However, most existing mixed precision algorithms are rather difficult to use for practitioners as they require access to the training data, have many hyper-parameters to tune or even depend on end-to-end retraining of the entire model. In this work, we present a simple post-training mixed precision algorithm that only requires a small unlabeled calibration dataset to automatically select suitable bit-widths for each layer for desirable on-device performance. Our algorithm requires no hyper-parameter tuning, is robust to data variation and takes int

Author

Pandey, Nilesh Prasad; Nagel, Markus; van Baalen, Mart; Huang, Yin; Patel, Chirag; Blankevoort, Tijmen

Published

2023

Language

EN