Can I read ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers on EtoBox?

ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers by Jiang, Yanfeng; Sun, Ning; Xie, Xueshuo; Yang, Fei; Li, Tao is a scholarly article available to read on EtoBox.

What is ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers about?

Vision Transformers (ViTs) have exhibited exceptional performance across diverse computer vision tasks, while their substantial parameter size incurs significantly increased memory and computational demands, impeding effective inference on resource-constrained devices. Quantization has emerged as a promising solution to mitigate these challenges, yet existing methods still suffer from significant accuracy loss at low-bit. We attribute this issue to the distinctive distributions of post-LayerNorm and post-GELU activations within ViTs, rendering conventional hardware-friendly quantizers ineffective, particularly in low-bit scenarios. To address this issue, we propose a novel framework called Activation-Distribution-Friendly post-training Quantization for Vision Transformers, ADFQ-ViT. Concretely, we introduce the Per-Patch Outlier-aware Quantizer to tackle irregular outliers in post-LayerNorm activations. This quantizer refines the granularity of the uniform quantizer to a per-patch level while retaining a minimal subset of values exceeding a threshold at full-precision. To handle the non-uniform distributions of post-GELU activations between positive and negative regions, we design

Author

Jiang, Yanfeng; Sun, Ning; Xie, Xueshuo; Yang, Fei; Li, Tao

Published

2024

Language

EN