Can I read TRT-ViT: TensorRT-oriented Vision Transformer on EtoBox?

TRT-ViT: TensorRT-oriented Vision Transformer by Xia, Xin; Li, Jiashi; Wu, Jie; Wang, Xing; Xiao, Xuefeng; Zheng, Min; Wang, Rui is a scholarly article available to read on EtoBox.

What is TRT-ViT: TensorRT-oriented Vision Transformer about?

We revisit the existing excellent Transformers from the perspective of practical application. Most of them are not even as efficient as the basic ResNets series and deviate from the realistic deployment scenario. It may be due to the current criterion to measure computation efficiency, such as FLOPs or parameters is one-sided, sub-optimal, and hardware-insensitive. Thus, this paper directly treats the TensorRT latency on the specific hardware as an efficiency metric, which provides more comprehensive feedback involving computational capacity, memory cost, and bandwidth. Based on a series of controlled experiments, this work derives four practical guidelines for TensorRT-oriented and deployment-friendly network design, e.g., early CNN and late Transformer at stage-level, early Transformer and late CNN at block-level. Accordingly, a family of TensortRT-oriented Transformers is presented, abbreviated as TRT-ViT. Extensive experiments demonstrate that TRT-ViT significantly outperforms existing ConvNets and vision Transformers with respect to the latency/accuracy trade-off across diverse visual tasks, e.g., image classification, object detection and semantic segmentation. For example, a

Author

Xia, Xin; Li, Jiashi; Wu, Jie; Wang, Xing; Xiao, Xuefeng; Zheng, Min; Wang, Rui

Published

2022

Language

EN