Can I read MobileInst: Video Instance Segmentation on the Mobile on EtoBox?

MobileInst: Video Instance Segmentation on the Mobile by Zhang, Renhong; Cheng, Tianheng; Yang, Shusheng; Jiang, Haoyi; Zhang, Shuai; Lyu, Jiancheng; Li, Xin; Ying, Xiaowen; Gao, Dashan; Liu, Wenyu; Wang, Xinggang is a scholarly article available to read on EtoBox.

What is MobileInst: Video Instance Segmentation on the Mobile about?

Video instance segmentation on mobile devices is an important yet very challenging edge AI problem. It mainly suffers from (1) heavy computation and memory costs for frame-by-frame pixel-level instance perception and (2) complicated heuristics for tracking objects. To address those issues, we present MobileInst, a lightweight and mobile-friendly framework for video instance segmentation on mobile devices. Firstly, MobileInst adopts a mobile vision transformer to extract multi-level semantic features and presents an efficient query-based dual-transformer instance decoder for mask kernels and a semantic-enhanced mask decoder to generate instance segmentation per frame. Secondly, MobileInst exploits simple yet effective kernel reuse and kernel association to track objects for video instance segmentation. Further, we propose temporal query passing to enhance the tracking ability for kernels. We conduct experiments on COCO and YouTube-VIS datasets to demonstrate the superiority of MobileInst and evaluate the inference latency on one single CPU core of Snapdragon 778G Mobile Platform, without other methods of acceleration. On the COCO dataset, MobileInst achieves 31.2 mask AP and 433 ms

Author

Zhang, Renhong; Cheng, Tianheng; Yang, Shusheng; Jiang, Haoyi; Zhang, Shuai; Lyu, Jiancheng; Li, Xin; Ying, Xiaowen; Gao, Dashan; Liu, Wenyu; Wang, Xinggang

Published

2023

Language

EN