Opening book details…
Can I read AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition on EtoBox?
AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition by Wang, Yulin; Yue, Yang; Xu, Xinhong; Hassani, Ali; Kulikov, Victor; Orlov, Nikita; Song, Shiji; Shi, Humphrey; Huang, Gao is a scholarly article available to read on EtoBox.
What is AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition about?
Recent research has revealed that reducing the temporal and spatial redundancy are both effective approaches towards efficient video recognition, e.g., allocating the majority of computation to a task-relevant subset of frames or the most valuable image regions of each frame. However, in most existing works, either type of redundancy is typically modeled with another absent. This paper explores the unified formulation of spatial-temporal dynamic computation on top of the recently proposed AdaFocusV2 algorithm, contributing to an improved AdaFocusV3 framework. Our method reduces the computational cost by activating the expensive high-capacity network only on some small but informative 3D video cubes. These cubes are cropped from the space formed by frame height, width, and video duration, while their locations are adaptively determined with a light-weighted policy network on a per-sample basis. At test time, the number of the cubes corresponding to each video is dynamically configured, i.e., video cubes are processed sequentially until a sufficiently reliable prediction is produced. Notably, AdaFocusV3 can be effectively trained by approximating the non-differentiable cropping opera
- Author
- Wang, Yulin; Yue, Yang; Xu, Xinhong; Hassani, Ali; Kulikov, Victor; Orlov, Nikita; Song, Shiji; Shi, Humphrey; Huang, Gao
- Published
- 2022
- Language
- EN