Can I read DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning on EtoBox?

DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning by Wu, Wenhao; Zhao, Yuxiang; Xu, Yanwu; Tan, Xiao; He, Dongliang; Zou, Zhikang; Ye, Jin; Li, Yingying; Yao, Mingde; Dong, Zichao; Shi, Yifeng is a scholarly article available to read on EtoBox.

What is DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning about?

Long-range and short-range temporal modeling are two complementary and crucial aspects of video recognition. Most of the state-of-the-arts focus on short-range spatio-temporal modeling and then average multiple snippet-level predictions to yield the final video-level prediction. Thus, their video-level prediction does not consider spatio-temporal features of how video evolves along the temporal dimension. In this paper, we introduce a novel Dynamic Segment Aggregation (DSA) module to capture relationship among snippets. To be more specific, we attempt to generate a dynamic kernel for a convolutional operation to aggregate long-range temporal information among adjacent snippets adaptively. The DSA module is an efficient plug-and-play module and can be combined with the off-the-shelf clip-based models (i.e., TSM, I3D) to perform powerful long-range modeling with minimal overhead. The final video architecture, coined as DSANet. We conduct extensive experiments on several video recognition benchmarks (i.e., Mini-Kinetics-200, Kinetics-400, Something-Something V1 and ActivityNet) to show its superiority. Our proposed DSA module is shown to benefit various video recognition models signif

Author

Wu, Wenhao; Zhao, Yuxiang; Xu, Yanwu; Tan, Xiao; He, Dongliang; Zou, Zhikang; Ye, Jin; Li, Yingying; Yao, Mingde; Dong, Zichao; Shi, Yifeng

Published

2021

Language

EN