Can I read Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network on EtoBox?

Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network by Chen, Jin; Song, Huihui; Zhang, Kaihua; Liu, Bo; Liu, Qingshan is a scholarly article available to read on EtoBox.

What is Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network about?

Due to a variety of motions across different frames, it is highly challenging to learn an effective spatiotemporal representation for accurate video saliency prediction (VSP). To address this issue, we develop an effective spatiotemporal feature alignment network tailored to VSP, mainly including two key sub-networks: a multi-scale deformable convolutional alignment network (MDAN) and a bidirectional convolutional Long Short-Term Memory (Bi-ConvLSTM) network. The MDAN learns to align the features of the neighboring frames to the reference one in a coarse-to-fine manner, which can well handle various motions. Specifically, the MDAN owns a pyramidal feature hierarchy structure that first leverages deformable convolution (Dconv) to align the lower-resolution features across frames, and then aggregates the aligned features to align the higher-resolution features, progressively enhancing the features from top to bottom. The output of MDAN is then fed into the Bi-ConvLSTM for further enhancement, which captures the useful long-time temporal information along forward and backward timing directions to effectively guide attention orientation shift prediction under complex scene transformati

Author

Chen, Jin; Song, Huihui; Zhang, Kaihua; Liu, Bo; Liu, Qingshan

Published

2020

Language

EN