Can I read STIP: A SpatioTemporal Information-Preserving and Perception-Augmented Model for High-Resolution Video Prediction on EtoBox?

STIP: A SpatioTemporal Information-Preserving and Perception-Augmented Model for High-Resolution Video Prediction by Chang, Zheng; Zhang, Xinfeng; Wang, Shanshe; Ma, Siwei; Gao, Wen is a scholarly article available to read on EtoBox.

What is STIP: A SpatioTemporal Information-Preserving and Perception-Augmented Model for High-Resolution Video Prediction about?

Although significant achievements have been achieved by recurrent neural network (RNN) based video prediction methods, their performance in datasets with high resolutions is still far from satisfactory because of the information loss problem and the perception-insensitive mean square error (MSE) based loss functions. In this paper, we propose a Spatiotemporal Information-Preserving and Perception-Augmented Model (STIP) to solve the above two problems. To solve the information loss problem, the proposed model aims to preserve the spatiotemporal information for videos during the feature extraction and the state transitions, respectively. Firstly, a Multi-Grained Spatiotemporal Auto-Encoder (MGST-AE) is designed based on the X-Net structure. The proposed MGST-AE can help the decoders recall multi-grained information from the encoders in both the temporal and spatial domains. In this way, more spatiotemporal information can be preserved during the feature extraction for high-resolution videos. Secondly, a Spatiotemporal Gated Recurrent Unit (STGRU) is designed based on the standard Gated Recurrent Unit (GRU) structure, which can efficiently preserve spatiotemporal information during th

Author

Chang, Zheng; Zhang, Xinfeng; Wang, Shanshe; Ma, Siwei; Gao, Wen

Published

2022

Language

EN