Can I read Self-Supervised Equivariant Scene Synthesis from Video on EtoBox?

Self-Supervised Equivariant Scene Synthesis from Video by Resnick, Cinjon; Litany, Or; Heiß, Cosmas; Larochelle, Hugo; Bruna, Joan; Cho, Kyunghyun is a scholarly article available to read on EtoBox.

What is Self-Supervised Equivariant Scene Synthesis from Video about?

We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equivariant with respect to their transformation across frames and the background being constant with respect to that same transformation. After training, we can manipulate image encodings in real time to create unseen combinations of the delineated components. As far as we know, we are the first method to perform unsupervised extraction and synthesis of interpretable background, character, and animation. We demonstrate results on three datasets: Moving MNIST with backgrounds, 2D video game sprites, and Fashion Modeling.

Author

Resnick, Cinjon; Litany, Or; Heiß, Cosmas; Larochelle, Hugo; Bruna, Joan; Cho, Kyunghyun

Published

2021

Language

EN