Can I read Towards an Interpretable Latent Space in Structured Models for Video Prediction on EtoBox?

Towards an Interpretable Latent Space in Structured Models for Video Prediction by Gupta, Rushil; Sharma, Vishal; Jain, Yash; Liang, Yitao; Broeck, Guy Van den; Singla, Parag is a scholarly article available to read on EtoBox.

What is Towards an Interpretable Latent Space in Structured Models for Video Prediction about?

We focus on the task of future frame prediction in video governed by underlying physical dynamics. We work with models which are object-centric, i.e., explicitly work with object representations, and propagate a loss in the latent space. Specifically, our research builds on recent work by Kipf et al. \cite{kipf&al20}, which predicts the next state via contrastive learning of object interactions in a latent space using a Graph Neural Network. We argue that injecting explicit inductive bias in the model, in form of general physical laws, can help not only make the model more interpretable, but also improve the overall prediction of model. As a natural by-product, our model can learn feature maps which closely resemble actual object positions in the image, without having any explicit supervision about the object positions at the training time. In comparison with earlier works \cite{jaques&al20}, which assume a complete knowledge of the dynamics governing the motion in the form of a physics engine, we rely only on the knowledge of general physical laws, such as, world consists of objects, which have position and velocity. We propose an additional decoder based loss in the pixel space,

Author

Gupta, Rushil; Sharma, Vishal; Jain, Yash; Liang, Yitao; Broeck, Guy Van den; Singla, Parag

Published

2021

Language

EN