Can I read Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets on EtoBox?

Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets by Dave, Ishan Rajendrakumar; Heilbron, Fabian Caba; Shah, Mubarak; Jenni, Simon is a scholarly article available to read on EtoBox.

What is Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets about?

Temporal video alignment aims to synchronize the key events like object interactions or action phase transitions in two videos. Such methods could benefit various video editing, processing, and understanding tasks. However, existing approaches operate under the restrictive assumption that a suitable video pair for alignment is given, significantly limiting their broader applicability. To address this, we re-pose temporal alignment as a search problem and introduce the task of Alignable Video Retrieval (AVR). Given a query video, our approach can identify well-alignable videos from a large collection of clips and temporally synchronize them to the query. To achieve this, we make three key contributions: 1) we introduce DRAQ, a video alignability indicator to identify and re-rank the best alignable video from a set of candidates; 2) we propose an effective and generalizable frame-level video feature design to improve the alignment performance of several off-the-shelf feature representations, and 3) we propose a novel benchmark and evaluation protocol for AVR using cycle-consistency metrics. Our experiments on 3 datasets, including large-scale Kinetics700, demonstrate the effectivenes

Author

Dave, Ishan Rajendrakumar; Heilbron, Fabian Caba; Shah, Mubarak; Jenni, Simon

Published

2024

Language

EN