Can I read Introducing Gating and Context into Temporal Action Detection on EtoBox?

Introducing Gating and Context into Temporal Action Detection by Reka, Aglind; Borza, Diana Laura; Reilly, Dominick; Balazia, Michal; Bremond, Francois is a scholarly article available to read on EtoBox.

What is Introducing Gating and Context into Temporal Action Detection about?

Temporal Action Detection (TAD), the task of localizing and classifying actions in untrimmed video, remains challenging due to action overlaps and variable action durations. Recent findings suggest that TAD performance is dependent on the structural design of transformers rather than on the self-attention mechanism. Building on this insight, we propose a refined feature extraction process through lightweight, yet effective operations. First, we employ a local branch that employs parallel convolutions with varying window sizes to capture both fine-grained and coarse-grained temporal features. This branch incorporates a gating mechanism to select the most relevant features. Second, we introduce a context branch that uses boundary frames as key-value pairs to analyze their relationship with the central frame through cross-attention. The proposed method captures temporal dependencies and improves contextual understanding. Evaluations of the gating mechanism and context branch on challenging datasets (THUMOS14 and EPIC-KITCHEN 100) show a consistent improvement over the baseline and existing methods.

Author

Reka, Aglind; Borza, Diana Laura; Reilly, Dominick; Balazia, Michal; Bremond, Francois

Published

2024

Language

EN