Can I read Noise-Tolerant Learning for Audio-Visual Action Recognition on EtoBox?

Noise-Tolerant Learning for Audio-Visual Action Recognition by Han, Haochen; Zheng, Qinghua; Luo, Minnan; Miao, Kaiyao; Tian, Feng; Chen, Yan is a scholarly article available to read on EtoBox.

What is Noise-Tolerant Learning for Audio-Visual Action Recognition about?

Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been proposed and offer remarkable recognition results, almost all of these methods rely on high-quality manual annotations and assume that modalities among multi-modal data provide semantically relevant information. Unfortunately, the widely used video datasets are usually coarse-annotated or collected from the Internet. Thus, it inevitably contains a portion of noisy labels and noisy correspondence. To address this challenge, we use the audio-visual action recognition task as a proxy and propose a noise-tolerant learning framework to find anti-interference model parameters against both noisy labels and noisy correspondence. Specifically, our method consists of two phases that aim to rectify noise by the inherent correlation between modalities. First, a noise-tolerant contrastive training phase is performed to make the model immune to the possible noisy-labeled data. To alleviate the influence of noisy correspondence, we propose a cross-modal noise es

Author

Han, Haochen; Zheng, Qinghua; Luo, Minnan; Miao, Kaiyao; Tian, Feng; Chen, Yan

Published

2022

Language

EN