Can I read Robust Active Speaker Detection in Noisy Environments on EtoBox?

Robust Active Speaker Detection in Noisy Environments by Vasireddy, Siva Sai Nagender; Zhang, Chenxu; Guo, Xiaohu; Tian, Yapeng is a scholarly article available to read on EtoBox.

What is Robust Active Speaker Detection in Noisy Environments about?

This paper addresses the issue of active speaker detection (ASD) in noisy environments and formulates a robust active speaker detection (rASD) problem. Existing ASD approaches leverage both audio and visual modalities, but non-speech sounds in the surrounding environment can negatively impact performance. To overcome this, we propose a novel framework that utilizes audio-visual speech separation as guidance to learn noise-free audio features. These features are then utilized in an ASD model, and both tasks are jointly optimized in an end-to-end framework. Our proposed framework mitigates residual noise and audio quality reduction issues that can occur in a naive cascaded two-stage framework that directly uses separated speech for ASD, and enables the two tasks to be optimized simultaneously. To further enhance the robustness of the audio features and handle inherent speech noises, we propose a dynamic weighted loss approach to train the speech separator. We also collected a real-world noise audio dataset to facilitate investigations. Experiments demonstrate that non-speech audio noises significantly impact ASD models, and our proposed approach improves ASD performance in noisy envi

Author

Vasireddy, Siva Sai Nagender; Zhang, Chenxu; Guo, Xiaohu; Tian, Yapeng

Published

2024

Language

EN