Can I read Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation on EtoBox?

Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation by Yadav, Hemant; Sitaram, Sunayana; Shah, Rajiv Ratn is a scholarly article available to read on EtoBox.

What is Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation about?

Speech modeling methods learn one embedding for a fixed segment of speech, typically in between 10-25 ms. The information present in speech can be divided into two categories: "what is being said" (content) and "how it is expressed" (other) and these two are orthogonal in nature causing the optimization algorithm to find a sub-optimal solution if forced to optimize together. This leads to sub-optimal performance in one or all downstream tasks as shown by previous studies. Current self-supervised learning (SSL) methods such as HuBERT are very good at modeling the content information present in speech. Data augmentation improves the performance on tasks which require effective modeling of other information but this leads to a divided capacity of the model. In this work, we conduct a preliminary study to understand the importance of modeling other information using separate learnable parameters. We propose a modified version of HuBERT, termed Other HuBERT (O-HuBERT), to test our hypothesis. Our findings are twofold: first, the O-HuBERT method is able to utilize all layers to build complex features to encode other information; second, a robust data augmentation strategy is essential fo

Author

Yadav, Hemant; Sitaram, Sunayana; Shah, Rajiv Ratn

Published

2024

Language

EN