Can I read Text is all You Need: Personalizing ASR Models Using Controllable Speech Synthesis on EtoBox?

Text is all You Need: Personalizing ASR Models Using Controllable Speech Synthesis by Karren Yang; Ting-Yao Hu; Jen-Hao Rick Chang; Hema Swetha Koppula; Oncel Tuzel is a scholarly article available to read on EtoBox.

What is Text is all You Need: Personalizing ASR Models Using Controllable Speech Synthesis about?

Adapting generic speech recognition models to specific individuals is a challenging problem due to the scarcity of personalized data. Recent works have proposed boosting the amount of training data using personalized text-to-speech synthesis. Here, we ask two fundamental questions about this strategy: when is synthetic data effective for personalization, and why is it effective in those cases? To address the first question, we adapt a state-of-the-art automatic speech recognition (ASR) model to target speakers from four benchmark datasets representative of different speaker types. We show that ASR personalization with synthetic data is effective in all cases, but particularly when (i) the target speaker is underrepresented in the global data, and (ii) the capacity of the global model is limited. To address the second question of why personalized synthetic data is effective, we use controllable speech synthesis (CSS) to generate speech with varied styles and content. Surprisingly, we find that the text content of the synthetic data, rather than style, is important for speaker adaptation. These results lead us to propose a data selection strategy for ASR personalization based on spee

Author

Karren Yang; Ting-Yao Hu; Jen-Hao Rick Chang; Hema Swetha Koppula; Oncel Tuzel

Publisher

IEEE

Published

2023

Language

EN