About this document
Unified Speech Tokenizer for LLMs by billeton is a document available to read on EtoBox.
The document proposes SpeechTokenizer, a unified speech tokenizer for speech language models. SpeechTokenizer aims to address limitations of existing semantic and acoustic speech tokens by disentangling speech information hierarchically across residual vector quantization (RVQ) layers. This unifies semantic and acoustic information in a single tokenizer. The authors establish a benchmark called SLMTokBench to evaluate speech tokens, finding that existing tokens are not ideal for speech language models. Spee
- Author
- billeton
- Language
- EN