About this document

SPIRIT LM: Multimodal Language Model by hehegah689 is a document available to read on EtoBox.

S PIRIT LM is a multimodal language model that integrates text and speech, trained on a 7B pretrained text model enhanced with speech data. It features two versions: BASE, which uses speech phonetic units, and EXPRESSIVE, which adds expressivity through pitch and style tokens. The model demonstrates capabilities in few-shot learning across modalities and offers a new benchmark for sentiment preservation in generative tasks involving both speech and text.

Author

hehegah689

Language

EN