Can I read Evaluating Byte and Wordpiece Level Models for Massively Multilingual Semantic Parsing on EtoBox?

Evaluating Byte and Wordpiece Level Models for Massively Multilingual Semantic Parsing by Nicosia, Massimo; Piccinno, Francesco is a scholarly article available to read on EtoBox.

What is Evaluating Byte and Wordpiece Level Models for Massively Multilingual Semantic Parsing about?

Token free approaches have been successfully applied to a series of word and span level tasks. In this work, we compare a byte-level (ByT5) and a wordpiece based (mT5) sequence to sequence model on the 51 languages of the MASSIVE multilingual semantic parsing dataset. We examine multiple experimental settings: (i) zero-shot, (ii) full gold data and (iii) zero-shot with synthetic data. By leveraging a state-of-the-art label projection method for machine translated examples, we are able to reduce the gap in exact match accuracy to only 5 points with respect to a model trained on gold data from all the languages. We additionally provide insights on the cross-lingual transfer of ByT5 and show how the model compares with respect to mT5 across all parameter sizes.

Author

Nicosia, Massimo; Piccinno, Francesco

Published

2022

Language

EN