Can I read Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries on EtoBox?

Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries by Jin, Yiqiao; Chandra, Mohit; Verma, Gaurav; Hu, Yibo; De Choudhury, Munmun; Kumar, Srijan is a scholarly article available to read on EtoBox.

What is Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries about?

Large language models (LLMs) are transforming the ways the general public accesses and consumes information. Their influence is particularly pronounced in pivotal sectors like healthcare, where lay individuals are increasingly appropriating LLMs as conversational agents for everyday queries. While LLMs demonstrate impressive language understanding and generation proficiencies, concerns regarding their safety remain paramount in these high-stake domains. Moreover, the development of LLMs is disproportionately focused on English. It remains unclear how these LLMs perform in the context of non-English languages, a gap that is critical for ensuring equity in the real-world use of these systems.This paper provides a framework to investigate the effectiveness of LLMs as multi-lingual dialogue systems for healthcare queries. Our empirically-derived framework XlingEval focuses on three fundamental criteria for evaluating LLM responses to naturalistic human-authored health-related questions: correctness, consistency, and verifiability. Through extensive experiments on four major global languages, including English, Spanish, Chinese, and Hindi, spanning three expert-annotated large health Q&

Author

Jin, Yiqiao; Chandra, Mohit; Verma, Gaurav; Hu, Yibo; De Choudhury, Munmun; Kumar, Srijan

Published

2023

Language

EN