About this scholarly article

Large Language Models Vote: Prompting for Rare Disease Identification by Oniani, David; Hilsman, Jordan; Dong, Hang; Gao, Fengyi; Verma, Shiven; Wang, Yanshan is a scholarly article available to read on EtoBox.

The emergence of generative Large Language Models (LLMs) emphasizes the need for accurate and efficient prompting approaches. LLMs are often applied in Few-Shot Learning (FSL) contexts, where tasks are executed with minimal training data. FSL has become popular in many Artificial Intelligence (AI) subdomains, including AI for health. Rare diseases affect a small fraction of the population. Rare disease identification from clinical notes inherently requires FSL techniques due to limited data availability. Manual data collection and annotation is both expensive and time-consuming. In this paper, we propose Models-Vote Prompting (MVP), a flexible prompting approach for improving the performance of LLM queries in FSL settings. MVP works by prompting numerous LLMs to perform the same tasks and then conducting a majority vote on the resulting outputs. This method achieves improved results to any one model in the ensemble on one-shot rare disease identification and classification tasks. We also release a novel rare disease dataset for FSL, available to those who signed the MIMIC-IV Data Use Agreement (DUA). Furthermore, in using MVP, each model is prompted multiple times, substantially in

Author

Oniani, David; Hilsman, Jordan; Dong, Hang; Gao, Fengyi; Verma, Shiven; Wang, Yanshan

Published

2023

Language

EN