Can I read Small Models are LLM Knowledge Triggers on Medical Tabular Prediction on EtoBox?

Small Models are LLM Knowledge Triggers on Medical Tabular Prediction by Yan, Jiahuan; Chen, Jintai; Hu, Chaowen; Zheng, Bo; Hu, Yaojun; Sun, Jimeng; Wu, Jian is a scholarly article available to read on EtoBox.

What is Small Models are LLM Knowledge Triggers on Medical Tabular Prediction about?

Recent development in large language models (LLMs) has demonstrated impressive domain proficiency on unstructured textual or multi-modal tasks. However, despite with intrinsic world knowledge, their application on structured tabular data prediction still lags behind, primarily due to the numerical insensitivity and modality discrepancy that brings a gap between LLM reasoning and statistical tabular learning. Unlike textual or vision data (e.g., electronic clinical notes or medical imaging data), tabular data is often presented in heterogeneous numerical values (e.g., CBC reports). This ubiquitous data format requires intensive expert annotation, and its numerical nature limits LLMs' capability to effectively transfer untapped domain expertise. In this paper, we propose SERSAL, a general self-prompting method by synergy learning with small models to enhance LLM tabular prediction in an unsupervised manner. Specifically, SERSAL utilizes the LLM's prior outcomes as original soft noisy annotations, which are dynamically leveraged to teach a better small student model. Reversely, the outcomes from the trained small model are used to teach the LLM to further refine its real capability. T

Author

Yan, Jiahuan; Chen, Jintai; Hu, Chaowen; Zheng, Bo; Hu, Yaojun; Sun, Jimeng; Wu, Jian

Published

2024

Language

EN