Can I read DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation on EtoBox?

DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation by Wu, Xueqing; Zheng, Rui; Sha, Jingzhen; Wu, Te-Lin; Zhou, Hanyu; Tang, Mohan; Chang, Kai-Wei; Peng, Nanyun; Huang, Haoran is a scholarly article available to read on EtoBox.

What is DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation about?

Data analysis is a crucial analytical process to generate in-depth studies and conclusive insights to comprehensively answer a given user query for tabular data. In this work, we aim to propose new resources and benchmarks to inspire future research on this crucial yet challenging and under-explored task. However, collecting data analysis annotations curated by experts can be prohibitively expensive. We propose to automatically generate high-quality answer annotations leveraging the code-generation capabilities of LLMs with a multi-turn prompting technique. We construct the DACO dataset, containing (1) 440 databases (of tabular data) collected from real-world scenarios, (2) ~2k query-answer pairs that can serve as weak supervision for model training, and (3) a concentrated but high-quality test set with human refined annotations that serves as our main evaluation benchmark. We train a 6B supervised fine-tuning (SFT) model on DACO dataset, and find that the SFT model learns reasonable data analysis capabilities. To further align the models with human preference, we use reinforcement learning to encourage generating analysis perceived by human as helpful, and design a set of dense re

Author

Wu, Xueqing; Zheng, Rui; Sha, Jingzhen; Wu, Te-Lin; Zhou, Hanyu; Tang, Mohan; Chang, Kai-Wei; Peng, Nanyun; Huang, Haoran

Published

2024

Language

EN