Can I read MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity on EtoBox?

MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity by Liu, Yangzhou; Cao, Yue; Gao, Zhangwei; Wang, Weiyun; Chen, Zhe; Wang, Wenhai; Tian, Hao; Lu, Lewei; Zhu, Xizhou; Lu, Tong; Qiao, Yu; Dai, Jifeng is a scholarly article available to read on EtoBox.

What is MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity about?

Despite the effectiveness of vision-language supervised fine-tuning in enhancing the performance of Vision Large Language Models (VLLMs). However, existing visual instruction tuning datasets include the following limitations: (1) Instruction annotation quality: despite existing VLLMs exhibiting strong performance, instructions generated by those advanced VLLMs may still suffer from inaccuracies, such as hallucinations. (2) Instructions and image diversity: the limited range of instruction types and the lack of diversity in image data may impact the model's ability to generate diversified and closer to real-world scenarios outputs. To address these challenges, we construct a high-quality, diverse visual instruction tuning dataset MMInstruct, which consists of 973K instructions from 24 domains. There are four instruction types: Judgement, Multiple-Choice, Long Visual Question Answering and Short Visual Question Answering. To construct MMInstruct, we propose an instruction generation data engine that leverages GPT-4V, GPT-3.5, and manual correction. Our instruction generation engine enables semi-automatic, low-cost, and multi-domain instruction generation at 1/6 the cost of manual con

Author

Liu, Yangzhou; Cao, Yue; Gao, Zhangwei; Wang, Weiyun; Chen, Zhe; Wang, Wenhai; Tian, Hao; Lu, Lewei; Zhu, Xizhou; Lu, Tong; Qiao, Yu; Dai, Jifeng

Published

2024

Language

EN