Can I read Why Machine Reading Comprehension Models Learn Shortcuts? on EtoBox?

Why Machine Reading Comprehension Models Learn Shortcuts? by Lai, Yuxuan; Zhang, Chen; Feng, Yansong; Huang, Quzhe; Zhao, Dongyan is a scholarly article available to read on EtoBox.

What is Why Machine Reading Comprehension Models Learn Shortcuts? about?

Recent studies report that many machine reading comprehension (MRC) models can perform closely to or even better than humans on benchmark datasets. However, existing works indicate that many MRC models may learn shortcuts to outwit these benchmarks, but the performance is unsatisfactory in real-world applications. In this work, we attempt to explore, instead of the expected comprehension skills, why these models learn the shortcuts. Based on the observation that a large portion of questions in current datasets have shortcut solutions, we argue that larger proportion of shortcut questions in training data make models rely on shortcut tricks excessively. To investigate this hypothesis, we carefully design two synthetic datasets with annotations that indicate whether a question can be answered using shortcut solutions. We further propose two new methods to quantitatively analyze the learning difficulty regarding shortcut and challenging questions, and revealing the inherent learning mechanism behind the different performance between the two kinds of questions. A thorough empirical analysis shows that MRC models tend to learn shortcut questions earlier than challenging questions, and t

Author

Lai, Yuxuan; Zhang, Chen; Feng, Yansong; Huang, Quzhe; Zhao, Dongyan

Published

2021

Language

EN