Can I read Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives on EtoBox?

Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives by Zhang, Xinliang Frederick; Beauchamp, Nick; Wang, Lu is a scholarly article available to read on EtoBox.

What is Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives about?

Reasoning about time and temporal relations is an integral aspect of human cognition, essential for perceiving the world and navigating our experiences. Though large language models (LLMs) have demonstrated impressive performance in many reasoning tasks, temporal reasoning remains challenging due to its intrinsic complexity. In this work, we first study an essential task of temporal reasoning -- temporal graph generation, to unveil LLMs' inherent, global reasoning capabilities. We show that this task presents great challenges even for the most powerful LLMs, such as GPT-3.5/4. We also notice a significant performance gap by small models (<10B) that lag behind LLMs by 50%. Next, we study how to close this gap with a budget constraint, e.g., not using model finetuning. We propose a new prompting technique tailored for temporal reasoning, Narrative-of-Thought (NoT), that first converts the events set to a Python class, then prompts a small model to generate a temporally grounded narrative, guiding the final generation of a temporal graph. Extensive experiments showcase the efficacy of NoT in improving various metrics. Notably, NoT attains the highest F1 on the Schema-11 evaluation set

Author

Zhang, Xinliang Frederick; Beauchamp, Nick; Wang, Lu

Published

2024

Language

EN