Can I read Knowledge Editing for Large Language Models: A Survey on EtoBox?

Knowledge Editing for Large Language Models: A Survey by Wang, Song; Zhu, Yaochen; Liu, Haochen; Zheng, Zaiyi; Chen, Chen; Li, Jundong is a scholarly article available to read on EtoBox.

What is Knowledge Editing for Large Language Models: A Survey about?

Large language models (LLMs) have recently transformed both the academic and industrial landscapes due to their remarkable capacity to understand, analyze, and generate texts based on their vast knowledge and reasoning ability. Nevertheless, one major drawback of LLMs is their substantial computational cost for pre-training due to their unprecedented amounts of parameters. The disadvantage is exacerbated when new knowledge frequently needs to be introduced into the pre-trained model. Therefore, it is imperative to develop effective and efficient techniques to update pre-trained LLMs. Traditional methods encode new knowledge in pre-trained LLMs through direct fine-tuning. However, naively re-training LLMs can be computationally intensive and risks degenerating valuable pre-trained knowledge irrelevant to the update in the model. Recently, Knowledge-based Model Editing (KME) has attracted increasing attention, which aims to precisely modify the LLMs to incorporate specific knowledge, without negatively influencing other irrelevant knowledge. In this survey, we aim to provide a comprehensive and in-depth overview of recent advances in the field of KME. We first introduce a general for

Author

Wang, Song; Zhu, Yaochen; Liu, Haochen; Zheng, Zaiyi; Chen, Chen; Li, Jundong

Published

2023

Language

EN