Can I read Those Aren't Your Memories, They're Somebody Else's: Seeding Misinformation in Chat Bot Memories on EtoBox?

Those Aren't Your Memories, They're Somebody Else's: Seeding Misinformation in Chat Bot Memories by Atkins, Conor; Zhao, Benjamin Zi Hao; Asghar, Hassan Jameel; Wood, Ian; Kaafar, Mohamed Ali is a scholarly article available to read on EtoBox.

What is Those Aren't Your Memories, They're Somebody Else's: Seeding Misinformation in Chat Bot Memories about?

One of the new developments in chit-chat bots is a long-term memory mechanism that remembers information from past conversations for increasing engagement and consistency of responses. The bot is designed to extract knowledge of personal nature from their conversation partner, e.g., stating preference for a particular color. In this paper, we show that this memory mechanism can result in unintended behavior. In particular, we found that one can combine a personal statement with an informative statement that would lead the bot to remember the informative statement alongside personal knowledge in its long term memory. This means that the bot can be tricked into remembering misinformation which it would regurgitate as statements of fact when recalling information relevant to the topic of conversation. We demonstrate this vulnerability on the BlenderBot 2 framework implemented on the ParlAI platform and provide examples on the more recent and significantly larger BlenderBot 3 model. We generate 150 examples of misinformation, of which 114 (76%) were remembered by BlenderBot 2 when combined with a personal statement. We further assessed the risk of this misinformation being recalled aft

Author

Atkins, Conor; Zhao, Benjamin Zi Hao; Asghar, Hassan Jameel; Wood, Ian; Kaafar, Mohamed Ali

Published

2023

Language

EN