Can I read Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models on EtoBox?

Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models by Duan, Sunny; Khona, Mikail; Iyer, Abhiram; Schaeffer, Rylan; Fiete, Ila R is a scholarly article available to read on EtoBox.

What is Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models about?

Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private data raise profound concerns about data privacy and security. Language models are trained on extensive corpora including potentially sensitive or proprietary information, and the risk of data leakage - where the model response reveals pieces of such information - remains inadequately understood. Prior work has investigated what factors drive memorization and have identified that sequence complexity and the number of repetitions drive memorization. Here, we focus on the evolution of memorization over training. We begin by reproducing findings that the probability of memorizing a sequence scales logarithmically with the number of times it is present in the data. We next show that sequences which are apparently not memorized after the first encounter can be "uncovered" throughout the course of training even without subsequent encounters, a phenomenon we term "latent memorization". The presence of latent memorization presents a challenge for data privacy as memorized sequences may be hidden at the final check

Author

Duan, Sunny; Khona, Mikail; Iyer, Abhiram; Schaeffer, Rylan; Fiete, Ila R

Published

2024

Language

EN