Can I read MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector on EtoBox?

MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector by Fu, Wenjie; Wang, Huandong; Gao, Chen; Liu, Guanghua; Li, Yong; Jiang, Tao is a scholarly article available to read on EtoBox.

What is MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector about?

The increasing parameters and expansive dataset of large language models (LLMs) highlight the urgent demand for a technical solution to audit the underlying privacy risks and copyright issues associated with LLMs. Existing studies have partially addressed this need through an exploration of the pre-training data detection problem, which is an instance of a membership inference attack (MIA). This problem involves determining whether a given piece of text has been used during the pre-training phase of the target LLM. Although existing methods have designed various sophisticated MIA score functions to achieve considerable detection performance in pre-trained LLMs, how to achieve high-confidence detection and how to perform MIA on aligned LLMs remain challenging. In this paper, we propose MIA-Tuner, a novel instruction-based MIA method, which instructs LLMs themselves to serve as a more precise pre-training data detector internally, rather than design an external MIA score function. Furthermore, we design two instruction-based safeguards to respectively mitigate the privacy risks brought by the existing methods and MIA-Tuner. To comprehensively evaluate the most recent state-of-the-art

Author

Fu, Wenjie; Wang, Huandong; Gao, Chen; Liu, Guanghua; Li, Yong; Jiang, Tao

Published

2024

Language

EN