Can I read Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games on EtoBox?

Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games by Ma, Chengdong; Yang, Ziran; Ci, Hai; Gao, Jun; Gao, Minquan; Pan, Xuehai; Yang, Yaodong is a scholarly article available to read on EtoBox.

What is Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games about?

The primary challenge in deploying Large Language Model (LLM) is ensuring its harmlessness. Red team can identify vulnerabilities by attacking LLM to attain safety. However, current efforts heavily rely on single-round prompt designs and unilateral red team optimizations against fixed blue teams. These static approaches lead to significant reductions in generation diversity, known as the mode collapse, which makes it difficult to discover the potential risks in the increasingly complex human-LLM interactions. Here we introduce dynamic Red Team Game (RTG) to comprehensively analyze the multi-round offensive and defensive interactions between red team and blue team. Furthermore, we develop a Gamified Red Team Solver (GRTS) with diversity measures to mitigate mode collapse and theoretically guarantee the convergence of approximate Nash equilibrium which results in better strategies for both teams. Empirical results demonstrate that GRTS explore diverse and implicit attacks to adaptively exploit various LLMs, surpassing the constraints of specific modes. Insightfully, the geometrical structure we unveil of the red team task aligns with the spinning top hypothesis, confirming the necess

Author

Ma, Chengdong; Yang, Ziran; Ci, Hai; Gao, Jun; Gao, Minquan; Pan, Xuehai; Yang, Yaodong

Published

2023

Language

EN