Can I read Mitigating Reward Over-Optimization on EtoBox?

Mitigating Reward Over-Optimization by 5mvtjr00i is a document available to read on EtoBox.

What is Mitigating Reward Over-Optimization about?

This paper presents the Behavior-Supported Policy Optimization (BSPO) method to address the challenge of reward over-optimization in reinforcement learning from human feedback (RLHF) for large language models (LLMs). BSPO utilizes a behavior policy to regularize the value function, effectively penalizing out-of-distribution responses while preserving in-distribution evaluations, thus reducing the frequency of OOD response generation. Theoretical proofs and empirical results demonstrate that BSPO guarantees

Author

5mvtjr00i

Language

EN