Can I read A Study of Implicit Ranking Unfairness in Large Language Models on EtoBox?

A Study of Implicit Ranking Unfairness in Large Language Models by Xu, Chen; Wang, Wenjie; Li, Yuxin; Pang, Liang; Xu, Jun; Chua, Tat-Seng is a scholarly article available to read on EtoBox.

What is A Study of Implicit Ranking Unfairness in Large Language Models about?

Recently, Large Language Models (LLMs) have demonstrated a superior ability to serve as ranking models. However, concerns have arisen as LLMs will exhibit discriminatory ranking behaviors based on users' sensitive attributes (\eg gender). Worse still, in this paper, we identify a subtler form of discrimination in LLMs, termed \textit{implicit ranking unfairness}, where LLMs exhibit discriminatory ranking patterns based solely on non-sensitive user profiles, such as user names. Such implicit unfairness is more widespread but less noticeable, threatening the ethical foundation. To comprehensively explore such unfairness, our analysis will focus on three research aspects: (1) We propose an evaluation method to investigate the severity of implicit ranking unfairness. (2) We uncover the reasons for causing such unfairness. (3) To mitigate such unfairness effectively, we utilize a pair-wise regression method to conduct fair-aware data augmentation for LLM fine-tuning. The experiment demonstrates that our method outperforms existing approaches in ranking fairness, achieving this with only a small reduction in accuracy. Lastly, we emphasize the need for the community to identify and mitiga

Author

Xu, Chen; Wang, Wenjie; Li, Yuxin; Pang, Liang; Xu, Jun; Chua, Tat-Seng

Published

2023

Language

EN