Can I read BiasRV: Uncovering Biased Sentiment Predictions at Runtime on EtoBox?

BiasRV: Uncovering Biased Sentiment Predictions at Runtime by Yang, Zhou; Asyrofi, Muhammad Hilmi; Lo, David is a scholarly article available to read on EtoBox.

What is BiasRV: Uncovering Biased Sentiment Predictions at Runtime about?

Sentiment analysis (SA) systems, though widely applied in many domains, have been demonstrated to produce biased results. Some research works have been done in automatically generating test cases to reveal unfairness in SA systems, but the community still lacks tools that can monitor and uncover biased predictions at runtime. This paper fills this gap by proposing BiasRV, the first tool to raise an alarm when a deployed SA system makes a biased prediction on a given input text. To implement this feature, BiasRV dynamically extracts a template from an input text and from the template generates gender-discriminatory mutants (semantically-equivalent texts that only differ in gender information). Based on popular metrics used to evaluate the overall fairness of an SA system, we define distributional fairness property for an individual prediction of an SA system. This property specifies a requirement that for one piece of text, mutants from different gender classes should be treated similarly as a whole. Verifying the distributional fairness property causes much overhead to the running system. To run more efficiently, BiasRV adopts a two-step heuristic: (1) sampling several mutants from

Author

Yang, Zhou; Asyrofi, Muhammad Hilmi; Lo, David

Published

2021

Language

EN