Can I read Counterfactual Risk Assessments, Evaluation, and Fairness on EtoBox?

Counterfactual Risk Assessments, Evaluation, and Fairness by Coston, Amanda; Mishler, Alan; Kennedy, Edward H.; Chouldechova, Alexandra is a scholarly article available to read on EtoBox.

What is Counterfactual Risk Assessments, Evaluation, and Fairness about?

Algorithmic risk assessments are increasingly used to help humans make decisions in high-stakes settings, such as medicine, criminal justice and education. In each of these cases, the purpose of the risk assessment tool is to inform actions, such as medical treatments or release conditions, often with the aim of reducing the likelihood of an adverse event such as hospital readmission or recidivism. Problematically, most tools are trained and evaluated on historical data in which the outcomes observed depend on the historical decision-making policy. These tools thus reflect risk under the historical policy, rather than under the different decision options that the tool is intended to inform. Even when tools are constructed to predict risk under a specific decision, they are often improperly evaluated as predictors of the target outcome. Focusing on the evaluation task, in this paper we define counterfactual analogues of common predictive performance and algorithmic fairness metrics that we argue are better suited for the decision-making context. We introduce a new method for estimating the proposed metrics using doubly robust estimation. We provide theoretical results that show that

Author

Coston, Amanda; Mishler, Alan; Kennedy, Edward H.; Chouldechova, Alexandra

Published

2019

Language

EN