Can I read Rethinking Evaluation Metric for Probability Estimation Models Using Esports Data on EtoBox?

Rethinking Evaluation Metric for Probability Estimation Models Using Esports Data by Choi, Euihyeon; Kim, Jooyoung; Lee, Wonkyung is a scholarly article available to read on EtoBox.

What is Rethinking Evaluation Metric for Probability Estimation Models Using Esports Data about?

Probability estimation models play an important role in various fields, such as weather forecasting, recommendation systems, and sports analysis. Among several models estimating probabilities, it is difficult to evaluate which model gives reliable probabilities since the ground-truth probabilities are not available. The win probability estimation model for esports, which calculates the win probability under a certain game state, is also one of the fields being actively studied in probability estimation. However, most of the previous works evaluated their models using accuracy, a metric that only can measure the performance of discrimination. In this work, we firstly investigate the Brier score and the Expected Calibration Error (ECE) as a replacement of accuracy used as a performance evaluation metric for win probability estimation models in esports field. Based on the analysis, we propose a novel metric called Balance score which is a simple yet effective metric in terms of six good properties that probability estimation metric should have. Under the general condition, we also found that the Balance score can be an effective approximation of the true expected calibration error whi

Author

Choi, Euihyeon; Kim, Jooyoung; Lee, Wonkyung

Published

2023

Language

EN