Can I read A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization on EtoBox?

A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization by Sypherd, Tyler; Diaz, Mario; Cava, John Kevin; Dasarathy, Gautam; Kairouz, Peter; Sankar, Lalitha is a scholarly article available to read on EtoBox.

What is A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization about?

We introduce a tunable loss function called $\alpha$-loss, parameterized by $\alpha \in (0,\infty]$, which interpolates between the exponential loss ($\alpha = 1/2$), the log-loss ($\alpha = 1$), and the 0-1 loss ($\alpha = \infty$), for the machine learning setting of classification. Theoretically, we illustrate a fundamental connection between $\alpha$-loss and Arimoto conditional entropy, verify the classification-calibration of $\alpha$-loss in order to demonstrate asymptotic optimality via Rademacher complexity generalization techniques, and build-upon a notion called strictly local quasi-convexity in order to quantitatively characterize the optimization landscape of $\alpha$-loss. Practically, we perform class imbalance, robustness, and classification experiments on benchmark image datasets using convolutional-neural-networks. Our main practical conclusion is that certain tasks may benefit from tuning $\alpha$-loss away from log-loss ($\alpha = 1$), and to this end we provide simple heuristics for the practitioner. In particular, navigating the $\alpha$ hyperparameter can readily provide superior model robustness to label flips ($\alpha > 1$) and sensitivity to imbalanced cla

Author

Sypherd, Tyler; Diaz, Mario; Cava, John Kevin; Dasarathy, Gautam; Kairouz, Peter; Sankar, Lalitha

Published

2019

Language

EN