Can I read Checkpoint Ensembles: Ensemble Methods from a Single Training Process on EtoBox?

Checkpoint Ensembles: Ensemble Methods from a Single Training Process by Chen, Hugh; Lundberg, Scott; Lee, Su-In is a scholarly article available to read on EtoBox.

What is Checkpoint Ensembles: Ensemble Methods from a Single Training Process about?

We present the checkpoint ensembles method that can learn ensemble models on a single training process. Although checkpoint ensembles can be applied to any parametric iterative learning technique, here we focus on neural networks. Neural networks' composable and simple neurons make it possible to capture many individual and interaction effects among features. However, small sample sizes and sampling noise may result in patterns in the training data that are not representative of the true relationship between the features and the outcome. As a solution, regularization during training is often used (e.g. dropout). However, regularization is no panacea -- it does not perfectly address overfitting. Even with methods like dropout, two methodologies are commonly used in practice. First is to utilize a validation set independent to the training set as a way to decide when to stop training. Second is to use ensemble methods to further reduce overfitting and take advantage of local optima (i.e. averaging over the predictions of several models). In this paper, we explore checkpoint ensembles -- a simple technique that combines these two ideas in one training process. Checkpoint ensembles imp

Author

Chen, Hugh; Lundberg, Scott; Lee, Su-In

Published

2017

Language

EN