Can I read Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning on EtoBox?

Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning by Wu, Jian; Toscano-Palmerin, Saul; Frazier, Peter I.; Wilson, Andrew Gordon is a scholarly article available to read on EtoBox.

What is Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning about?

Bayesian optimization is popular for optimizing time-consuming black-box objectives. Nonetheless, for hyperparameter tuning in deep neural networks, the time required to evaluate the validation error for even a few hyperparameter settings remains a bottleneck. Multi-fidelity optimization promises relief using cheaper proxies to such objectives --- for example, validation error for a network trained using a subset of the training points or fewer iterations than required for convergence. We propose a highly flexible and practical approach to multi-fidelity Bayesian optimization, focused on efficiently optimizing hyperparameters for iteratively trained supervised learning models. We introduce a new acquisition function, the trace-aware knowledge-gradient, which efficiently leverages both multiple continuous fidelity controls and trace observations --- values of the objective at a sequence of fidelities, available when varying fidelity using training iterations. We provide a provably convergent method for optimizing our acquisition function and show it outperforms state-of-the-art alternatives for hyperparameter tuning of deep neural networks and large-scale kernel learning.

Author

Wu, Jian; Toscano-Palmerin, Saul; Frazier, Peter I.; Wilson, Andrew Gordon

Published

2019

Language

EN