Can I read Meta Learning by Optimization on EtoBox?

Meta Learning by Optimization by Pengyu Yuan; Hien Van Nguyen is a book available to read on EtoBox.

What is Meta Learning by Optimization about?

Few-shot learning setting is challenging for gradient-based optimization for several reasons. First, popular gradient-based optimizers are highly flexible with the updating rules applied to a large number of parameters. This property makes it prone to overfitting when there are only a few training samples. Since the loss function is nonconvex, these algorithms do not guarantee the speed of convergence, beyond the fact that they can converge to local optima after many iterations provided that the initialization and hyperparameters are selected correctly. Second, the network usually starts from a random parameter initialization for each new task, making it challenging to converge with only a few iterations. Transfer learning and domain adaptation techniques can mitigate this problem by initializing parameters with those of a pretrained network from another task where more labels are available. However, the benefit of these approaches greatly decreases when the training and testing tasks diverge. Developing a systematic mechanism to benefit from common initialization while guaranteeing fast convergence is vital to dealing with few-shot learning setting. To this end, optimization-based

Author

Pengyu Yuan; Hien Van Nguyen

Publisher

Elsevier

Published

2023

Language

EN