Can I read Sample Weight Estimation Using Meta-Updates for Online Continual Learning on EtoBox?

Sample Weight Estimation Using Meta-Updates for Online Continual Learning by Hemati, Hamed; Borth, Damian is a scholarly article available to read on EtoBox.

What is Sample Weight Estimation Using Meta-Updates for Online Continual Learning about?

The loss function plays an important role in optimizing the performance of a learning system. A crucial aspect of the loss function is the assignment of sample weights within a mini-batch during loss computation. In the context of continual learning (CL), most existing strategies uniformly treat samples when calculating the loss value, thereby assigning equal weights to each sample. While this approach can be effective in certain standard benchmarks, its optimal effectiveness, particularly in more complex scenarios, remains underexplored. This is particularly pertinent in training "in the wild," such as with self-training, where labeling is automated using a reference model. This paper introduces the Online Meta-learning for Sample Importance (OMSI) strategy that approximates sample weights for a mini-batch in an online CL stream using an inner- and meta-update mechanism. This is done by first estimating sample weight parameters for each sample in the mini-batch, then, updating the model with the adapted sample weights. We evaluate OMSI in two distinct experimental settings. First, we show that OMSI enhances both learning and retained accuracy in a controlled noisy-labeled data str

Author

Hemati, Hamed; Borth, Damian

Published

2024

Language

EN