Can I read Reducing Catastrophic Forgetting with Learning on Synthetic Data on EtoBox?

Reducing Catastrophic Forgetting with Learning on Synthetic Data by Masarczyk, Wojciech; Tautkute, Ivona is a scholarly article available to read on EtoBox.

What is Reducing Catastrophic Forgetting with Learning on Synthetic Data about?

Catastrophic forgetting is a problem caused by neural networks' inability to learn data in sequence. After learning two tasks in sequence, performance on the first one drops significantly. This is a serious disadvantage that prevents many deep learning applications to real-life problems where not all object classes are known beforehand; or change in data requires adjustments to the model. To reduce this problem we investigate the use of synthetic data, namely we answer a question: Is it possible to generate such data synthetically which learned in sequence does not result in catastrophic forgetting? We propose a method to generate such data in two-step optimisation process via meta-gradients. Our experimental results on Split-MNIST dataset show that training a model on such synthetic data in sequence does not result in catastrophic forgetting. We also show that our method of generating data is robust to different learning scenarios.

Author

Masarczyk, Wojciech; Tautkute, Ivona

Published

2020

Language

EN