Can I read Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks on EtoBox?

Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks by Palit, Sanchar; Banerjee, Biplab; Chaudhuri, Subhasis is a scholarly article available to read on EtoBox.

What is Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks about?

We propose a Bayesian neural network-based continual learning algorithm using Variational Inference, aiming to overcome several drawbacks of existing methods. Specifically, in continual learning scenarios, storing network parameters at each step to retain knowledge poses challenges. This is compounded by the crucial need to mitigate catastrophic forgetting, particularly given the limited access to past datasets, which complicates maintaining correspondence between network parameters and datasets across all sessions. Current methods using Variational Inference with KL divergence risk catastrophic forgetting during uncertain node updates and coupled disruptions in certain nodes. To address these challenges, we propose the following strategies. To reduce the storage of the dense layer parameters, we propose a parameter distribution learning method that significantly reduces the storage requirements. In the continual learning framework employing variational inference, our study introduces a regularization term that specifically targets the dynamics and population of the mean and variance of the parameters. This term aims to retain the benefits of KL divergence while addressing related

Author

Palit, Sanchar; Banerjee, Biplab; Chaudhuri, Subhasis

Published

2024

Language

EN