Can I read Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint on EtoBox?

Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint by Quélennec, Aël; Tartaglione, Enzo; Mozharovskyi, Pavlo; Nguyen, Van-Tam is a scholarly article available to read on EtoBox.

What is Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint about?

In the realm of efficient on-device learning under extreme memory and computation constraints, a significant gap in successful approaches persists. Although considerable effort has been devoted to efficient inference, the main obstacle to efficient learning is the prohibitive cost of backpropagation. The resources required to compute gradients and update network parameters often exceed the limits of tightly constrained memory budgets. This paper challenges conventional wisdom and proposes a series of experiments that reveal the existence of superior sub-networks. Furthermore, we hint at the potential for substantial gains through a dynamic neuron selection strategy when fine-tuning a target task. Our efforts extend to the adaptation of a recent dynamic neuron selection strategy pioneered by Bragagnolo et al. (NEq), revealing its effectiveness in the most stringent scenarios. Our experiments demonstrate, in the average case, the superiority of a NEq-inspired approach over a random selection. This observation prompts a compelling avenue for further exploration in the area, highlighting the opportunity to design a new class of algorithms designed to facilitate parameter update selecti

Author

Quélennec, Aël; Tartaglione, Enzo; Mozharovskyi, Pavlo; Nguyen, Van-Tam

Published

2023

Language

EN