Can I read Explicitly Trained Spiking Sparsity in Spiking Neural Networks with Backpropagation on EtoBox?

Explicitly Trained Spiking Sparsity in Spiking Neural Networks with Backpropagation by Allred, Jason M.; Spencer, Steven J.; Srinivasan, Gopalakrishnan; Roy, Kaushik is a scholarly article available to read on EtoBox.

What is Explicitly Trained Spiking Sparsity in Spiking Neural Networks with Backpropagation about?

Spiking Neural Networks (SNNs) are being explored for their potential energy efficiency resulting from sparse, event-driven computations. Many recent works have demonstrated effective backpropagation for deep Spiking Neural Networks (SNNs) by approximating gradients over discontinuous neuron spikes or firing events. A beneficial side-effect of these surrogate gradient spiking backpropagation algorithms is that the spikes, which trigger additional computations, may now themselves be directly considered in the gradient calculations. We propose an explicit inclusion of spike counts in the loss function, along with a traditional error loss, causing the backpropagation learning algorithms to optimize weight parameters for both accuracy and spiking sparsity. As supported by existing theory of over-parameterized neural networks, there are many solution states with effectively equivalent accuracy. As such, appropriate weighting of the two loss goals during training in this multi-objective optimization process can yield an improvement in spiking sparsity without a significant loss of accuracy. We additionally explore a simulated annealing-inspired loss weighting technique to increase the we

Author

Allred, Jason M.; Spencer, Steven J.; Srinivasan, Gopalakrishnan; Roy, Kaushik

Published

2020

Language

EN