Can I read ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA on EtoBox?

ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA by Han, Song; Kang, Junlong; Mao, Huizi; Hu, Yiming; Li, Xin; Li, Yubin; Xie, Dongliang; Luo, Hong; Yao, Song; Wang, Yu; Yang, Huazhong; Dally, William J. is a scholarly article available to read on EtoBox.

What is ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA about?

Long Short-Term Memory (LSTM) is widely used in speech recognition. In order to achieve higher prediction accuracy, machine learning scientists have built larger and larger models. Such large model is both computation intensive and memory intensive. Deploying such bulky model results in high power consumption and leads to high total cost of ownership (TCO) of a data center. In order to speedup the prediction and make it energy efficient, we first propose a load-balance-aware pruning method that can compress the LSTM model size by 20x (10x from pruning and 2x from quantization) with negligible loss of the prediction accuracy. The pruned model is friendly for parallel processing. Next, we propose scheduler that encodes and partitions the compressed model to each PE for parallelism, and schedule the complicated LSTM data flow. Finally, we design the hardware architecture, named Efficient Speech Recognition Engine (ESE) that works directly on the compressed model. Implemented on Xilinx XCKU060 FPGA running at 200MHz, ESE has a performance of 282 GOPS working directly on the compressed LSTM network, corresponding to 2.52 TOPS on the uncompressed one, and processes a full LSTM for speech

Author

Han, Song; Kang, Junlong; Mao, Huizi; Hu, Yiming; Li, Xin; Li, Yubin; Xie, Dongliang; Luo, Hong; Yao, Song; Wang, Yu; Yang, Huazhong; Dally, William J.

Published

2016

Language

EN