Can I read ElastiQuant: elastic quantization strategy for communication efficient distributed machine learning in IoT on EtoBox?

ElastiQuant: elastic quantization strategy for communication efficient distributed machine learning in IoT by Bharath Sudharsan; John G. Breslin; Muhammad Intizar Ali; Peter Corcoran; Rajiv Ranjan is a scholarly article available to read on EtoBox.

What is ElastiQuant: elastic quantization strategy for communication efficient distributed machine learning in IoT about?

In training distributed machine learning, communicating model updates among workers has always been a bottleneck. The magnitude of impact on the quality of resultant models is higher when distributed training on low hardware specification devices and in uncertain real-world IoT networks where congestion, latency, bandwidth issues are common. In this scenario, gradient quantization plus encoding is an effective way to reduce cost when communicating model updates. Other approaches can be to limit the clientserver communication frequency, adaptive compression by varying the spacing between quantization levels, reusing outdated gradients, deep compression to reduce transmission packet size, and adaptive tuning of the number of bits transmitted per round. The optimization levels provided by such and other non-comprehensive approaches do not suffice for high-dimensional NN models with large size model updates.This paper presents ElastiQuant, an elastic quantization strategy that aims to reduce the impact caused by limitations in distributed IoT training scenarios. The distinguishable highlights of this comprehensive work are: (i) theoretical assurances and bounds on variance and number o

Author

Bharath Sudharsan; John G. Breslin; Muhammad Intizar Ali; Peter Corcoran; Rajiv Ranjan

Publisher

ACM

Published

2022

Language

EN