Can I read SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization on EtoBox?

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization by Dahan, Tehila; Levy, Kfir Y. is a scholarly article available to read on EtoBox.

What is SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization about?

We consider distributed learning scenarios where M machines interact with a parameter server along several communication rounds in order to minimize a joint objective function. Focusing on the heterogeneous case, where different machines may draw samples from different data-distributions, we design the first local update method that provably benefits over the two most prominent distributed baselines: namely Minibatch-SGD and Local-SGD. Key to our approach is a slow querying technique that we customize to the distributed setting, which in turn enables a better mitigation of the bias caused by local updates.

Author

Dahan, Tehila; Levy, Kfir Y.

Published

2023

Language

EN