Can I read Impact of RoCE Congestion Control Policies on Distributed Training of DNNs on EtoBox?

Impact of RoCE Congestion Control Policies on Distributed Training of DNNs by Khan, Tarannum; Rashidi, Saeed; Sridharan, Srinivas; Shurpali, Pallavi; Akella, Aditya; Krishna, Tushar is a scholarly article available to read on EtoBox.

What is Impact of RoCE Congestion Control Policies on Distributed Training of DNNs about?

RDMA over Converged Ethernet (RoCE) has gained significant attraction for datacenter networks due to its compatibility with conventional Ethernet-based fabric. However, the RDMA protocol is efficient only on (nearly) lossless networks, emphasizing the vital role of congestion control on RoCE networks. Unfortunately, the native RoCE congestion control scheme, based on Priority Flow Control (PFC), suffers from many drawbacks such as unfairness, head-of-line-blocking, and deadlock. Therefore, in recent years many schemes have been proposed to provide additional congestion control for RoCE networks to minimize PFC drawbacks. However, these schemes are proposed for general datacenter environments. In contrast to the general datacenters that are built using commodity hardware and run general-purpose workloads, high-performance distributed training platforms deploy high-end accelerators and network components and exclusively run training workloads using collectives (All-Reduce, All-To-All) communication libraries for communication. Furthermore, these platforms usually have a private network, separating their communication traffic from the rest of the datacenter traffic. Scalable topology-

Author

Khan, Tarannum; Rashidi, Saeed; Sridharan, Srinivas; Shurpali, Pallavi; Akella, Aditya; Krishna, Tushar

Published

2022

Language

EN