Can I read Energy-Efficient Collective Reduce and Allreduce Operations on Distributed GPUs on EtoBox?

Energy-Efficient Collective Reduce and Allreduce Operations on Distributed GPUs by Lena Oden; Benjamin Klenk; Holger Froning is a scholarly article available to read on EtoBox.

What is Energy-Efficient Collective Reduce and Allreduce Operations on Distributed GPUs about?

## GPUs gain high popularity in High Performance Computing, due to their massive parallelism and high performance per Watt. Despite their popularity, data transfer between multiple GPUs in a cluster remains a problem. Most communication models require the CPU to control the data flow; also intermediate staging copies to host memory are often inevitable. These two facts lead to higher CPU and memory utilization. As a result, overall performance decreases and power consumption increases. Collective operations like reduce and allreduce are very common in scientific simulations and also very sensitive to performance. Due to their massive parallelism, GPUs are very suitable for such operations, but they only excel in performance if they can process the problem in-core. Global GPU Address Spaces (GGAS) enable a direct GPU-to-GPU communication for heterogeneous clusters, which is completely in-line with the GPUs thread-collective execution model and does not require CPU assistance or staging copies in host memory. As we will see, GGAS helps to process collective operations among distributed GPUs in-core. In this paper, we introduce the implementation and optimization of collective reduce

Author

Lena Oden; Benjamin Klenk; Holger Froning

Publisher

IEEE

Published

2014

Language

EN