Can I read Cooperative Minibatching in Graph Neural Networks on EtoBox?

Cooperative Minibatching in Graph Neural Networks by Balin, Muhammed Fatih; LaSalle, Dominique; Çatalyürek, Ümit V. is a scholarly article available to read on EtoBox.

What is Cooperative Minibatching in Graph Neural Networks about?

Training large scale Graph Neural Networks (GNNs) requires significant computational resources, and the process is highly data-intensive. One of the most effective ways to reduce resource requirements is minibatch training coupled with graph sampling. GNNs have the unique property that items in a minibatch have overlapping data. However, the commonly implemented Independent Minibatching approach assigns each Processing Element (PE, i.e., cores and/or GPUs) its own minibatch to process, leading to duplicated computations and input data access across PEs. This amplifies the Neighborhood Explosion Phenomenon (NEP), which is the main bottleneck limiting scaling. To reduce the effects of NEP in the multi-PE setting, we propose a new approach called Cooperative Minibatching. Our approach capitalizes on the fact that the size of the sampled subgraph is a concave function of the batch size, leading to significant reductions in the amount of work as batch sizes increase. Hence, it is favorable for processors equipped with a fast interconnect to work on a large minibatch together as a single larger processor, instead of working on separate smaller minibatches, even though global batch size i

Author

Balin, Muhammed Fatih; LaSalle, Dominique; Çatalyürek, Ümit V.

Published

2023

Language

EN