Can I read Parallel Query Processing: To Separate Communication from Computation on EtoBox?

Parallel Query Processing: To Separate Communication from Computation by Hao Zhang; Jeffrey Xu Yu; Yikai Zhang; Kangfei Zhao is a scholarly article available to read on EtoBox.

What is Parallel Query Processing: To Separate Communication from Computation about?

In this paper, we study parallel query processing with a focus on reducing the communication cost, which is the dominating factor in parallel query processing. The communication cost becomes large if the intermediate results between operators are large in intra-operator parallelism. In the existing approaches, it optimizes an SQL query by arranging relational algebra operators to reduce the total cost, where, for each operator, it involves (i) distribution of data partitioned to computing nodes by communication, and (ii) computation on computing nodes locally. The communication and computation are dealt with inside an operator and are not separable. In other words, it is difficult to avoid large intermediate results and hence reduce the communication cost. To reduce communication cost, we separate communication from computation using several new operators proposed in this paper. One is a pair operator (⊗) to pair the partitions of a relation R with the partitions of a relation S, where a partition is specified by a hash function. With the pair operator defined, we can explicitly deal with communication to deliver pairs of partitions to computing nodes. Together with ⊗, we can also

Author

Hao Zhang; Jeffrey Xu Yu; Yikai Zhang; Kangfei Zhao

Publisher

ACM

Published

2022

Language

EN