Can I read Scalable Stream Join Processing with Expensive Predicates: Workload Distribution and Adaptation by Time-slicing on EtoBox?

Scalable Stream Join Processing with Expensive Predicates: Workload Distribution and Adaptation by Time-slicing by Song Wang; Elke Rundensteiner is a scholarly article available to read on EtoBox.

What is Scalable Stream Join Processing with Expensive Predicates: Workload Distribution and Adaptation by Time-slicing about?

Multi-way stream joins with expensive join predicates lead to great challenge for real-time (or close to real-time) stream processing. Given the memory-and CPU-intensive nature of such stream join queries, scalable processing on a cluster must be employed. This paper proposes a novel scheme for distributed processing of generic multi-way joins with window constraints, called Pipelined State Partitioning (PSP). We target generic joins with arbitrarily join conditions, which are used in non-trivial stream applications such as image matching and biometric recognizing. The PSP scheme partitions the states into disjoint slices in the time domain, and then distributes the fine-grained states in the cluster, forming a virtual computation ring. Compared to replicationbased distribution of non-equi-joins, PSP scheme is superior since: (1) zero state duplication and thus no repeated computations, (2) pipelined processing of every input tuple on multiple nodes to achieve low response time, and (3) cost-based adaptive workload distribution. We have implemented the proposed PSP schemes within the CAPE DSMS. Our experimental study demonstrates the significant performance improvements compared to

Author

Song Wang; Elke Rundensteiner

Publisher

ACM

Published

2009

Language

EN