Can I read Achieving Performance Balance Among Spark Frameworks with Two-Level Schedulers on EtoBox?

Achieving Performance Balance Among Spark Frameworks with Two-Level Schedulers by Aleksandra Kuzmanovska; Hans van Den Bogert; Rudolf Mak; Dick Epema is a scholarly article available to read on EtoBox.

What is Achieving Performance Balance Among Spark Frameworks with Two-Level Schedulers about?

When multiple data-processing frameworks with time-varying workloads are simultaneously present in a single cluster or data-center, an apparent goal is to have them experience equal performance, expressed in whatever performance metrics are applicable. In modern data-center environments, Two-Level Schedulers (TLSs) that leave the scheduling of individual jobs to the schedulers within the data-processing frameworks are typically used for managing the resources of data-processing frameworks. Two such TLSs with opposite designs are Mesos and Koala-F. Mesos employs fine-grained resource allocation and aims at Dominant Resource Fairness (DRF) among framework instances by offering resources to them for the duration of a single task. In contrast, Koala-F aims at performance fairness among framework instances by employing dynamic coarsegrained resource allocation of sets of complete nodes based on performance feedback from individual instances. The goal of this paper is to explore the trade-offs between these two TLS designs when trying to achieve performance balance among frameworks. We select Apache Spark as a representative of data-processing frameworks, and perform experiments on a mod

Author

Aleksandra Kuzmanovska; Hans van Den Bogert; Rudolf Mak; Dick Epema

Publisher

IEEE

Published

2018

Language

EN