Can I read In-situ I/O processing: a case for location flexibility on EtoBox?

In-situ I/O processing: a case for location flexibility by Fang Zheng; Hasan Abbasi; Jianting Cao; Jai Dayal; Karsten Schwan; Matthew Wolf; Scott Klasky; Norbert Podhorszki is a scholarly article available to read on EtoBox.

What is In-situ I/O processing: a case for location flexibility about?

Increasingly severe I/O bottlenecks on High-End Computing machines are prompting scientists to process output data during simulation time, "in-situ", and before placing data on disks. This paper argues for flexibility in the implementation of such in-situ data analytics, using measurements and a performance model that demonstrate the potential advantages and limitations of performing analytics at different levels of the I/O hierarchy, including on a machine's compute nodes vs. on separate "staging" nodes dedicated to analysis tasks. Model and measurement results are guided by realistic large-scale applications running on leadership class machines, and I/O and analytics actions are described as computational dataflow graphs -termed I/O graphsthat combine data movement with 'in transit' operations on data as it is being moved across the I/O hierarchy. Results demonstrate the importance of flexibility in analytics placement and characterize the attributes of analytics operations that lead to different placement decisions.

Author

Fang Zheng; Hasan Abbasi; Jianting Cao; Jai Dayal; Karsten Schwan; Matthew Wolf; Scott Klasky; Norbert Podhorszki

Publisher

ACM

Published

2011

Language

EN