Can I read More on Finding a Single Number to Indicate Overall Performance of a Benchmark Suite on EtoBox?

More on Finding a Single Number to Indicate Overall Performance of a Benchmark Suite by Lizy Kurian John is a Computer Science article available to read on EtoBox.

What is More on Finding a Single Number to Indicate Overall Performance of a Benchmark Suite about?

The topic of finding a single number to summarize overall performance over a benchmark suite is continuing to be a difficult issue 14 years after Smith's paper . While significant insight into the problem has been provided by Smith [1], Hennessey and Patterson [2], Cragon [3], etc, the research community still seems to be unclear on the correct mean to use for different performance metrics. How should metrics obtained from individual benchmarks be aggregated to present a summary of the performance over the entire suite? What are valid central tendency measures over the whole benchmark suite for speedup, CPI, IPC, MIPS, MFLOPS, cache miss rates, cache hit rates, branch misprediction rates, etc? Arithmetic mean has been touted to be appropriate for time-based metrics, while harmonic mean is touted to be appropriate for rate-based metrics. Is cache miss rate a rate-based metric and hence is harmonic mean appropriate? Geometric mean is a valid measure of central tendency for ratios or dimensionless quantities [3], however, it is also advised that geometric mean should not be used for summarizing any performance measure . Speedup, which is a popular metric in most architecture papers to

Who reads More on Finding a Single Number to Indicate Overall Performance of a Benchmark Suite?

Author

Lizy Kurian John

Publisher

ACM

Published

2004

Language

EN

Field

Computer Science (Physical Sciences)