Can I read Principles for Robust Evaluation Infrastructure on EtoBox?

Principles for Robust Evaluation Infrastructure by Justin Zobel; William Webber; Mark Sanderson; Alistair Moffat is a scholarly article available to read on EtoBox.

What is Principles for Robust Evaluation Infrastructure about?

The standard "Cranfield" approach to the evaluation of information retrieval systems has been used and refined for nearly fifty years, and has been a key element in the development of large-scale retrieval systems. The resources created by such systematic evaluations have enabled thorough retrospective investigation of the strengths and limitations of particular variants of this evaluation approach; over the last few years, such investigation has for example led to identification of serious flaws in some experiments. Knowledge of these flaws can prevent their perpetuation into future work and informs the design of new experiments and infrastructures. In this position statement we briefly review some aspects of evaluation and, based on our research and observations over the last decade, outline some principles on which we believe new infrastructure should rest.

Author

Justin Zobel; William Webber; Mark Sanderson; Alistair Moffat

Publisher

ACM

Published

2011

Language

EN