Can I read Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11 - The effects of choice in routing relevance judgments on EtoBox?

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11 - The effects of choice in routing relevance judgments by Law, Edith; Bennett, Paul N.; Horvitz, Eric is a scholarly article available to read on EtoBox.

What is Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11 - The effects of choice in routing relevance judgments about?

The emergence of human computation systems, including Mechanical Turk and games with a purpose, has made it feasible to distribute relevance judgment tasks to workers over the Web. Most human computation systems assign tasks to individuals randomly, and such assignments may match workers with tasks that they may be unqualified or unmotivated to perform. We compare two groups of workers, those given a choice of queries to judge versus those who are not, in terms of their self-rated competence and their actual performance. Results show that when given a choice of task, workers choose ones for which they have greater expertise, interests, confidence, and understanding.

Author

Law, Edith; Bennett, Paul N.; Horvitz, Eric

Publisher

ACM Press

Published

2011

Language

EN