Can I read Crowd-Certain: Label Aggregation in Crowdsourced and Ensemble Learning Classification on EtoBox?

Crowd-Certain: Label Aggregation in Crowdsourced and Ensemble Learning Classification by Majdi, Mohammad S.; Rodriguez, Jeffrey J. is a scholarly article available to read on EtoBox.

What is Crowd-Certain: Label Aggregation in Crowdsourced and Ensemble Learning Classification about?

Crowdsourcing systems have been used to accumulate massive amounts of labeled data for applications such as computer vision and natural language processing. However, because crowdsourced labeling is inherently dynamic and uncertain, developing a technique that can work in most situations is extremely challenging. In this paper, we introduce Crowd-Certain, a novel approach for label aggregation in crowdsourced and ensemble learning classification tasks that offers improved performance and computational efficiency for different numbers of annotators and a variety of datasets. The proposed method uses the consistency of the annotators versus a trained classifier to determine a reliability score for each annotator. Furthermore, Crowd-Certain leverages predicted probabilities, enabling the reuse of trained classifiers on future sample data, thereby eliminating the need for recurrent simulation processes inherent in existing methods. We extensively evaluated our approach against ten existing techniques across ten different datasets, each labeled by varying numbers of annotators. The findings demonstrate that Crowd-Certain outperforms the existing methods (Tao, Sheng, KOS, MACE, MajorityV

Author

Majdi, Mohammad S.; Rodriguez, Jeffrey J.

Published

2023

Language

EN