Can I read Classifier Selection with Permutation Tests on EtoBox?

Classifier Selection with Permutation Tests by Arias, Marta; Arratia, Argimiro; Duarte-Lopez, Ariel is a scholarly article available to read on EtoBox.

What is Classifier Selection with Permutation Tests about?

This work presents a content-based recommender system for machine learning classifier algorithms. Given a new data set, a recommendation of what classifier is likely to perform best is made based on classifier performance over similar known data sets. This similarity is measured according to a data set characterization that includes several state-of-the-art metrics taking into account physical structure, statis- tics, and information theory. A novelty with respect to prior work is the use of a robust approach based on permutation tests to directly assess whether a given learning algorithm is able to exploit the attributes in a data set to predict class labels, and compare it to the more commonly used F-score metric for evalu- ating classifier performance. To evaluate our approach, we have conducted an extensive experimentation including 8 of the main machine learning classification methods with varying configurations and 65 bi- nary data sets, leading to over 2331 experiments. Our results show that using the information from the permutation test clearly improves the quality of the recommendations.

Author

Arias, Marta; Arratia, Argimiro; Duarte-Lopez, Ariel

Published

2017

Language

EN