A systematic analysis of performance measures for classification tasks |
| |
| Authors: | Marina Sokolova Guy Lapalme |
| |
| Institution: | 1. Electronic Health Information Lab, Children’s Hospital of Eastern Ontario, Ottawa, Canada;2. Département d’informatique et de recherche opérationnelle Université de Montréal, Montréal, Canada |
| |
| Abstract: | This paper presents a systematic analysis of twenty four performance measures used in the complete spectrum of Machine Learning classification tasks, i.e., binary, multi-class, multi-labelled, and hierarchical. For each classification task, the study relates a set of changes in a confusion matrix to specific characteristics of data. Then the analysis concentrates on the type of changes to a confusion matrix that do not change a measure, therefore, preserve a classifier’s evaluation (measure invariance). The result is the measure invariance taxonomy with respect to all relevant label distribution changes in a classification problem. This formal analysis is supported by examples of applications where invariance properties of measures lead to a more reliable evaluation of classifiers. Text classification supplements the discussion with several case studies. |
| |
| Keywords: | Performance evaluation Machine Learning Text classification |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
