A second look at Egghe’s universal IR surface and a simple derivation of a complete set of universal IR evaluation points |
| |
| Authors: | Marc Schatkun |
| |
| Institution: | a22707 Lull St., West Hills, CA 91304, USA |
| |
| Abstract: | Egghe’s three papers regarding the universal IR surface (2004, 2007, 2008) clearly represent an original and significant contribution to the IR evaluation literature. However, Egghe’s attempt to find a complete set of universal IR evaluation points (P,R,F,M) fell short of his goal: his universal IR surface equation did not suffice in and of itself, and his continuous extension argument was insufficient to find all the remaining points (quadruples). Egghe found only two extra universal IR evaluation points, (1,1,0,0) and (0,0,1,1), but it turns out that a total of 15 additional, valid, universal IR evaluation points exist. The gap first appeared in Egghe’s earliest paper and was carried into subsequent papers. The mathematical method used here for finding the additional universal IR evaluation points involves defining the relevance metrics P,R,F,M in terms of the Swets variables a,b,c,d. Then the maximum possible number of additional quadruples is deduced, and finally, all the invalid quadruples are eliminated so that only the valid, universal IR points remain. Six of these points may be interpreted as being continuous extensions of the universal IR surface, while the other nine points may be interpreted as being “off the universal IR surface.” This completely solves the problem of finding the maximum range possible of universal IR evaluation points. |
| |
| Keywords: | IR evaluation points Universal IR surface Evaluation Precision Recall Fallout Miss |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
