Why current IR engines fail |
| |
| Authors: | Chris Buckley |
| |
| Institution: | (1) Sabir Research Inc., Gaithersburg, MD, USA |
| |
| Abstract: | Observations from a unique investigation of failure analysis of Information Retrieval research engines held in 2003 are presented.
The Reliable Information Access Workshop invited seven leading IR research groups to supply both their systems and their experts
to an effort to analyze why their systems fail on some topics and whether the failures are due to system flaws, approach flaws,
or the topic itself. There were surprising results from this cross-system failure analysis. One is that despite systems retrieving
very different documents, the major cause of failure for any particular topic was almost always the same across all systems.
Another is that relationships between aspects of a topic are not especially important for state-of-the-art systems; the systems
are failing at a much more basic level where the top-retrieved documents are not reflecting some aspect at all. The investigatory
framework and the lessons learned can serve as a model for needed future research in this area. |
| |
| Keywords: | |
| 本文献已被 SpringerLink 等数据库收录！ |
|
