IRevalOO: An Object Oriented Framework for Retrieval Evaluation

Roitero, Kevin, Maddalena, Eddy, Ponte, Yannick and Mizzaro, Stefano (2018) IRevalOO: An Object Oriented Framework for Retrieval Evaluation. In SIGIR 2018 - The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Association for Computing Machinery. pp. 913-916 . (doi:10.1145/3209978.3210084).

Abstract

We propose IRevalOO, a flexible Object Oriented framework that (i) can be used as-is as a replacement of the widely adopted trec_eval software, and (ii) can be easily extended (or "instantiated", in framework terminology) to implement different scenarios of test collection based retrieval evaluation. Instances of IRevalOO can provide a usable and convenient alternative to the state-of-the-art software commonly used by different initiatives (TREC, NTCIR, CLEF, FIRE, etc.). Also, those instances can be easily adapted to satisfy future customization needs of researchers, as: implementing and experimenting with new metrics, even based on new notions of relevance; using different formats for system output and "qrels"; and in general visualizing, comparing, and managing retrieval evaluation results.

This record has no associated files available for download.

Contributors

Download statistics