The University of Southampton
University of Southampton Institutional Repository

IRevalOO: An Object Oriented Framework for Retrieval Evaluation

IRevalOO: An Object Oriented Framework for Retrieval Evaluation
IRevalOO: An Object Oriented Framework for Retrieval Evaluation
We propose IRevalOO, a flexible Object Oriented framework that (i) can be used as-is as a replacement of the widely adopted trec_eval software, and (ii) can be easily extended (or "instantiated", in framework terminology) to implement different scenarios of test collection based retrieval evaluation. Instances of IRevalOO can provide a usable and convenient alternative to the state-of-the-art software commonly used by different initiatives (TREC, NTCIR, CLEF, FIRE, etc.). Also, those instances can be easily adapted to satisfy future customization needs of researchers, as: implementing and experimenting with new metrics, even based on new notions of relevance; using different formats for system output and "qrels"; and in general visualizing, comparing, and managing retrieval evaluation results.
TREC, evaluation, test collections, trec_eval
913-916
ACM Press
Roitero, Kevin
71dbbb60-a1e9-431e-930d-fbd498c6559f
Maddalena, Eddy
397dbaba-4363-4c11-8e52-4a7ba4df4bae
Ponte, Yannick
dc508b63-69d3-44a3-8b8e-5ac271fb6a62
Mizzaro, Stefano
7be30144-afe5-42bb-a861-59a451decc20
Roitero, Kevin
71dbbb60-a1e9-431e-930d-fbd498c6559f
Maddalena, Eddy
397dbaba-4363-4c11-8e52-4a7ba4df4bae
Ponte, Yannick
dc508b63-69d3-44a3-8b8e-5ac271fb6a62
Mizzaro, Stefano
7be30144-afe5-42bb-a861-59a451decc20

Roitero, Kevin, Maddalena, Eddy, Ponte, Yannick and Mizzaro, Stefano (2018) IRevalOO: An Object Oriented Framework for Retrieval Evaluation. In SIGIR 2018 - The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM Press. pp. 913-916 . (doi:10.1145/3209978.3210084).

Record type: Conference or Workshop Item (Paper)

Abstract

We propose IRevalOO, a flexible Object Oriented framework that (i) can be used as-is as a replacement of the widely adopted trec_eval software, and (ii) can be easily extended (or "instantiated", in framework terminology) to implement different scenarios of test collection based retrieval evaluation. Instances of IRevalOO can provide a usable and convenient alternative to the state-of-the-art software commonly used by different initiatives (TREC, NTCIR, CLEF, FIRE, etc.). Also, those instances can be easily adapted to satisfy future customization needs of researchers, as: implementing and experimenting with new metrics, even based on new notions of relevance; using different formats for system output and "qrels"; and in general visualizing, comparing, and managing retrieval evaluation results.

This record has no associated files available for download.

More information

Published date: 27 June 2018
Keywords: TREC, evaluation, test collections, trec_eval

Identifiers

Local EPrints ID: 423225
URI: http://eprints.soton.ac.uk/id/eprint/423225
PURE UUID: e5dafb57-5e01-429c-82da-a39c6a3ce741

Catalogue record

Date deposited: 19 Sep 2018 16:30
Last modified: 22 Jul 2022 16:55

Export record

Altmetrics

Contributors

Author: Kevin Roitero
Author: Eddy Maddalena
Author: Yannick Ponte
Author: Stefano Mizzaro

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×