The ActiveCrowdToolkit: An Open-Source Tool for Benchmarking Active Learning Algorithms for Crowdsourcing Research


Venanzi, Matteo, Parson, Oliver, Rogers, Alex and Jennings, Nicholas R. (2015) The ActiveCrowdToolkit: An Open-Source Tool for Benchmarking Active Learning Algorithms for Crowdsourcing Research At The Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP-2015).

Download

[img] PDF main.pdf - Other
Download (3MB)
[img] PDF active_crowd_toolkit_short.pdf - Accepted Manuscript
Download (1MB)

Description/Abstract

We present an open-source toolkit that allows the easy comparison of the performance of active learning methods over a series of datasets. The toolkit allows such strategies to be constructed by combining a judgement aggregation model, task selection method and worker selection method. The toolkit also provides a user interface which allows researchers to gain insight into worker performance and task classification at runtime.

Item Type: Conference or Workshop Item (Paper)
Venue - Dates: The Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP-2015), 2015-08-18
Organisations: Agents, Interactions & Complexity
ePrint ID: 380666
Date :
Date Event
18 August 2015Published
Date Deposited: 18 Aug 2015 11:01
Last Modified: 22 Feb 2017 08:18
Projects:
HUMAN-AGENT COLLECTIVES: FROM FOUNDATIONS TO APPLICATIONS [ORCHID]
Funded by: EPSRC (EP/I011587/1)
Led by: Nick Jennings
1 January 2011 to 31 December 2015
Further Information:Google Scholar
URI: http://eprints.soton.ac.uk/id/eprint/380666

Actions (login required)

View Item View Item