The ActiveCrowdToolkit: an open-source tool for benchmarking active learning algorithms for crowdsourcing research
The ActiveCrowdToolkit: an open-source tool for benchmarking active learning algorithms for crowdsourcing research
We present an open-source toolkit that allows the easy comparison of the performance of active learning methods over a series of datasets. The toolkit allows such strategies to be constructed by combining a judgement aggregation model, task selection method and worker selection method. The toolkit also provides a user interface which allows researchers to gain insight into worker performance and task classification at runtime.
Venanzi, Matteo
ba24a77f-31a6-4c05-a647-babf8f660440
Parson, Oliver
9630bcd4-3d91-4b2a-b94a-24bdb84efab6
Rogers, Alex
f9130bc6-da32-474e-9fab-6c6cb8077fdc
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
18 August 2015
Venanzi, Matteo
ba24a77f-31a6-4c05-a647-babf8f660440
Parson, Oliver
9630bcd4-3d91-4b2a-b94a-24bdb84efab6
Rogers, Alex
f9130bc6-da32-474e-9fab-6c6cb8077fdc
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Venanzi, Matteo, Parson, Oliver, Rogers, Alex and Jennings, Nicholas R.
(2015)
The ActiveCrowdToolkit: an open-source tool for benchmarking active learning algorithms for crowdsourcing research.
The Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP-2015), , San Diego, United States.
08 - 11 Nov 2015.
10 pp
.
Record type:
Conference or Workshop Item
(Paper)
Abstract
We present an open-source toolkit that allows the easy comparison of the performance of active learning methods over a series of datasets. The toolkit allows such strategies to be constructed by combining a judgement aggregation model, task selection method and worker selection method. The toolkit also provides a user interface which allows researchers to gain insight into worker performance and task classification at runtime.
Text
active crowd toolkit short.pdf
- Accepted Manuscript
More information
Published date: 18 August 2015
Venue - Dates:
The Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP-2015), , San Diego, United States, 2015-11-08 - 2015-11-11
Organisations:
Agents, Interactions & Complexity
Identifiers
Local EPrints ID: 380666
URI: http://eprints.soton.ac.uk/id/eprint/380666
PURE UUID: 3b7ff677-fd0a-4d69-ae56-15f489e09171
Catalogue record
Date deposited: 18 Aug 2015 11:01
Last modified: 01 Mar 2023 17:48
Export record
Contributors
Author:
Matteo Venanzi
Author:
Oliver Parson
Author:
Alex Rogers
Author:
Nicholas R. Jennings
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics