The University of Southampton
University of Southampton Institutional Repository

What prize is right? How to learn the optimal structure for crowdsourcing contests

What prize is right? How to learn the optimal structure for crowdsourcing contests
What prize is right? How to learn the optimal structure for crowdsourcing contests
In crowdsourcing, one effective method for encouraging participants to perform tasks is to run contests where participants compete against each other for rewards. However, there are numerous ways to implement such contests in specific projects. They could vary in their structure (e.g., performance evaluation and the number of prizes) and parameters (e.g., the maximum number of participants and the amount of prize money). Additionally, with a given budget and a time limit, choosing incentives (i.e., contest structures with specific parameter values) that maximise the overall utility is not trivial, as their respective effectiveness in a specific project is usually unknown a priori. Thus, in this paper, we propose a novel algorithm, BOIS (Bayesian-optimisation-based incentive selection), to learn the optimal structure and tune its parameters effectively. In detail, the learning and tuning problems are solved simultaneously by using online learning in combination with Bayesian optimisation. The results of our extensive simulations show that the performance of our algorithm is up to 85% of the optimal and up to 63% better than state-of-the-art benchmarks.
Incentive, Crowdsourcing, Bayesian optimisation
85-97
Springer
Truong, Nhat, Van Quoc
ea0089af-c714-4d31-91dc-e7a3ab7a8bde
Stein, Sebastian
cb2325e7-5e63-475e-8a69-9db2dfbdb00b
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Jennings, Nick
0d0a0add-2739-4521-8915-c1e5a1e320b5
Nayak, Abhaya
Sharma, Alok
Truong, Nhat, Van Quoc
ea0089af-c714-4d31-91dc-e7a3ab7a8bde
Stein, Sebastian
cb2325e7-5e63-475e-8a69-9db2dfbdb00b
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Jennings, Nick
0d0a0add-2739-4521-8915-c1e5a1e320b5
Nayak, Abhaya
Sharma, Alok

Truong, Nhat, Van Quoc, Stein, Sebastian, Tran-Thanh, Long and Jennings, Nick (2019) What prize is right? How to learn the optimal structure for crowdsourcing contests. Nayak, Abhaya and Sharma, Alok (eds.) In PRICAI 2019: Trends in Artificial Intelligence. vol. 1160, Springer. pp. 85-97 . (doi:10.1007/978-3-030-29908-8_7).

Record type: Conference or Workshop Item (Paper)

Abstract

In crowdsourcing, one effective method for encouraging participants to perform tasks is to run contests where participants compete against each other for rewards. However, there are numerous ways to implement such contests in specific projects. They could vary in their structure (e.g., performance evaluation and the number of prizes) and parameters (e.g., the maximum number of participants and the amount of prize money). Additionally, with a given budget and a time limit, choosing incentives (i.e., contest structures with specific parameter values) that maximise the overall utility is not trivial, as their respective effectiveness in a specific project is usually unknown a priori. Thus, in this paper, we propose a novel algorithm, BOIS (Bayesian-optimisation-based incentive selection), to learn the optimal structure and tune its parameters effectively. In detail, the learning and tuning problems are solved simultaneously by using online learning in combination with Bayesian optimisation. The results of our extensive simulations show that the performance of our algorithm is up to 85% of the optimal and up to 63% better than state-of-the-art benchmarks.

Text
pricai19 - Accepted Manuscript
Download (1MB)

More information

Accepted/In Press date: 20 April 2019
e-pub ahead of print date: 23 August 2019
Venue - Dates: The 16th Pacific Rim International Conference on Artificial Intelligence, , Yanuca Island, Fiji, 2019-08-26 - 2019-08-30
Keywords: Incentive, Crowdsourcing, Bayesian optimisation

Identifiers

Local EPrints ID: 432967
URI: http://eprints.soton.ac.uk/id/eprint/432967
PURE UUID: 47a1e723-4e88-491b-ac33-f184cf3d87b2
ORCID for Nhat, Van Quoc Truong: ORCID iD orcid.org/0000-0003-4945-8197
ORCID for Sebastian Stein: ORCID iD orcid.org/0000-0003-2858-8857
ORCID for Long Tran-Thanh: ORCID iD orcid.org/0000-0003-1617-8316

Catalogue record

Date deposited: 05 Aug 2019 16:30
Last modified: 16 Mar 2024 03:57

Export record

Altmetrics

Contributors

Author: Nhat, Van Quoc Truong ORCID iD
Author: Sebastian Stein ORCID iD
Author: Long Tran-Thanh ORCID iD
Author: Nick Jennings
Editor: Abhaya Nayak
Editor: Alok Sharma

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×