The University of Southampton
University of Southampton Institutional Repository

Streaming Bayesian inference for crowdsourced classification

Streaming Bayesian inference for crowdsourced classification
Streaming Bayesian inference for crowdsourced classification
A key challenge in crowdsourcing is inferring the ground truth from noisy and unreliable data. To do so, existing approaches rely on collecting redundant information from the crowd, and aggregating it with some probabilistic method. However, oftentimes such methods are computationally inefficient, are restricted to some specific settings, or lack theoretical guarantees. In this paper, we revisit the problem of binary classification from crowdsourced data. Specifically we propose Streaming Bayesian Inference for Crowdsourcing (SBIC), a new algorithm that does not suffer from any of these limitations. First, SBIC has low complexity and can be used in a real-time online setting. Second, SBIC has the same accuracy as the best state-of-the-art algorithms in all settings. Third, SBIC has provable asymptotic guarantees both in the online and offline settings.
Manino, Edoardo
e5cec65c-c44b-45de-8255-7b1d8edfc04d
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Jennings, Nicholas
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Manino, Edoardo
e5cec65c-c44b-45de-8255-7b1d8edfc04d
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Jennings, Nicholas
ab3d94cc-247c-4545-9d1e-65873d6cdb30

Manino, Edoardo, Tran-Thanh, Long and Jennings, Nicholas (2019) Streaming Bayesian inference for crowdsourced classification. 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), , Vancouver, Canada. 08 - 14 Dec 2019. 11 pp .

Record type: Conference or Workshop Item (Paper)

Abstract

A key challenge in crowdsourcing is inferring the ground truth from noisy and unreliable data. To do so, existing approaches rely on collecting redundant information from the crowd, and aggregating it with some probabilistic method. However, oftentimes such methods are computationally inefficient, are restricted to some specific settings, or lack theoretical guarantees. In this paper, we revisit the problem of binary classification from crowdsourced data. Specifically we propose Streaming Bayesian Inference for Crowdsourcing (SBIC), a new algorithm that does not suffer from any of these limitations. First, SBIC has low complexity and can be used in a real-time online setting. Second, SBIC has the same accuracy as the best state-of-the-art algorithms in all settings. Third, SBIC has provable asymptotic guarantees both in the online and offline settings.

Text
Paper v4 - Version of Record
Download (404kB)
Text
Appendix v4
Download (203kB)

More information

Accepted/In Press date: 2019
e-pub ahead of print date: December 2019
Venue - Dates: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), , Vancouver, Canada, 2019-12-08 - 2019-12-14

Identifiers

Local EPrints ID: 435912
URI: http://eprints.soton.ac.uk/id/eprint/435912
PURE UUID: 988e738c-4142-42ac-971b-8c9e27f8474f
ORCID for Edoardo Manino: ORCID iD orcid.org/0000-0003-0028-5440
ORCID for Long Tran-Thanh: ORCID iD orcid.org/0000-0003-1617-8316

Catalogue record

Date deposited: 22 Nov 2019 17:30
Last modified: 19 Nov 2020 17:30

Export record

Contributors

Author: Edoardo Manino ORCID iD
Author: Long Tran-Thanh ORCID iD
Author: Nicholas Jennings

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×