Interpretation of Crowdsourced Activities Using Provenance Network Analysis
Interpretation of Crowdsourced Activities Using Provenance Network Analysis
Understanding the dynamics of a crowdsourcing application and controlling the quality of the data it generates is challenging, partly due to the lack of tools to do so. Provenance is a domain-independent means to represent what happened in an application, which can help verify data and infer their quality. It can also reveal the processes that led to a data item and the interactions of contributors with it. Provenance patterns can manifest real-world phenomena such as a significant interest in a piece of content, providing an indication of its quality, or even issues such as undesirable interactions within a group of contributors. This paper presents an application-independent methodology for analysing provenance graphs, constructed from provenance records, to learn about such patterns and to use them for assessing some key properties of crowdsourced data, such as their quality, in an automated manner. Validating this method on the provenance records of CollabMap, an online crowdsourcing mapping application, we demonstrated an accuracy level of over 95% for the trust classification of data generated by the crowd therein.
978-1-57735-607-3
Huynh, Trung Dong
ddea6cf3-5a82-4c99-8883-7c31cf22dd36
Ebden, Mark
f46be90b-365e-4ea3-909a-4b92e4287f68
Venanzi, Matteo
ba24a77f-31a6-4c05-a647-babf8f660440
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3
Roberts, Stephen
fef5d01c-92bd-44cf-93f0-923ec24f8875
Moreau, Luc
033c63dd-3fe9-4040-849f-dfccbe0406f8
November 2013
Huynh, Trung Dong
ddea6cf3-5a82-4c99-8883-7c31cf22dd36
Ebden, Mark
f46be90b-365e-4ea3-909a-4b92e4287f68
Venanzi, Matteo
ba24a77f-31a6-4c05-a647-babf8f660440
Ramchurn, Sarvapali
1d62ae2a-a498-444e-912d-a6082d3aaea3
Roberts, Stephen
fef5d01c-92bd-44cf-93f0-923ec24f8875
Moreau, Luc
033c63dd-3fe9-4040-849f-dfccbe0406f8
Huynh, Trung Dong, Ebden, Mark, Venanzi, Matteo, Ramchurn, Sarvapali, Roberts, Stephen and Moreau, Luc
(2013)
Interpretation of Crowdsourced Activities Using Provenance Network Analysis.
The First AAAI Conference on Human Computation and Crowdsourcing, Palm Springs, United States.
06 - 09 Nov 2013.
Record type:
Conference or Workshop Item
(Paper)
Abstract
Understanding the dynamics of a crowdsourcing application and controlling the quality of the data it generates is challenging, partly due to the lack of tools to do so. Provenance is a domain-independent means to represent what happened in an application, which can help verify data and infer their quality. It can also reveal the processes that led to a data item and the interactions of contributors with it. Provenance patterns can manifest real-world phenomena such as a significant interest in a piece of content, providing an indication of its quality, or even issues such as undesirable interactions within a group of contributors. This paper presents an application-independent methodology for analysing provenance graphs, constructed from provenance records, to learn about such patterns and to use them for assessing some key properties of crowdsourced data, such as their quality, in an automated manner. Validating this method on the provenance records of CollabMap, an online crowdsourcing mapping application, we demonstrated an accuracy level of over 95% for the trust classification of data generated by the crowd therein.
Text
provanalytics.pdf
- Accepted Manuscript
More information
Published date: November 2013
Venue - Dates:
The First AAAI Conference on Human Computation and Crowdsourcing, Palm Springs, United States, 2013-11-06 - 2013-11-09
Organisations:
Web & Internet Science, Agents, Interactions & Complexity
Identifiers
Local EPrints ID: 357199
URI: http://eprints.soton.ac.uk/id/eprint/357199
ISBN: 978-1-57735-607-3
PURE UUID: c4dc9b17-2e87-4a5f-a4bc-02c8900b52e3
Catalogue record
Date deposited: 23 Sep 2013 10:23
Last modified: 09 Apr 2020 00:29
Export record
Contributors
Author:
Trung Dong Huynh
Author:
Mark Ebden
Author:
Matteo Venanzi
Author:
Sarvapali Ramchurn
Author:
Stephen Roberts
Author:
Luc Moreau
University divisions
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics