Debiasing crowdsourced quantitative characteristics in local businesses and services

Information about quantitative characteristics in local businesses and services, such as the number of people waiting in line in a cafe and the number of available fitness machines in a gym, is important for informed decision, crowd management and event detection. In this paper, we investigate the potential of leveraging crowds as sensors to report such quantitative characteristics and investigate how to recover the true quantity values from noisy crowdsourced information. Through experiments, we find that crowd sensors have both bias and variance in quantity sensing, and task difficulties impact the sensing accuracy. Based on these findings, we propose an unsupervised probabilistic model to jointly assess task difficulties, ability of crowd sensors and true quantity values. Our model differs from existing categorical truth finding models as ours is specifically designed to tackle quantitative truth. In addition to devising an efficient model inference algorithm in a batch mode, we also design an even faster online version for handling streaming data. Experimental results in various scenarios demonstrate the effectiveness of our model.

Crowdsourcing, Humans as sensors, Probabilistic graphical models, Truth discovery

10.1145/2737095.2737116

190-201

Association for Computing Machinery

Ouyang, Robin Wentao

a0886331-0eed-46ee-9b72-843ef2bb192f

Kaplan, Lance

a645f3d7-9b77-40fb-ada7-903bedb28a25

Martin, Paul

d56dd39b-d9dc-45e0-b2be-8af13deac0d6

Toniolo, Alice

e54ad578-9232-471a-a5d7-cd3a7bc70872

Srivastava, Mani

77c0ad90-6073-4dd8-908f-05087d856cd6

Norman, Timothy J.

663e522f-807c-4569-9201-dc141c8eb50d

13 April 2015

Ouyang, Robin Wentao

a0886331-0eed-46ee-9b72-843ef2bb192f

Kaplan, Lance

a645f3d7-9b77-40fb-ada7-903bedb28a25

Martin, Paul

d56dd39b-d9dc-45e0-b2be-8af13deac0d6

Toniolo, Alice

e54ad578-9232-471a-a5d7-cd3a7bc70872

Srivastava, Mani

77c0ad90-6073-4dd8-908f-05087d856cd6

Norman, Timothy J.

663e522f-807c-4569-9201-dc141c8eb50d

Ouyang, Robin Wentao, Kaplan, Lance, Martin, Paul, Toniolo, Alice, Srivastava, Mani and Norman, Timothy J. (2015) Debiasing crowdsourced quantitative characteristics in local businesses and services. In IPSN 2015 - Proceedings of the 14th International Symposium on Information Processing in Sensor Networks (Part of CPS Week). Association for Computing Machinery. pp. 190-201 . (doi:10.1145/2737095.2737116).

Record type: Conference or Workshop Item (Paper)

Abstract

This record has no associated files available for download.

More information

Published date: 13 April 2015

Additional Information: Funding Information: This research is based upon work supported in part by the U.S. ARL and U.K. Ministry of Defense under Agreement Number W911NF-06-3-0001, and by the NSF under award CNS-1213140. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views or represent the official policies of the NSF, the U.S. ARL, the U.S. Government, the U.K. Ministry of Defense or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. Publisher Copyright: Copyright 2015 ACM. Copyright: Copyright 2016 Elsevier B.V., All rights reserved.

Venue - Dates: 14th International Symposium on Information Processing in Sensor Networks, IPSN 2015, , Seattle, United States, 2015-04-13 - 2015-04-16

Keywords: Crowdsourcing, Humans as sensors, Probabilistic graphical models, Truth discovery

Learn more about Agents, Interactions and Complexity research