The University of Southampton
University of Southampton Institutional Repository

Getting by with a little help from the crowd: optimal human computation approaches to social image labeling

Loni, Babak, Hare, Jonathon, Georgescu, Mihai, Riegler, Michael, Morchid, Mohamed, Dufour, Richard and Larson, Martha (2014) Getting by with a little help from the crowd: optimal human computation approaches to social image labeling At CrowdMM 2014, United States. 03 - 07 Nov 2014. 6 pp. (doi:10.1145/2660114.2660123).

Record type: Conference or Workshop Item (Paper)


Validating user tags helps to refine them, making them more useful for finding images. In the case of interpretation-sensitive tags, however, automatic (i.e., pixel-based) approaches cannot be expected to deliver optimal results. Instead, human input is key. This paper studies how crowdsourcing-based approaches to image tag validation can achieve parsimony in their use of human input from the crowd, in the form of votes collected from workers on a crowdsourcing platform. Experiments in the domain of social fashion images are carried out using the dataset published by the Crowdsourcing Task of the Mediaeval 2013 Multimedia Benchmark. Experimental results reveal that when a larger number of crowd-contributed votes are available, it is difficult to beat a majority vote. However, additional information sources, i.e., crowdworker history and visual image features, allow us to maintain similar validation performance while making use of less crowd-contributed input. Further, investing in “expensive" experts who collaborate to create definitions of interpretation-sensitive concepts does not nec- essarily pay off. Instead, experts can cause interpretations of concepts to drift away from conventional wisdom. In short, validation of interpretation-sensitive user tags for social images is possible, with “just a little help from the crowd."

PDF GettingByWithLittleHelp_CrowdMM.pdf - Accepted Manuscript
Download (1MB)

More information

Published date: November 2014
Venue - Dates: CrowdMM 2014, United States, 2014-11-03 - 2014-11-07
Organisations: Web & Internet Science


Local EPrints ID: 370278
PURE UUID: 3563ac04-126a-4cb0-9e74-e1861a69f9f3
ORCID for Jonathon Hare: ORCID iD

Catalogue record

Date deposited: 20 Oct 2014 17:50
Last modified: 17 Jul 2017 21:52

Export record



Author: Babak Loni
Author: Jonathon Hare ORCID iD
Author: Mihai Georgescu
Author: Michael Riegler
Author: Mohamed Morchid
Author: Richard Dufour
Author: Martha Larson

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton:

ePrints Soton supports OAI 2.0 with a base URL of

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.