The University of Southampton
University of Southampton Institutional Repository

Semantic face signatures: recognizing and retrieving faces by verbal descriptions

Semantic face signatures: recognizing and retrieving faces by verbal descriptions
Semantic face signatures: recognizing and retrieving faces by verbal descriptions
The adverse visual conditions of surveillance environments,and the need to identify humans at a distance,have stimulated research in soft biometric attributes. Theseattributes can be used to describe a human’s physical traitssemantically and can be acquired without their cooperation.Soft biometrics can also be employed to retrieve identity from adatabase using verbal descriptions for suspects. In this paper, weexplore unconstrained human face identification with semanticface attributes derived automatically from images. The processuses a deformable face model with keypoint localisation whichis aligned with attributes derived by semantic description. Ournew framework exploits the semantic feature space to inferface signatures from images, and bridges the semantic gapbetween humans and machines with respect to face attributes.We use an unconstrained dataset, LFW-MS4, consisting of allthe subjects from View-1 of the LFW database that have 4 ormore samples. Our new approach demonstrates that retrievalvia estimated comparative facial soft biometrics yields a matchin the top 10.23% of returned subjects. Furthermore, modellingof face image features in the semantic space can achieve anequal error rate of 12.71%. These results reveal the latentbenefits of modelling visual facial features in a semantic space.Moreover, they highlight the potential of using images and verbaldescriptions to generate comparative soft biometrics for subjectidentification and retrieval.
1556-6013
706 - 716
Almudhahka, Nawaf, Yousef
929b4dbb-016d-44bb-9755-b9e0adb6ded0
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Almudhahka, Nawaf, Yousef
929b4dbb-016d-44bb-9755-b9e0adb6ded0
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9

Almudhahka, Nawaf, Yousef, Nixon, Mark and Hare, Jonathon (2018) Semantic face signatures: recognizing and retrieving faces by verbal descriptions. IEEE Transactions on Information Forensics and Security, 13 (3), 706 - 716. (doi:10.1109/TIFS.2017.2765519).

Record type: Article

Abstract

The adverse visual conditions of surveillance environments,and the need to identify humans at a distance,have stimulated research in soft biometric attributes. Theseattributes can be used to describe a human’s physical traitssemantically and can be acquired without their cooperation.Soft biometrics can also be employed to retrieve identity from adatabase using verbal descriptions for suspects. In this paper, weexplore unconstrained human face identification with semanticface attributes derived automatically from images. The processuses a deformable face model with keypoint localisation whichis aligned with attributes derived by semantic description. Ournew framework exploits the semantic feature space to inferface signatures from images, and bridges the semantic gapbetween humans and machines with respect to face attributes.We use an unconstrained dataset, LFW-MS4, consisting of allthe subjects from View-1 of the LFW database that have 4 ormore samples. Our new approach demonstrates that retrievalvia estimated comparative facial soft biometrics yields a matchin the top 10.23% of returned subjects. Furthermore, modellingof face image features in the semantic space can achieve anequal error rate of 12.71%. These results reveal the latentbenefits of modelling visual facial features in a semantic space.Moreover, they highlight the potential of using images and verbaldescriptions to generate comparative soft biometrics for subjectidentification and retrieval.

Text FINAL_VERSION - Accepted Manuscript
Download (1MB)

More information

Submitted date: August 2015
Accepted/In Press date: 9 October 2017
e-pub ahead of print date: 23 October 2017
Published date: March 2018

Identifiers

Local EPrints ID: 414933
URI: https://eprints.soton.ac.uk/id/eprint/414933
ISSN: 1556-6013
PURE UUID: c274301f-c0a0-44f6-be2c-2b2ca8f46714
ORCID for Mark Nixon: ORCID iD orcid.org/0000-0002-9174-5934
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 17 Oct 2017 16:30
Last modified: 06 Jun 2018 13:18

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×