The University of Southampton
University of Southampton Institutional Repository

Semantic face signatures: recognizing and retrieving faces by verbal descriptions

Semantic face signatures: recognizing and retrieving faces by verbal descriptions
Semantic face signatures: recognizing and retrieving faces by verbal descriptions
The adverse visual conditions of surveillance environments and the need to identify humans at a distance have stimulated research in soft biometric attributes. These attributes can be used to describe a human's physical traits semantically and can be acquired without their cooperation. Soft biometrics can also be employed to retrieve identity from a database using verbal descriptions of suspects. In this paper, we explore unconstrained human face identification with semantic face attributes derived automatically from images. The process uses a deformable face model with keypoint localisation which is aligned with attributes derived from semantic descriptions. Our new framework exploits the semantic feature space to infer face signatures from images and bridges the semantic gap between humans and machines with respect to face attributes. We use an unconstrained dataset, LFW-MS4, consisting of all the subjects from view-1 of the LFW database that have four or more samples. Our new approach demonstrates that retrieval via estimated comparative facial soft biometrics yields a match in the top 10.23% of returned subjects. Furthermore, modelling of face image features in the semantic space can achieve an equal error rate of 12.71%. These results reveal the latent benefits of modelling visual facial features in a semantic space. Moreover, they highlight the potential of using images and verbal descriptions to generate comparative soft biometrics for subject identification and retrieval.
1556-6013
706-716
Almudhahka, Nawaf, Yousef
929b4dbb-016d-44bb-9755-b9e0adb6ded0
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Almudhahka, Nawaf, Yousef
929b4dbb-016d-44bb-9755-b9e0adb6ded0
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9

Almudhahka, Nawaf, Yousef, Nixon, Mark and Hare, Jonathon (2018) Semantic face signatures: recognizing and retrieving faces by verbal descriptions. IEEE Transactions on Information Forensics and Security, 13 (3), 706-716. (doi:10.1109/TIFS.2017.2765519).

Record type: Article

Abstract

The adverse visual conditions of surveillance environments and the need to identify humans at a distance have stimulated research in soft biometric attributes. These attributes can be used to describe a human's physical traits semantically and can be acquired without their cooperation. Soft biometrics can also be employed to retrieve identity from a database using verbal descriptions of suspects. In this paper, we explore unconstrained human face identification with semantic face attributes derived automatically from images. The process uses a deformable face model with keypoint localisation which is aligned with attributes derived from semantic descriptions. Our new framework exploits the semantic feature space to infer face signatures from images and bridges the semantic gap between humans and machines with respect to face attributes. We use an unconstrained dataset, LFW-MS4, consisting of all the subjects from view-1 of the LFW database that have four or more samples. Our new approach demonstrates that retrieval via estimated comparative facial soft biometrics yields a match in the top 10.23% of returned subjects. Furthermore, modelling of face image features in the semantic space can achieve an equal error rate of 12.71%. These results reveal the latent benefits of modelling visual facial features in a semantic space. Moreover, they highlight the potential of using images and verbal descriptions to generate comparative soft biometrics for subject identification and retrieval.

Text
FINAL_VERSION - Accepted Manuscript
Download (1MB)

More information

Submitted date: August 2015
Accepted/In Press date: 9 October 2017
e-pub ahead of print date: 23 October 2017
Published date: March 2018

Identifiers

Local EPrints ID: 414933
URI: http://eprints.soton.ac.uk/id/eprint/414933
ISSN: 1556-6013
PURE UUID: c274301f-c0a0-44f6-be2c-2b2ca8f46714
ORCID for Mark Nixon: ORCID iD orcid.org/0000-0002-9174-5934
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 17 Oct 2017 16:30
Last modified: 28 Apr 2022 04:44

Export record

Altmetrics

Contributors

Author: Nawaf, Yousef Almudhahka
Author: Mark Nixon ORCID iD
Author: Jonathon Hare ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×