The University of Southampton
University of Southampton Institutional Repository

Semantic face signatures: recognizing and retrieving faces by verbal descriptions

Semantic face signatures: recognizing and retrieving faces by verbal descriptions
Semantic face signatures: recognizing and retrieving faces by verbal descriptions
The adverse visual conditions of surveillance environments,
and the need to identify humans at a distance,
have stimulated research in soft biometric attributes. These
attributes can be used to describe a human’s physical traits
semantically and can be acquired without their cooperation.
Soft biometrics can also be employed to retrieve identity from a
database using verbal descriptions for suspects. In this paper, we
explore unconstrained human face identification with semantic
face attributes derived automatically from images. The process
uses a deformable face model with keypoint localisation which
is aligned with attributes derived by semantic description. Our
new framework exploits the semantic feature space to infer
face signatures from images, and bridges the semantic gap
between humans and machines with respect to face attributes.
We use an unconstrained dataset, LFW-MS4, consisting of all
the subjects from View-1 of the LFW database that have 4 or
more samples. Our new approach demonstrates that retrieval
via estimated comparative facial soft biometrics yields a match
in the top 10.23% of returned subjects. Furthermore, modelling
of face image features in the semantic space can achieve an
equal error rate of 12.71%. These results reveal the latent
benefits of modelling visual facial features in a semantic space.
Moreover, they highlight the potential of using images and verbal
descriptions to generate comparative soft biometrics for subject
identification and retrieval.
1556-6013
Almudhahka, Nawaf, Yousef
929b4dbb-016d-44bb-9755-b9e0adb6ded0
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Almudhahka, Nawaf, Yousef, Nixon, Mark and Hare, Jonathon (2017) Semantic face signatures: recognizing and retrieving faces by verbal descriptions IEEE Transactions on Information Forensics and Security, 14, (8) (Submitted).

Almudhahka, Nawaf, Yousef, Nixon, Mark and Hare, Jonathon (2017) Semantic face signatures: recognizing and retrieving faces by verbal descriptions IEEE Transactions on Information Forensics and Security, 14, (8) (Submitted).

Record type: Article

Abstract

The adverse visual conditions of surveillance environments,
and the need to identify humans at a distance,
have stimulated research in soft biometric attributes. These
attributes can be used to describe a human’s physical traits
semantically and can be acquired without their cooperation.
Soft biometrics can also be employed to retrieve identity from a
database using verbal descriptions for suspects. In this paper, we
explore unconstrained human face identification with semantic
face attributes derived automatically from images. The process
uses a deformable face model with keypoint localisation which
is aligned with attributes derived by semantic description. Our
new framework exploits the semantic feature space to infer
face signatures from images, and bridges the semantic gap
between humans and machines with respect to face attributes.
We use an unconstrained dataset, LFW-MS4, consisting of all
the subjects from View-1 of the LFW database that have 4 or
more samples. Our new approach demonstrates that retrieval
via estimated comparative facial soft biometrics yields a match
in the top 10.23% of returned subjects. Furthermore, modelling
of face image features in the semantic space can achieve an
equal error rate of 12.71%. These results reveal the latent
benefits of modelling visual facial features in a semantic space.
Moreover, they highlight the potential of using images and verbal
descriptions to generate comparative soft biometrics for subject
identification and retrieval.

Text FINAL_VERSION
Restricted to Repository staff only until 12 December 2017.

More information

Submitted date: August 2015
Accepted/In Press date: 2017

Identifiers

Local EPrints ID: 414933
URI: http://eprints.soton.ac.uk/id/eprint/414933
ISSN: 1556-6013
PURE UUID: c274301f-c0a0-44f6-be2c-2b2ca8f46714
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 17 Oct 2017 16:30
Last modified: 17 Oct 2017 16:30

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×