Facial profile recognition using comparative soft biometrics

Abstract

The identification of suspects in surveillance footage is crucial for maintaining public safety, preventing crime, conserving police resources, and aiding forensic investigations. Although eyewitness testimonies are valuable assets in numerous criminal cases, it is presently rather challenging to identify individuals in real-world closed-circuit television (CCTV) footage solely based on eyewitness descriptions. As a result, there has been a significant rise in the interest of using soft biometrics, which are physical and behavioural attributes that are used to semantically describe people under adverse surveillance conditions. Traditional biometrics are used when images or videos are available. Nevertheless, in certain instances, only eyewitness testimonies are available. In such scenarios, soft biometrics are applied to transform the eyewitness testimony into a collection of features that can be utilised for automated recognition. Furthermore, when images, videos and eyewitness testimonies are available, the fusion of soft biometrics with traditional biometrics becomes essential.
The objective of this thesis is to investigate the integration of soft biometrics with traditional biometrics, enabling the search of video footage and biometric data based on descriptive information to identify suspects. The existing literature on facial soft biometrics mainly focuses on the frontal face, However, this approach fails to acknowledge the importance of facial profiles, which have been demonstrated to be highly accurate. It is crucial to consider facial profiles because there are situations in which only these profiles are captured in images and videos from surveillance and security cameras. In such instances, existing facial recognition algorithms designed for frontal views are ineffective, emphasising the necessity for recognition systems specifically tailored for profile faces.
This thesis builds upon previous research on using soft biometrics for human recognition, with a specific emphasis on the potential of soft biometrics in identifying facial profiles. Soft biometrics involves crowdsourcing human annotations through ordered and similarity comparisons. Advanced machine learning techniques are also used to estimate comparative attributes from images. In addition, we analyse the attribute’s correspondence between the traditional biometric and soft biometric based on facial profiles. Therefore, we have bridged the gap between human perception and computer vision for facial profile biometric. In comparison to prior work on facial profiles, the developed approaches have demonstrated a higher level of performance. Our findings indicate that the performance of the system further improves after fusing the semantic and visual spaces. Furthermore, this thesis examines the bilateral symmetry of human facial profiles and develops a method for extracting features from facial profiles, inspired by the few-shot learning framework. Our algorithm, which is based on few-shot learning, achieves an impressive level of accuracy even when working with datasets containing a large number of subjects and low number of samples per subject.

More information

Identifiers

Catalogue record

Export record