On using gait to enhance face extraction for visual surveillance

Abstract

Visual surveillance finds increasing deployment for monitoring urban environments. Operators need to be able to determine identity from surveillance images and often use face recognition for this purpose. Unfortunately, the quality of the recorded imagery can be insufficient for this task. This study describes a programme of research aimed to ameliorate this limitation. Many face biometrics systems use controlled environments where subjects are viewed directly facing the camera. This is less likely to occur in surveillance environments, so it is necessary to handle pose variations of the human head, low frame rate, and low resolution input images. We describe the first use of gait to enable face acquisition and recognition, by analysis of 3D head motion and gait trajectory, with super-resolution analysis. The face extraction procedure consists of three stages: i) head pose estimation by a 3D ellipsoidal model; ii) face region extraction by using a 2D or a 3D gait trajectory; and iii) frontal face extraction and reconstruction by estimating head pose and using super-resolution techniques. The head pose is estimated by using a 3D ellipsoidal model and non-linear optimisation. Region- and distance-based feature refinement methods are used and a direct mapping from the 2D image coordinate to the object coordinate is developed. In face region extraction the potential face region is extracted based on the 2D gait trajectory model when a person walks towards a camera. We model a looming field and show how this field affects the image sequences of the human walking. By fitting a 2D gait trajectory model the face region can then be tracked. For the general case of the human walking a 3D gait trajectory model and heel strike positions are used to extract the face region in 3D space. Wavelet decomposition is used to detect the gait cycle and a new heel strike detection method is developed. In face extraction a high resolution frontal face image is reconstructed with low resolution face images by analysing super-resolution. Based on the head pose and 3D ellipsoidal model the invalid low resolution face images are filtered and the frontal view face is reconstructed. By adapting the existing super-resolution the high resolution frontal face image can be synthesised, which is demonstrated to be suitable for face recognition. The contributions of this research include the construction of a 3D model for pose estimation from planar imagery and the first use of gait information to enhance the face extraction and recognition process allowing for deployment in surveillance scenarios.

More information

Identifiers

ORCID for Mark S. Nixon: orcid.org/0000-0002-9174-5934

Catalogue record

Export record