Temporally consistent 3D human pose estimation using dual 360° cameras

Shere, M., Kim, Hansung and Hilton, Adrian (2021) Temporally consistent 3D human pose estimation using dual 360° cameras. Winter Conference on Applications of Computer Vision 2021. 05 - 09 Jan 2021. pp. 81-90 . (doi:10.1109/WACV48630.2021.00013).

Record type: Conference or Workshop Item (Paper)

Abstract

This paper presents a 3D human pose estimation system that uses a stereo pair of 360° sensors to capture the complete scene from a single location. The approach combines the advantages of omnidirectional capture, the accuracy of multiple view 3D pose estimation and the portability of monocular acquisition. Joint monocular belief maps for joint locations are estimated from 360° images and are used to fit a 3D skeleton to each frame. Temporal data association and smoothing is performed to produce accurate 3D pose estimates throughout the sequence. We evaluate our system on the Panoptic Studio dataset, as well as real 360° video for tracking multiple people, demonstrating an average Mean Per Joint Position Error of 12.47cm with 30cm baseline cameras. We also demonstrate improved capabilities over perspective and 360° multi-view systems when presented with limited camera views of the subject.

This record has no associated files available for download.

More information

Accepted/In Press date: 2 November 2020

Published date: 5 January 2021

Additional Information: Funding Information: This work is supported by both the EPSRC (grant number EP/N 509383/1) and BBC Research and Development. The authors would like to thank the reviewers for their constructive comments. Publisher Copyright: © 2021 IEEE.

Venue - Dates: Winter Conference on Applications of Computer Vision 2021, 2021-01-05 - 2021-01-09

Learn more about Vision, Learning and Control research