Depth estimation for indoor single omnidirectional images

Abstract

Omnidirectional cameras are becoming popular in various applications owing to their ability to capture the full surrounding scene in one frame. However, depth estimation for an omnidirectional scene is more difficult than for general images due to its different system properties and distortions. Monocular depth estimation for single-view using deep learning can be a good solution, but it requires a large labelled depth dataset with various scenes. Currently published omnidirectional depth datasets cover limited types of scenes and are not suitable for depth estimation for various real-world scenes. In addition, the existing methods are basically data-driven, and the depth estimation process based on deep learning is still a black box. In order to overcome these problems, we first proposed a depth estimation architecture for a single omnidirectional image using domain adaptation, only with limited labelled real-world scenes. With the challenge of getting labelled real-world datasets and stability of the performance, we updated the components of architecture and proposed a reverse-gradient warming-up threshold discriminator (RWTD) to estimate real-world depth maps from synthetic ground truth. It takes labelled synthetic scenes of a source domain and unlabelled real-world scenes of a target domain as inputs to predict the corresponding depth maps. To solve the black-box depth estimation process, we analyse the role of gravity in depth estimation and propose a slicing method based on the gravity direction. Equally crucial to our investigation is the examination of the contributions of different cues to the results of indoor depth estimation. The results show that the four factors of colour, saturation, local texture and shape show different extent contributions, and among them, the shape feature plays a dominant role in the performance of depth estimation. These works present solutions for depth estimation of omnidirectional images in real-world applications and delve into the critical role of gravity alignment, as well as the exploration of how machines perceive depth, providing a foundation for subsequent research.

More information

Identifiers

ORCID for Yihong Wu: orcid.org/0000-0003-3340-2535

ORCID for Hansung Kim: orcid.org/0000-0003-4907-0491

ORCID for Mahesan Niranjan: orcid.org/0000-0001-7021-140X

Catalogue record

Export record