Learning Features from georeferenced seafloor imagery with location guided autoencoders

Although modern machine learning has the potential to greatly speed up the interpretation of imagery, the varied nature of the seabed and limited availability of expert annotations form barriers to its widespread use in seafloor mapping applications. This motivates research into unsupervised methods that function without large databases of human annotations. This paper develops an unsupervised feature learning method for georeferenced seafloor visual imagery that considers patterns both within the footprint of a single image frame and broader scale spatial characteristics. Features within images are learnt using an autoencoder developed based on the AlexNet deep convolutional neural network. Features larger than each image frame are learnt using a novel loss function that regularises autoencoder training using the Kullback–Leibler divergence function to loosely assume that images captured within a close distance of each other look more similar than those that are far away. The method is used to semantically interpret images taken by an autonomous underwater vehicle at the Southern Hydrates Ridge, an active gas hydrate field and site of a seafloor cabled observatory at a depth of 780 m. The method's performance when applied to clustering and content‐based image retrieval is assessed against a ground truth consisting of more than 18,000 human annotations. The study shows that the location based loss function increases the rate of information retrieval by a factor of two for seafloor mapping applications. The effects of physics‐based colour correction and image rescaling are also investigated, showing that the improved consistency of spatial information achieved by rescaling is beneficial for recognising artificial objects such as cables and infrastructures, but is less effective for natural objects that have greater dimensional variability.

autoencoder, computer vision, mapping, underwater robotics, unsupervised learning

10.1002/rob.21961

1556-4959

52-67

Yamada, Takaki

81c66c35-0e2b-4342-80fa-cbee6ff9ce5f

Prugel-Bennett, Adam

b107a151-1751-4d8b-b8db-2c395ac4e14e

Thornton, Blair

8293beb5-c083-47e3-b5f0-d9c3cee14be9

January 2021

Yamada, Takaki

81c66c35-0e2b-4342-80fa-cbee6ff9ce5f

Prugel-Bennett, Adam

b107a151-1751-4d8b-b8db-2c395ac4e14e

Thornton, Blair

8293beb5-c083-47e3-b5f0-d9c3cee14be9

Yamada, Takaki, Prugel-Bennett, Adam and Thornton, Blair (2021) Learning Features from georeferenced seafloor imagery with location guided autoencoders. Journal of Field Robotics, 38 (1), 52-67. (doi:10.1002/rob.21961).

Record type: Article

Abstract

Text

Learning Features from Georeferenced Seafloor Imagery with Location Guided Autoencoders - Accepted Manuscript

Available under License Creative Commons Attribution.

Download (13MB)

Text

rob.21961 - Version of Record

Available under License Creative Commons Attribution.

Download (15MB)

More information

Accepted/In Press date: 9 May 2020

e-pub ahead of print date: 28 May 2020

Published date: January 2021

Additional Information: Funding Information: This study was carried out under the UK Natural Environment Research Council's Biocam project NE/P020887/1, part of the Oceanids Marine Sensor Capital program. The data used in this study was collected during the Schmidt Ocean Institute's FK180731 #Adaptive Robotics campaign, with support from the Japanese Government's Zipangu in the Ocean Strategic Innovation Program. The authors thank Kazunori Nagano, Tetsuo Koike and Harumi Sugimatsu of the Institute of Industrial Science at the University of Tokyo, Japan, Adrian Bodenmann, Jennifer Walker, and Jim Wei Lim of the University of Southampton, UK, Gabriel Oliver and Miquel Massot-Campos of the SRV group at the University of the Balearic Islands, Spain, and the crew of the RV Falkor for their operational support deploying the AUV ae2000f to collect the imagery used in this study. The authors thank the Marine Systems group at the Australian Centre for Field Robotics, University of Sydney, for providing access to their SLAM and 3D reconstruction pipelines used to generate the navigational solution and mosaics used in this study, Jin Wei Lim and Jose Cappelletto for their assistance in annotating the dataset. The IRIDIS High Performance Computing Facility and associated support services at the University of Southampton were used in the completion of this study. Funding Information: This study was carried out under the UK Natural Environment Research Council's Biocam project NE/P020887/1, part of the Oceanids Marine Sensor Capital program. The data used in this study was collected during the Schmidt Ocean Institute's FK180731 #Adaptive Robotics campaign, with support from the Japanese Government's Zipangu in the Ocean Strategic Innovation Program. The authors thank Kazunori Nagano, Tetsuo Koike and Harumi Sugimatsu of the Institute of Industrial Science at the University of Tokyo, Japan, Adrian Bodenmann, Jennifer Walker, and Jim Wei Lim of the University of Southampton, UK, Gabriel Oliver and Miquel Massot‐Campos of the SRV group at the University of the Balearic Islands, Spain, and the crew of the RV Falkor for their operational support deploying the AUV ae2000f to collect the imagery used in this study. The authors thank the Marine Systems group at the Australian Centre for Field Robotics, University of Sydney, for providing access to their SLAM and 3D reconstruction pipelines used to generate the navigational solution and mosaics used in this study, Jin Wei Lim and Jose Cappelletto for their assistance in annotating the dataset. The IRIDIS High Performance Computing Facility and associated support services at the University of Southampton were used in the completion of this study. Publisher Copyright: © 2020 The Authors. Journal of Field Robotics published by Wiley Periodicals LLC

Keywords: autoencoder, computer vision, mapping, underwater robotics, unsupervised learning

Learn more about Vision, Learning and Control research