Yamada, Takaki, Massot Campos, Miguel, Curtis, Emma, Juliet, Pizarro, Oscar, Williams, Stefan B., Huvenne, Veerle and Thornton, Blair (2021) Metadata enhanced feature learning for efficient interpretation of AUV gathered seafloor visual imagery.
Abstract
Camera equipped Autonomous Underwater Vehicles (AUVs) typically gather tens to hundreds of thousands of georeferenced seafloor images in a single deployment. However, taking full advantage of this growing repository of data is a major challenge for the scientific progress. Although modern machine learning techniques, e.g. Deep Learning, are potentially useful to interpret these images, much of the progress in this field has been driven by the availability of large training datasets of expert human annotations that are available for terrestrial ad satellite imaging applications. Such datasets do not currently exist in the marine domain, and even if they did, it is not clear if the sensitivity of marine images to observation conditions, such as altitude, water turbidity and different illumination sources will limit the utility of such initiatives. Most applications of deep learning to marine imagery have used training datasets that have been specifically generated on a per-survey basis, and although the results are encouraging, the high workload involved in generating dataset specific expert training labels is unlikely to be justifiable in most applications.
To address this issue, we investigate the use of unsupervised feature learning (or representation learning), where lower-dimensional feature vectors are derived from original high-dimensional image data through Convolutional Neural Networks (CNN) without any human annotations. Once the feature vectors of the original images, which keep only useful information to distinguish the habitats and substrates, are obtained, various interpretation techniques such as clustering, contents retrieval, few-shot learning can be efficiently applied.
In this work, we demonstrate autoencoder and contrastive learning-based feature learning techniques specially designed for seafloor visual imagery [1]. The proposed methods can leverage the metadata gathered with the images by AUV, e.g. georeference, water-temperature, saliency. We confirm that metadata-guiding significantly improves the feature learning and demonstrate applications to unsupervised and semi-supervised mapping of habitat, substrate and infrastructure distribution at the Southern Hydrate Ridge (Oregon, USA, 12k images), Darwin Mounds (UK, 20k images) and Tasmania (Australia, 110k images) datasets with validation against human annotation results.
[1] Takaki Yamada, Adam Prugel-Bennet, Blair Thornton, Learning Features from Georeferenced Seafloor Imagery with Location Guided Autoencoders, Journal of Field Robotics 38, 52-67, DOI: 10.1002/rob.21961
This record has no associated files available for download.
More information
Identifiers
Catalogue record
Export record
Contributors
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.