The University of Southampton
University of Southampton Institutional Repository

Leveraging metadata in representation learning with georeferenced seafloor imagery

Leveraging metadata in representation learning with georeferenced seafloor imagery
Leveraging metadata in representation learning with georeferenced seafloor imagery

Camera equipped Autonomous Underwater Vehicles (AUVs) are now routinely used in seafloor surveys. Obtaining effective representations from the images they collect can enable perception-aware robotic exploration such as information-gain-guided path planning and target-driven visual navigation. This letter develops a novel self-supervised representation learning method for seafloor images collected by AUVs. The method allows deep-learning convolutional autoencoders to leverage multiple sources of metadata to regularise their learning, prioritising features observed in images that can be correlated with patterns in their metadata. The impact of the proposed regularisation is examined on a dataset consisting of more than 30 k colour seafloor images gathered by an AUV off the coast of Tasmania. The metadata used to regularise learning in this dataset consists of the horizontal location and depth of the observed seafloor. The results show that including metadata in self-supervised representation learning can increase image classification accuracy by up to 15% and never degrades learning performance. We show how effective representation learning can be applied to achieve class balanced representative image identification for summarised understanding of imbalanced class distributions in an unsupervised way.

computer vision, Marine robotics, metadata, representation learning, visual learning
2377-3766
7815-7822
Yamada, Takaki
81c66c35-0e2b-4342-80fa-cbee6ff9ce5f
Massot Campos, Miguel
a55d7b32-c097-4adf-9483-16bbf07f9120
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
Williams, Stephan
d0b8c6f1-ec56-43e2-bcd9-9d34f23a45e8
Pizarro, Oscar
a9ed2c7e-ae8d-4c92-bd02-7e9981e4d4f1
Thornton, Blair
8293beb5-c083-47e3-b5f0-d9c3cee14be9
Yamada, Takaki
81c66c35-0e2b-4342-80fa-cbee6ff9ce5f
Massot Campos, Miguel
a55d7b32-c097-4adf-9483-16bbf07f9120
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
Williams, Stephan
d0b8c6f1-ec56-43e2-bcd9-9d34f23a45e8
Pizarro, Oscar
a9ed2c7e-ae8d-4c92-bd02-7e9981e4d4f1
Thornton, Blair
8293beb5-c083-47e3-b5f0-d9c3cee14be9

Yamada, Takaki, Massot Campos, Miguel, Prugel-Bennett, Adam, Williams, Stephan, Pizarro, Oscar and Thornton, Blair (2021) Leveraging metadata in representation learning with georeferenced seafloor imagery. IEEE Robotics and Automation Letters, 6 (4), 7815-7822, [9507317]. (doi:10.1109/LRA.2021.3101881).

Record type: Article

Abstract

Camera equipped Autonomous Underwater Vehicles (AUVs) are now routinely used in seafloor surveys. Obtaining effective representations from the images they collect can enable perception-aware robotic exploration such as information-gain-guided path planning and target-driven visual navigation. This letter develops a novel self-supervised representation learning method for seafloor images collected by AUVs. The method allows deep-learning convolutional autoencoders to leverage multiple sources of metadata to regularise their learning, prioritising features observed in images that can be correlated with patterns in their metadata. The impact of the proposed regularisation is examined on a dataset consisting of more than 30 k colour seafloor images gathered by an AUV off the coast of Tasmania. The metadata used to regularise learning in this dataset consists of the horizontal location and depth of the observed seafloor. The results show that including metadata in self-supervised representation learning can increase image classification accuracy by up to 15% and never degrades learning performance. We show how effective representation learning can be applied to achieve class balanced representative image identification for summarised understanding of imbalanced class distributions in an unsupervised way.

Text
Leveraging_Metadata_in_Representation_Learning_for_Georeferenced_Seafloor_Imagery_Final_with_ieee_copyright - Accepted Manuscript
Download (4MB)
Text
Yamada_2021_RAL
Download (4MB)

More information

Accepted/In Press date: 9 July 2021
e-pub ahead of print date: 4 August 2021
Additional Information: Paper jointly presented at 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) T. Yamada, M. Massot-Campos, A. Prugel-Bennett, S. B. Williams, O. Pizarro and B. Thornton, "Leveraging Metadata in Representation Learning with Georeferenced Seafloor Imagery," in IEEE Robotics and Automation Letters, doi: 10.1109/LRA.2021.3101881. https://ieeexplore.ieee.org/document/9507317 © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Keywords: computer vision, Marine robotics, metadata, representation learning, visual learning

Identifiers

Local EPrints ID: 450428
URI: http://eprints.soton.ac.uk/id/eprint/450428
ISSN: 2377-3766
PURE UUID: f7885f20-5612-41c2-9bdb-cdce1fe3e06f
ORCID for Takaki Yamada: ORCID iD orcid.org/0000-0002-5090-7239
ORCID for Miguel Massot Campos: ORCID iD orcid.org/0000-0002-1202-0362

Catalogue record

Date deposited: 28 Jul 2021 16:30
Last modified: 18 Mar 2024 05:27

Export record

Altmetrics

Contributors

Author: Takaki Yamada ORCID iD
Author: Adam Prugel-Bennett
Author: Stephan Williams
Author: Oscar Pizarro
Author: Blair Thornton

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×