The University of Southampton
University of Southampton Institutional Repository

Unsupervised representation learning via information compression

Unsupervised representation learning via information compression
Unsupervised representation learning via information compression
This paper explores a new paradigm for decomposing an image by seeking a compressed representation of the image through an information bottleneck. The compression is achieved iteratively by refining the reconstruction by adding patches that reduce the residual error. This is achieved by a network that is given the current residual errors and proposes bounding boxes that are down-sampled and passed to a variational auto-encoder (VAE). This acts as the bottleneck. The latent code is decoded by the VAE decoder and up-sampled to correct the reconstruction within the bounding box. The objective is to minimise the size of the latent codes of the VAE and the length of code needed to transmit the residual error. The iterations end when the size of the latent code exceeds the reduction in transmitting the residual error. We show that a very simple implementation is capable of finding meaningful bounding boxes and using those bounding boxes for downstream applications. We compare our model with other unsupervised object discovery models.
Information bottleneck, Object discovery, Unsupervised representation learning, VAE
567-578
Springer
Zeng, Zezhen
d340d998-568a-434f-95eb-ef39ee335912
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
El Yacoubi, Mounîm
Granger, Eric
Chi Yuen, Pong
Pal, Umapada
Vincent, Nicole
Zeng, Zezhen
d340d998-568a-434f-95eb-ef39ee335912
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Prugel-Bennett, Adam
b107a151-1751-4d8b-b8db-2c395ac4e14e
El Yacoubi, Mounîm
Granger, Eric
Chi Yuen, Pong
Pal, Umapada
Vincent, Nicole

Zeng, Zezhen, Hare, Jonathon and Prugel-Bennett, Adam (2022) Unsupervised representation learning via information compression. El Yacoubi, Mounîm, Granger, Eric, Chi Yuen, Pong, Pal, Umapada and Vincent, Nicole (eds.) In Pattern Recognition and Artificial Intelligence : Third International Conference, ICPRAI 2022, Paris, France. vol. 13363, Springer. pp. 567-578 . (doi:10.1007/978-3-031-09037-0_46).

Record type: Conference or Workshop Item (Paper)

Abstract

This paper explores a new paradigm for decomposing an image by seeking a compressed representation of the image through an information bottleneck. The compression is achieved iteratively by refining the reconstruction by adding patches that reduce the residual error. This is achieved by a network that is given the current residual errors and proposes bounding boxes that are down-sampled and passed to a variational auto-encoder (VAE). This acts as the bottleneck. The latent code is decoded by the VAE decoder and up-sampled to correct the reconstruction within the bounding box. The objective is to minimise the size of the latent codes of the VAE and the length of code needed to transmit the residual error. The iterations end when the size of the latent code exceeds the reduction in transmitting the residual error. We show that a very simple implementation is capable of finding meaningful bounding boxes and using those bounding boxes for downstream applications. We compare our model with other unsupervised object discovery models.

Text
sub_33 - Accepted Manuscript
Download (327kB)

More information

e-pub ahead of print date: 2 June 2022
Published date: 2022
Additional Information: Publisher Copyright: © 2022, Springer Nature Switzerland AG.
Keywords: Information bottleneck, Object discovery, Unsupervised representation learning, VAE

Identifiers

Local EPrints ID: 468261
URI: http://eprints.soton.ac.uk/id/eprint/468261
PURE UUID: 285dbfd7-f768-4f82-95e3-21359972c120
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 09 Aug 2022 16:37
Last modified: 17 Mar 2024 07:24

Export record

Altmetrics

Contributors

Author: Zezhen Zeng
Author: Jonathon Hare ORCID iD
Author: Adam Prugel-Bennett
Editor: Mounîm El Yacoubi
Editor: Eric Granger
Editor: Pong Chi Yuen
Editor: Umapada Pal
Editor: Nicole Vincent

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×