The University of Southampton
University of Southampton Institutional Repository

Anatomically constrained ResNets exhibit opponent receptive fields; so what?

Anatomically constrained ResNets exhibit opponent receptive fields; so what?
Anatomically constrained ResNets exhibit opponent receptive fields; so what?
Primate visual systems are well known to exhibit varying degrees of bottlenecks in the early visual pathway. Recent works have shown that the presence of a bottleneck between 'retinal' and 'ventral' parts of artificial models of visual systems, simulating the optic nerve, can cause the emergence of cellular properties that have been observed in primates: namely centre-surround organisation and opponency. To date, however, state-of-the-art convolutional network architectures for classification problems have not incorporated such an early bottleneck. In this paper, we ask what happens if such a bottleneck is added to a ResNet-50 model trained to classify the ImageNet data set. Our experiments show that some of the emergent properties observed in simpler models still appear in these considerably deeper and more complex models, however, there are some notable differences particularly with regard to spectral opponency. The introduction of the bottleneck is experimentally shown to introduce a small but consistent shape bias into the network. Tight bottlenecks are also shown to only have a very slight affect on the top-1 accuracy of the models when trained and tested on ImageNet.
Harris, Ethan William Albert
6d531059-ebaa-451c-b242-5394f0288266
Mihai, Andreea Daniela
f8910fe1-18e7-45b3-8923-b34b5cd136fa
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Harris, Ethan William Albert
6d531059-ebaa-451c-b242-5394f0288266
Mihai, Andreea Daniela
f8910fe1-18e7-45b3-8923-b34b5cd136fa
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9

Harris, Ethan William Albert, Mihai, Andreea Daniela and Hare, Jonathon (2020) Anatomically constrained ResNets exhibit opponent receptive fields; so what? In Shared Visual Representations in Human & Machine Intelligence: 2020 NeurIPS Workshop. 10 pp .

Record type: Conference or Workshop Item (Paper)

Abstract

Primate visual systems are well known to exhibit varying degrees of bottlenecks in the early visual pathway. Recent works have shown that the presence of a bottleneck between 'retinal' and 'ventral' parts of artificial models of visual systems, simulating the optic nerve, can cause the emergence of cellular properties that have been observed in primates: namely centre-surround organisation and opponency. To date, however, state-of-the-art convolutional network architectures for classification problems have not incorporated such an early bottleneck. In this paper, we ask what happens if such a bottleneck is added to a ResNet-50 model trained to classify the ImageNet data set. Our experiments show that some of the emergent properties observed in simpler models still appear in these considerably deeper and more complex models, however, there are some notable differences particularly with regard to spectral opponency. The introduction of the bottleneck is experimentally shown to introduce a small but consistent shape bias into the network. Tight bottlenecks are also shown to only have a very slight affect on the top-1 accuracy of the models when trained and tested on ImageNet.

Text
camera_ready - Version of Record
Download (1MB)

More information

Published date: 12 December 2020

Identifiers

Local EPrints ID: 445761
URI: http://eprints.soton.ac.uk/id/eprint/445761
PURE UUID: 2ca100ca-974b-464e-a4a4-2040fb678dbf
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283

Catalogue record

Date deposited: 07 Jan 2021 17:32
Last modified: 07 Jan 2021 17:32

Export record

Contributors

Author: Ethan William Albert Harris
Author: Andreea Daniela Mihai
Author: Jonathon Hare ORCID iD

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×