The University of Southampton
University of Southampton Institutional Repository

Replica symmetry breaking in bipartite spin glasses and neural networks

Replica symmetry breaking in bipartite spin glasses and neural networks
Replica symmetry breaking in bipartite spin glasses and neural networks

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin-glass model, which is formally similar to a restricted Boltzmann machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi's solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data themselves.

2470-0045
Hartnett, Gavin S.
09c9c28e-7494-4793-8b1c-237b313c076a
Parker, Edward
df6150eb-0ef3-49de-9550-5430f7fb5210
Geist, Edward
b9bc98d7-ef73-483e-a6c4-5981511472d8
Hartnett, Gavin S.
09c9c28e-7494-4793-8b1c-237b313c076a
Parker, Edward
df6150eb-0ef3-49de-9550-5430f7fb5210
Geist, Edward
b9bc98d7-ef73-483e-a6c4-5981511472d8

Hartnett, Gavin S., Parker, Edward and Geist, Edward (2018) Replica symmetry breaking in bipartite spin glasses and neural networks. Physical Review E, 98 (2), [022116]. (doi:10.1103/PhysRevE.98.022116).

Record type: Article

Abstract

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin-glass model, which is formally similar to a restricted Boltzmann machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi's solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data themselves.

Text
1801.10176 - Accepted Manuscript
Download (2MB)

More information

Accepted/In Press date: 15 August 2018
e-pub ahead of print date: 15 August 2018
Published date: 15 August 2018

Identifiers

Local EPrints ID: 424467
URI: http://eprints.soton.ac.uk/id/eprint/424467
ISSN: 2470-0045
PURE UUID: ce419f8e-875b-43a7-bff7-e5b0ea7c6634

Catalogue record

Date deposited: 05 Oct 2018 11:37
Last modified: 06 Oct 2020 19:58

Export record

Altmetrics

Contributors

Author: Gavin S. Hartnett
Author: Edward Parker
Author: Edward Geist

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×