The University of Southampton
University of Southampton Institutional Repository

How Biased is Your Multi-Layered Perceptron?

Brown, M., An, P.E., Harris, C.J. and Wang, H. (1993) How Biased is Your Multi-Layered Perceptron? At Proc. World Congress on Neural Networks. , 507--511.

Record type: Conference or Workshop Item (Other)

Abstract

Gradient descent and instantaneous gradient descent learning rules are popular methods for training neural models. Backwards Error Propagation (BEP) applied to Multi-Layer Prectron (MLP) is one example of nonlinear gradient descent, and Widrow's Adaptive Linear Combiner (ALC) and the Albus CMAC are both generally trained using (instantaneous) gradient descent rules. However, these learning algorithms are often applied without regard for the condition of the resultant optimisation problem. Often the basic model can be transformed such that its modelling capabilities remain unchanged, but the condition of the optimisation problem is improved. In this paper, the basic theory behind gradient descent adaptive algorithms will be stated, and then it will be applied to a wide range of common neural networks. In the simplest case it will be shown how the condition of the ALC is dependent on the domain over which the training data is distributed, and the important concept of orthogonal functions will be introduced. This network is then compared with alternative models (eg. B-splines) which have identical modelling capabilities but a different internal representation. The theory is then applied to MLPs trained using gradient descent algorithms and it is shown that MLPs with sigmoids whose outputs lie in the range [0,1] are inherently ill-conditioned, although this can be overcome by simple translation and scaling procedures. All of these results can be generalised to instantaneous gradient descent procedures.

Full text not available from this repository.

More information

Published date: 1993
Additional Information: Organisation: INNS Address: Portland, OR
Venue - Dates: Proc. World Congress on Neural Networks, 1993-01-01
Organisations: Southampton Wireless Group

Identifiers

Local EPrints ID: 250244
URI: http://eprints.soton.ac.uk/id/eprint/250244
PURE UUID: b4afa917-fcd7-4b15-a629-48a91900a088

Catalogue record

Date deposited: 04 May 1999
Last modified: 18 Jul 2017 10:43

Export record

Contributors

Author: M. Brown
Author: P.E. An
Author: C.J. Harris
Author: H. Wang

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×