A Global Gradient Noise Covariance Expression for Stationary Real Gaussian Inputs
An, P.E., Brown, M. and Harris, C.J. (1995) A Global Gradient Noise Covariance Expression for Stationary Real Gaussian Inputs. IEEE Trans. on Neural Networks, 6, (6)
Download
Full text not available from this repository.
Description/Abstract
Supervised parameter adaptation in many artificial neural networks is largely based on an instantaneous version of gradient descent called the Least-Mean-Square (LMS) algorithm. As the gradient is estimated using single samples of the input ensemble, its convergence properties generally deviate significantly from that of the true gradient descent because of the noise in the gradient estimate. It is thus important to study the gradient noise characteristics so that the convergence of the LMS algorithm can be analyzed from a new perspective. This paper considers only neural models which are linear with respect to their adaptable parameters, and has two major contributions. Firstly, it derives an expression for the gradient noise covariance under the assumption that the input samples are real, stationary, Gaussian distributed but can be partially correlated. This expression relates the gradient correlation and input correlation matrices to the gradient noise covariance, and explains why the gradient noise generally correlates maximally with the steepest principal axis and minimally with the one of the smallest curvature, regardless of the magnitude of the weight error. Secondly, a recursive expression for the weight error correlation matrix is derived in a straightforward manner using the gradient noise covariance, and comparisons are drawn with the complex LMS algorithm.
| Item Type: | Article |
|---|---|
| Additional Information: | accepted for publication |
| Divisions: | Faculty of Physical Sciences and Engineering > Electronics and Computer Science > Comms, Signal Processing & Control |
| Item ID: | 250282 |
| Date Deposited: | 04 May 1999 |
| Last Modified: | 02 Mar 2012 11:37 |
| Contributors: | An, P.E. (Author) Brown, M. (Author) Harris, C.J. (Author) |
| Date: | 1995 |
| Additional Information: | accepted for publication |
| Status: | Published |
| Further Information: | Google Scholar |
| ISI Citation Count: | 2 |
| URI: | http://eprints.soton.ac.uk/id/eprint/250282 |
Actions (login required)
 |
View Item |
