The University of Southampton
University of Southampton Institutional Repository

Mutual Information Theory for Adaptive Mixture Models

Mutual Information Theory for Adaptive Mixture Models
Mutual Information Theory for Adaptive Mixture Models
Many pattern recognition systems need to estimate an underlying probability density function (pdf). Mixture models are commonly used for this purpose in which an underlying pdf is estimated by a finite mixing of distributions. The basic computational element of a density mixture model is a component with a nonlinear mapping function, which takes part in mixing. Selecting an optimal set of components for mixture models is important to ensure an efficient and accurate estimate of an underlying pdf. Previous work has commonly estimated an underlying pdf based on the information contained in patterns. In this paper, mutual information theory is employed to measure whether two components are statistically dependent. If a component has small mutual information, it is statistically independent of the other components. Hence, that component makes a significant contribution to the system pdf and should not be removed. However, if a particular component has large mutual information, it is unlikely to be statistically independent of the other components and may be removed without significant damage to the estimated pdf. Continuing to remove components with large and positive mutual information will give a density mixture model with an optimal structure, which is very close to the true pdf.
396-403
Zwolinski, M.
adfcb8e7-877f-4bd7-9b55-7553b6cb3ea0
Yang, Z.R.
b71f8582-83df-4c36-9925-136cdfd207f9
Zwolinski, M.
adfcb8e7-877f-4bd7-9b55-7553b6cb3ea0
Yang, Z.R.
b71f8582-83df-4c36-9925-136cdfd207f9

Zwolinski, M. and Yang, Z.R. (2001) Mutual Information Theory for Adaptive Mixture Models IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, (4), pp. 396-403.

Record type: Article

Abstract

Many pattern recognition systems need to estimate an underlying probability density function (pdf). Mixture models are commonly used for this purpose in which an underlying pdf is estimated by a finite mixing of distributions. The basic computational element of a density mixture model is a component with a nonlinear mapping function, which takes part in mixing. Selecting an optimal set of components for mixture models is important to ensure an efficient and accurate estimate of an underlying pdf. Previous work has commonly estimated an underlying pdf based on the information contained in patterns. In this paper, mutual information theory is employed to measure whether two components are statistically dependent. If a component has small mutual information, it is statistically independent of the other components. Hence, that component makes a significant contribution to the system pdf and should not be removed. However, if a particular component has large mutual information, it is unlikely to be statistically independent of the other components and may be removed without significant damage to the estimated pdf. Continuing to remove components with large and positive mutual information will give a density mixture model with an optimal structure, which is very close to the true pdf.

PDF TPAMI.pdf - Other
Restricted to Registered users only
Download (619kB)

More information

Published date: April 2001
Organisations: EEE

Identifiers

Local EPrints ID: 257384
URI: http://eprints.soton.ac.uk/id/eprint/257384
PURE UUID: 579ad1b0-a0af-4b48-887d-d264758afe62
ORCID for M. Zwolinski: ORCID iD orcid.org/0000-0002-2230-625X

Catalogue record

Date deposited: 06 May 2003
Last modified: 18 Jul 2017 09:39

Export record

Contributors

Author: M. Zwolinski ORCID iD
Author: Z.R. Yang

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×