The University of Southampton
University of Southampton Institutional Repository

Probability density estimation with tunable kernels using orthogonal forward regression

Probability density estimation with tunable kernels using orthogonal forward regression
Probability density estimation with tunable kernels using orthogonal forward regression
A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately.
1083-4419
1101-1114
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80
Hong, Xia
e6551bb3-fbc0-4990-935e-43b706d8c679
Harris, Chris J.
c4fd3763-7b3f-4db1-9ca3-5501080f797a
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80
Hong, Xia
e6551bb3-fbc0-4990-935e-43b706d8c679
Harris, Chris J.
c4fd3763-7b3f-4db1-9ca3-5501080f797a

Chen, Sheng, Hong, Xia and Harris, Chris J. (2010) Probability density estimation with tunable kernels using orthogonal forward regression IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 40, (4), pp. 1101-1114.

Record type: Article

Abstract

A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately.

Text SMCpB10-8.pdf - Version of Record
Download (443kB)

More information

Published date: August 2010
Organisations: Southampton Wireless Group

Identifiers

Local EPrints ID: 271410
URI: https://eprints.soton.ac.uk/id/eprint/271410
ISSN: 1083-4419
PURE UUID: 33595438-e5fd-4d7c-93d1-08c2f4594f7a

Catalogue record

Date deposited: 16 Jul 2010 11:23
Last modified: 11 Nov 2017 10:40

Export record

Contributors

Author: Sheng Chen
Author: Xia Hong
Author: Chris J. Harris

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Library staff edit
Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×