The University of Southampton
University of Southampton Institutional Repository

Some greedy learning algorithms for sparse regression and classification with mercer kernels

Some greedy learning algorithms for sparse regression and classification with mercer kernels
Some greedy learning algorithms for sparse regression and classification with mercer kernels
We present greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernel-based algorithms applied to large datasets. In the spirit of Natarajan's greedy algorithm (Natarajan, 1995), we iteratively minimize the L2 loss function subject to a specified constraint on the degree of sparsity required of the final model or till a specified stopping criterion is reached. We discuss various greedy criteria for basis selection and numerical schemes for improving the robustness and computational efficiency. Subsequently, algorithms based on residual minimization and thin QR factorization are presented for constructing sparse regression and classification models. During the course of the incremental model construction, the algorithms are terminated using model selection principles such as the minimum descriptive length (MDL) and Akaike's information criterion (AIC). Finally, experimental results on benchmark data are presented to demonstrate the competitiveness of the algorithms developed in this paper.
781-801
Nair, Prasanth B.
d4d61705-bc97-478e-9e11-bcef6683afe7
Choudhury, Arindam
defdc858-1c15-45b9-9bd6-642c5c706467
Keane, Andy J.
26d7fa33-5415-4910-89d8-fb3620413def
Nair, Prasanth B.
d4d61705-bc97-478e-9e11-bcef6683afe7
Choudhury, Arindam
defdc858-1c15-45b9-9bd6-642c5c706467
Keane, Andy J.
26d7fa33-5415-4910-89d8-fb3620413def

Nair, Prasanth B., Choudhury, Arindam and Keane, Andy J. (2002) Some greedy learning algorithms for sparse regression and classification with mercer kernels. Journal of Machine Learning Research, 3, 781-801.

Record type: Article

Abstract

We present greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernel-based algorithms applied to large datasets. In the spirit of Natarajan's greedy algorithm (Natarajan, 1995), we iteratively minimize the L2 loss function subject to a specified constraint on the degree of sparsity required of the final model or till a specified stopping criterion is reached. We discuss various greedy criteria for basis selection and numerical schemes for improving the robustness and computational efficiency. Subsequently, algorithms based on residual minimization and thin QR factorization are presented for constructing sparse regression and classification models. During the course of the incremental model construction, the algorithms are terminated using model selection principles such as the minimum descriptive length (MDL) and Akaike's information criterion (AIC). Finally, experimental results on benchmark data are presented to demonstrate the competitiveness of the algorithms developed in this paper.

Text
nair02a.pdf - Version of Record
Restricted to Repository staff only
Request a copy

More information

Published date: 2002

Identifiers

Local EPrints ID: 22248
URI: http://eprints.soton.ac.uk/id/eprint/22248
PURE UUID: 925e7730-7ef7-42a0-a83d-904eff3c7830
ORCID for Andy J. Keane: ORCID iD orcid.org/0000-0001-7993-1569

Catalogue record

Date deposited: 20 Mar 2006
Last modified: 16 Mar 2024 02:53

Export record

Contributors

Author: Prasanth B. Nair
Author: Arindam Choudhury
Author: Andy J. Keane ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×