Some greedy learning algorithms for sparse regression and classification with mercer kernels


Nair, Prasanth B., Choudhury, Arindam and Keane, Andy J. (2002) Some greedy learning algorithms for sparse regression and classification with mercer kernels. Journal of Machine Learning Research, 3, 781-801.

Download

[img] PDF - Publishers print
Restricted to System admin

Download (181Kb) | Request a copy

Description/Abstract

We present greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernel-based algorithms applied to large datasets. In the spirit of Natarajan's greedy algorithm (Natarajan, 1995), we iteratively minimize the L2 loss function subject to a specified constraint on the degree of sparsity required of the final model or till a specified stopping criterion is reached. We discuss various greedy criteria for basis selection and numerical schemes for improving the robustness and computational efficiency. Subsequently, algorithms based on residual minimization and thin QR factorization are presented for constructing sparse regression and classification models. During the course of the incremental model construction, the algorithms are terminated using model selection principles such as the minimum descriptive length (MDL) and Akaike's information criterion (AIC). Finally, experimental results on benchmark data are presented to demonstrate the competitiveness of the algorithms developed in this paper.

Item Type: Article
ISSNs: 1532-4435 (print)
Related URLs:
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: University Structure - Pre August 2011 > School of Engineering Sciences
University Structure - Pre August 2011 > School of Electronics and Computer Science
ePrint ID: 22248
Date Deposited: 20 Mar 2006
Last Modified: 27 Mar 2014 18:11
Contact Email Address: P.B.Nair@soton.ac.uk
URI: http://eprints.soton.ac.uk/id/eprint/22248

Actions (login required)

View Item View Item