The University of Southampton
University of Southampton Institutional Repository

Elastic net orthogonal forward regression

Elastic net orthogonal forward regression
Elastic net orthogonal forward regression
An efficient two-level model identification method aiming at maximising a model's generalisation capability is proposed for a large class of linear-in-the-parameters models from the observational data. A new elastic net orthogonal forward regression (ENOFR) algorithm is employed at the lower level to carry out simultaneous model selection and elastic net parameter estimation. The two regularisation parameters in the elastic net are optimised using a particle swarm optimisation (PSO) algorithm at the upper level by minimising the leave one out (LOO) mean square error (LOOMSE). There are two elements of original contributions. Firstly a nelastic net cost function is defined and applied based on orthogonal decomposition, which facilitates the automatic model structure selection process with no need of using a predetermined error tolerance to terminate the forward selection process. Secondly it is shown that the LOOMSE based on the resultant ENOFR models can be analytically computed without actually splitting the data set, and the associate computation cost is small due to the ENOFR procedure. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative modelevaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.
0925-2312
551-560
Hong, Xia
e6551bb3-fbc0-4990-935e-43b706d8c679
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80
Hong, Xia
e6551bb3-fbc0-4990-935e-43b706d8c679
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80

Hong, Xia and Chen, Sheng (2015) Elastic net orthogonal forward regression. Neurocomputing, 148, 551-560. (doi:10.1016/j.neucom.2014.07.008).

Record type: Article

Abstract

An efficient two-level model identification method aiming at maximising a model's generalisation capability is proposed for a large class of linear-in-the-parameters models from the observational data. A new elastic net orthogonal forward regression (ENOFR) algorithm is employed at the lower level to carry out simultaneous model selection and elastic net parameter estimation. The two regularisation parameters in the elastic net are optimised using a particle swarm optimisation (PSO) algorithm at the upper level by minimising the leave one out (LOO) mean square error (LOOMSE). There are two elements of original contributions. Firstly a nelastic net cost function is defined and applied based on orthogonal decomposition, which facilitates the automatic model structure selection process with no need of using a predetermined error tolerance to terminate the forward selection process. Secondly it is shown that the LOOMSE based on the resultant ENOFR models can be analytically computed without actually splitting the data set, and the associate computation cost is small due to the ENOFR procedure. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative modelevaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.

PDF
NEUROC2015-1.pdf - Version of Record
Restricted to Repository staff only
Request a copy

More information

Published date: 19 January 2015
Organisations: Southampton Wireless Group

Identifiers

Local EPrints ID: 369489
URI: https://eprints.soton.ac.uk/id/eprint/369489
ISSN: 0925-2312
PURE UUID: d9f95021-1e82-41c9-9ceb-274b7e50a14c

Catalogue record

Date deposited: 01 Oct 2014 11:32
Last modified: 17 Jul 2017 21:56

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×