The University of Southampton
University of Southampton Institutional Repository

Structural Modelling with Sparse Kernels

Structural Modelling with Sparse Kernels
Structural Modelling with Sparse Kernels
A widely acknowledged drawback of many statistical modelling techniques, commonly used in machine learning, is that the resulting model is extremely difficult to interpret. A number of new concepts and algorithms have been introduced by researchers to address this problem. They focus primarily on determining which inputs are relevant in predicting the output. This work describes a transparent, advanced non-linear modelling approach that enables the constructed predictive models to be visualised, allowing model validation and assisting in interpretation. The technique combines the representational advantage of a sparse ANOVA decomposition, with the good generalisation ability of a kernel machine. It achieves this by employing two forms of regularisation: a 1-norm based structural regulariser to enforce transparency, and a 2-norm based regulariser to control smoothness. The resulting model structure can be visualised showing the overall effects of different inputs, their interactions, and the strength of the interactions. The robustness of the technique is illustrated using a range of both artificial and "real world" datasets. The performance is compared to other modelling techniques, and it is shown to exhibit competitive generalisation performance together with improved interpretability.
137-163
Gunn, S.R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
Kandola, J.S.
c976459a-d502-4688-b741-334c06796ca8
Gunn, S.R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
Kandola, J.S.
c976459a-d502-4688-b741-334c06796ca8

Gunn, S.R. and Kandola, J.S. (2002) Structural Modelling with Sparse Kernels. Machine Learning, 48 (1), 137-163.

Record type: Article

Abstract

A widely acknowledged drawback of many statistical modelling techniques, commonly used in machine learning, is that the resulting model is extremely difficult to interpret. A number of new concepts and algorithms have been introduced by researchers to address this problem. They focus primarily on determining which inputs are relevant in predicting the output. This work describes a transparent, advanced non-linear modelling approach that enables the constructed predictive models to be visualised, allowing model validation and assisting in interpretation. The technique combines the representational advantage of a sparse ANOVA decomposition, with the good generalisation ability of a kernel machine. It achieves this by employing two forms of regularisation: a 1-norm based structural regulariser to enforce transparency, and a 2-norm based regulariser to control smoothness. The resulting model structure can be visualised showing the overall effects of different inputs, their interactions, and the strength of the interactions. The robustness of the technique is illustrated using a range of both artificial and "real world" datasets. The performance is compared to other modelling techniques, and it is shown to exhibit competitive generalisation performance together with improved interpretability.

Text
SparseKernel.pdf - Other
Download (1MB)

More information

Published date: 2002
Organisations: Electronic & Software Systems

Identifiers

Local EPrints ID: 256458
URI: https://eprints.soton.ac.uk/id/eprint/256458
PURE UUID: d17a7229-6fdc-47c8-84cb-bf7126fb35eb

Catalogue record

Date deposited: 01 Mar 2005
Last modified: 30 Jul 2019 19:02

Export record

Contributors

Author: S.R. Gunn
Author: J.S. Kandola

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×