Designing committees of models through deliberate weighting of data points
Designing committees of models through deliberate weighting of data points
In the adaptive derivation of mathematical models from data, each data point should contribute with a weight reflecting the amount of confidence one has in it. When no additional information for data confidence is available, all the data points should be considered equal, and are also generally given the same weight. In the formation of committees of models, however, this is often not the case and the data points may exercise unequal, even random, influence over the committee formation.
In this paper, a principled approach to committee design is presented. The construction of a committee design matrix is detailed through which each data point will contribute to the committee formation with a fixed weight, while contributing with different individual weights to the derivation of the different constituent models, thus encouraging model diversity whilst not biasing the committee inadvertently towards any particular data points. Not distinctly an algorithm, it is instead a framework within which several different committee approaches may be realised.
Whereas the focus in the paper lies entirely on regression, the principles discussed extend readily to classification.
neural networks, ensembles, committees, bagging, regression
39-66
Christensen, Stephan W.
7bc0383b-1758-4660-bdee-805c7f821379
Sinclair, Ian
6005f6c1-f478-434e-a52d-d310c18ade0d
Reed, Philippa A.S.
8b79d87f-3288-4167-bcfc-c1de4b93ce17
2003
Christensen, Stephan W.
7bc0383b-1758-4660-bdee-805c7f821379
Sinclair, Ian
6005f6c1-f478-434e-a52d-d310c18ade0d
Reed, Philippa A.S.
8b79d87f-3288-4167-bcfc-c1de4b93ce17
Christensen, Stephan W., Sinclair, Ian and Reed, Philippa A.S.
(2003)
Designing committees of models through deliberate weighting of data points.
Journal of Machine Learning Research, 4 (1), .
Abstract
In the adaptive derivation of mathematical models from data, each data point should contribute with a weight reflecting the amount of confidence one has in it. When no additional information for data confidence is available, all the data points should be considered equal, and are also generally given the same weight. In the formation of committees of models, however, this is often not the case and the data points may exercise unequal, even random, influence over the committee formation.
In this paper, a principled approach to committee design is presented. The construction of a committee design matrix is detailed through which each data point will contribute to the committee formation with a fixed weight, while contributing with different individual weights to the derivation of the different constituent models, thus encouraging model diversity whilst not biasing the committee inadvertently towards any particular data points. Not distinctly an algorithm, it is instead a framework within which several different committee approaches may be realised.
Whereas the focus in the paper lies entirely on regression, the principles discussed extend readily to classification.
Text
4-39-christensen.pdf
- Version of Record
Restricted to Registered users only
Request a copy
More information
Published date: 2003
Keywords:
neural networks, ensembles, committees, bagging, regression
Organisations:
Engineering Mats & Surface Engineerg Gp
Identifiers
Local EPrints ID: 43078
URI: http://eprints.soton.ac.uk/id/eprint/43078
PURE UUID: 90e04bb1-d949-4408-bd53-b38292af3ae6
Catalogue record
Date deposited: 10 Jan 2007
Last modified: 16 Mar 2024 02:44
Export record
Contributors
Author:
Stephan W. Christensen
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics