The University of Southampton
University of Southampton Institutional Repository

Robust Automatic Methods for Outlier and Error Detection

Robust Automatic Methods for Outlier and Error Detection
Robust Automatic Methods for Outlier and Error Detection
Editing in surveys of economic populations is often complicated by the fact that outliers due to errors in the data are mixed in with correct, but extreme, data values. In this paper we describe and evaluate two automatic techniques for error identification in such long tailed data distributions. The first is a forward search procedure based on finding a sequence of error-free subsets of the error contaminated data and then using regression modelling within these subsets to identify errors. The second uses a robust regression tree modelling procedure to identify errors. Both approaches can be implemented on a univariate basis or on a multivariate basis. An application to a business survey data set that contains a mix of extreme errors and true outliers is
described.
M03/17
Southampton Statistical Sciences Research Institute, University of Southampton
Chambers, Ray
96331700-f45e-4483-a887-fef921888ff2
Hentges, Adão
fe5f9c66-e8cf-40f5-83b7-c69d95277849
Zhao, Xinqiang
624c169e-a7fa-4a9a-bc78-a9c1b9d93992
Chambers, Ray
96331700-f45e-4483-a887-fef921888ff2
Hentges, Adão
fe5f9c66-e8cf-40f5-83b7-c69d95277849
Zhao, Xinqiang
624c169e-a7fa-4a9a-bc78-a9c1b9d93992

Chambers, Ray, Hentges, Adão and Zhao, Xinqiang (2003) Robust Automatic Methods for Outlier and Error Detection (S3RI Methodology Working Papers, M03/17) Southampton, UK. Southampton Statistical Sciences Research Institute, University of Southampton 29pp.

Record type: Monograph (Project Report)

Abstract

Editing in surveys of economic populations is often complicated by the fact that outliers due to errors in the data are mixed in with correct, but extreme, data values. In this paper we describe and evaluate two automatic techniques for error identification in such long tailed data distributions. The first is a forward search procedure based on finding a sequence of error-free subsets of the error contaminated data and then using regression modelling within these subsets to identify errors. The second uses a robust regression tree modelling procedure to identify errors. Both approaches can be implemented on a univariate basis or on a multivariate basis. An application to a business survey data set that contains a mix of extreme errors and true outliers is
described.

Text
8167-01.pdf - Other
Download (3MB)

More information

Published date: 2003

Identifiers

Local EPrints ID: 8167
URI: http://eprints.soton.ac.uk/id/eprint/8167
PURE UUID: 4d17cb45-a9c1-4860-a3a4-4c1414319735

Catalogue record

Date deposited: 11 Jul 2004
Last modified: 15 Mar 2024 04:51

Export record

Contributors

Author: Ray Chambers
Author: Adão Hentges
Author: Xinqiang Zhao

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×