The University of Southampton
University of Southampton Institutional Repository

On the equivalence between Total Least Squares and Maximum Likelihood PCA

Record type: Article

The maximum likelihood PCA (MLPCA) method has been devised in chemometrics as a generalization of the well-known PCA method in order to derive consistent estimators in the presence of errors with known error distribution. For similar reasons, the total least squares (TLS) method has been generalized in the field of computational mathematics and engineering to maintain consistency of the parameter estimates in linear models with measurement errors of known distribution. The basic motivation for TLS is the following. Let a set of multidimensional data points (vectors) be given. How can one obtain a linear model that explains these data? The idea is to modify all data points in such a way that some norm of the modification is minimized subject to the constraint that the modified vectors satisfy a linear relation. Although the name “total least squares” appeared in the literature only 25 years ago, this method of fitting is certainly not new and has a long history in the statistical literature, where the method is known as “orthogonal regression”, “errors-in-variables regression” or “measurement error modeling”. The purpose of this paper is to explore the tight equivalences between MLPCA and element-wise weighted TLS (EW-TLS). Despite their seemingly different problem formulation, it is shown that both methods can be reduced to the same mathematical kernel problem, i.e. finding the closest (in a certain sense) weighted low rank matrix approximation where the weight is derived from the distribution of the errors in the data. Different solution approaches, as used in MLPCA and EW-TLS, are discussed. In particular, we will discuss the weighted low rank approximation (WLRA), the MLPCA, the EW-TLS and the generalized TLS (GTLS) problems. These four approaches tackle an equivalent weighted low rank approximation problem, but different algorithms are used to come up with the best approximation matrix. We will compare their computation times on chemical data and discuss their convergence behavior.

PDF mieke.pdf - Other
Download (313kB)


Schuermans, M., Markovsky, I., Wentzell, P. and Van Huffel, S. (2005) On the equivalence between Total Least Squares and Maximum Likelihood PCA Analytica Chimica Acta, 544, (1-2), pp. 254-267.

More information

Published date: 2005
Keywords: TLS, MLPCA, Rank reduction, Measurement errors
Organisations: Southampton Wireless Group


Local EPrints ID: 263302
ISSN: 0003-2670
PURE UUID: e61f88fb-368f-4c9b-ac7e-6af3e4750fb0

Catalogue record

Date deposited: 06 Jan 2007
Last modified: 18 Jul 2017 07:47

Export record


Author: M. Schuermans
Author: I. Markovsky
Author: P. Wentzell
Author: S. Van Huffel

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton:

ePrints Soton supports OAI 2.0 with a base URL of

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.