On the equivalence between Total Least Squares and Maximum Likelihood PCA

Schuermans, M., Markovsky, I., Wentzell, P. and Van Huffel, S. (2005) On the equivalence between Total Least Squares and Maximum Likelihood PCA. Analytica Chimica Acta, 544, (1-2), 254-267.


[img] PDF
Download (306Kb)


The maximum likelihood PCA (MLPCA) method has been devised in chemometrics as a generalization of the well-known PCA method in order to derive consistent estimators in the presence of errors with known error distribution. For similar reasons, the total least squares (TLS) method has been generalized in the field of computational mathematics and engineering to maintain consistency of the parameter estimates in linear models with measurement errors of known distribution. The basic motivation for TLS is the following. Let a set of multidimensional data points (vectors) be given. How can one obtain a linear model that explains these data? The idea is to modify all data points in such a way that some norm of the modification is minimized subject to the constraint that the modified vectors satisfy a linear relation. Although the name “total least squares” appeared in the literature only 25 years ago, this method of fitting is certainly not new and has a long history in the statistical literature, where the method is known as “orthogonal regression”, “errors-in-variables regression” or “measurement error modeling”. The purpose of this paper is to explore the tight equivalences between MLPCA and element-wise weighted TLS (EW-TLS). Despite their seemingly different problem formulation, it is shown that both methods can be reduced to the same mathematical kernel problem, i.e. finding the closest (in a certain sense) weighted low rank matrix approximation where the weight is derived from the distribution of the errors in the data. Different solution approaches, as used in MLPCA and EW-TLS, are discussed. In particular, we will discuss the weighted low rank approximation (WLRA), the MLPCA, the EW-TLS and the generalized TLS (GTLS) problems. These four approaches tackle an equivalent weighted low rank approximation problem, but different algorithms are used to come up with the best approximation matrix. We will compare their computation times on chemical data and discuss their convergence behavior.

Item Type: Article
Related URLs:
Keywords: TLS; MLPCA; Rank reduction; Measurement errors
Divisions : Faculty of Physical Sciences and Engineering > Electronics and Computer Science > Southampton Wireless Group
ePrint ID: 263302
Accepted Date and Publication Date:
Date Deposited: 06 Jan 2007
Last Modified: 31 Mar 2016 14:07
Further Information:Google Scholar
URI: http://eprints.soton.ac.uk/id/eprint/263302

Actions (login required)

View Item View Item

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics