The University of Southampton
University of Southampton Institutional Repository

Estimation of Gaussian mixture models via tensor moments with application to online learning

Estimation of Gaussian mixture models via tensor moments with application to online learning
Estimation of Gaussian mixture models via tensor moments with application to online learning
In this paper, we present an alternating gradient descent algorithm for estimating parameters of a spherical Gaussian mixture model by the method of moments (AGD-MoM). We formulate the problem as a constrained optimisation problem which simultaneously matches the third order moments from the data, represented as a tensor, and the second order moment, which is the empirical covariance matrix. We derive the necessary gradients (and second derivatives), and use them to implement alternating gradient search to estimate the parameters of the model. We show that the proposed method is applicable in both a batch as well as in a streaming (online) setting. Using synthetic and benchmark datasets, we demonstrate empirically that the proposed algorithm outperforms the more classical algorithms like Expectation Maximisation and variational Bayes.
Alternating gradient descent, Method of moments, Online learning, Tensor analysis
0167-8655
285-292
Brodzki, Jacek
b1fe25fd-5451-4fd0-b24b-c59b75710543
Rahmani, Donya
94dd9b62-8a78-4e4b-9008-2d29aaf3e47f
Fay, Damien
b0ca1e4b-82a6-44a2-aff0-938465e954f4
Takeda, Akiko
a91c34d7-5fc7-4c89-a810-04fa071956d5
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f
Brodzki, Jacek
b1fe25fd-5451-4fd0-b24b-c59b75710543
Rahmani, Donya
94dd9b62-8a78-4e4b-9008-2d29aaf3e47f
Fay, Damien
b0ca1e4b-82a6-44a2-aff0-938465e954f4
Takeda, Akiko
a91c34d7-5fc7-4c89-a810-04fa071956d5
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f

Brodzki, Jacek, Rahmani, Donya, Fay, Damien, Takeda, Akiko and Niranjan, Mahesan (2020) Estimation of Gaussian mixture models via tensor moments with application to online learning. Pattern Recognition Letters, 131, 285-292. (doi:10.1016/j.patrec.2020.01.001).

Record type: Article

Abstract

In this paper, we present an alternating gradient descent algorithm for estimating parameters of a spherical Gaussian mixture model by the method of moments (AGD-MoM). We formulate the problem as a constrained optimisation problem which simultaneously matches the third order moments from the data, represented as a tensor, and the second order moment, which is the empirical covariance matrix. We derive the necessary gradients (and second derivatives), and use them to implement alternating gradient search to estimate the parameters of the model. We show that the proposed method is applicable in both a batch as well as in a streaming (online) setting. Using synthetic and benchmark datasets, we demonstrate empirically that the proposed algorithm outperforms the more classical algorithms like Expectation Maximisation and variational Bayes.

Text
Rahmani, Fey, Niranjan, Takeda, Brodzki - Estimation of Gaussian Mixture Models via Tensor Moments with Application to Online Learning article - Accepted Manuscript
Download (6MB)

More information

Accepted/In Press date: 2 January 2020
e-pub ahead of print date: 3 January 2020
Published date: March 2020
Keywords: Alternating gradient descent, Method of moments, Online learning, Tensor analysis

Identifiers

Local EPrints ID: 437158
URI: http://eprints.soton.ac.uk/id/eprint/437158
ISSN: 0167-8655
PURE UUID: cb3a32f3-7a6d-47bc-bc6a-8673fab0c343
ORCID for Jacek Brodzki: ORCID iD orcid.org/0000-0002-4524-1081
ORCID for Mahesan Niranjan: ORCID iD orcid.org/0000-0001-7021-140X

Catalogue record

Date deposited: 20 Jan 2020 17:31
Last modified: 17 Mar 2024 05:13

Export record

Altmetrics

Contributors

Author: Jacek Brodzki ORCID iD
Author: Donya Rahmani
Author: Damien Fay
Author: Akiko Takeda
Author: Mahesan Niranjan ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×