Estimation of Gaussian mixture models via tensor moments with application to online learning
Estimation of Gaussian mixture models via tensor moments with application to online learning
In this paper, we present an alternating gradient descent algorithm for estimating parameters of a spherical Gaussian mixture model by the method of moments (AGD-MoM). We formulate the problem as a constrained optimisation problem which simultaneously matches the third order moments from the data, represented as a tensor, and the second order moment, which is the empirical covariance matrix. We derive the necessary gradients (and second derivatives), and use them to implement alternating gradient search to estimate the parameters of the model. We show that the proposed method is applicable in both a batch as well as in a streaming (online) setting. Using synthetic and benchmark datasets, we demonstrate empirically that the proposed algorithm outperforms the more classical algorithms like Expectation Maximisation and variational Bayes.
Alternating gradient descent, Method of moments, Online learning, Tensor analysis
285-292
Brodzki, Jacek
b1fe25fd-5451-4fd0-b24b-c59b75710543
Rahmani, Donya
94dd9b62-8a78-4e4b-9008-2d29aaf3e47f
Fay, Damien
b0ca1e4b-82a6-44a2-aff0-938465e954f4
Takeda, Akiko
a91c34d7-5fc7-4c89-a810-04fa071956d5
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f
March 2020
Brodzki, Jacek
b1fe25fd-5451-4fd0-b24b-c59b75710543
Rahmani, Donya
94dd9b62-8a78-4e4b-9008-2d29aaf3e47f
Fay, Damien
b0ca1e4b-82a6-44a2-aff0-938465e954f4
Takeda, Akiko
a91c34d7-5fc7-4c89-a810-04fa071956d5
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f
Brodzki, Jacek, Rahmani, Donya, Fay, Damien, Takeda, Akiko and Niranjan, Mahesan
(2020)
Estimation of Gaussian mixture models via tensor moments with application to online learning.
Pattern Recognition Letters, 131, .
(doi:10.1016/j.patrec.2020.01.001).
Abstract
In this paper, we present an alternating gradient descent algorithm for estimating parameters of a spherical Gaussian mixture model by the method of moments (AGD-MoM). We formulate the problem as a constrained optimisation problem which simultaneously matches the third order moments from the data, represented as a tensor, and the second order moment, which is the empirical covariance matrix. We derive the necessary gradients (and second derivatives), and use them to implement alternating gradient search to estimate the parameters of the model. We show that the proposed method is applicable in both a batch as well as in a streaming (online) setting. Using synthetic and benchmark datasets, we demonstrate empirically that the proposed algorithm outperforms the more classical algorithms like Expectation Maximisation and variational Bayes.
Text
Rahmani, Fey, Niranjan, Takeda, Brodzki - Estimation of Gaussian Mixture Models via Tensor Moments with Application to Online Learning article
- Accepted Manuscript
More information
Accepted/In Press date: 2 January 2020
e-pub ahead of print date: 3 January 2020
Published date: March 2020
Keywords:
Alternating gradient descent, Method of moments, Online learning, Tensor analysis
Identifiers
Local EPrints ID: 437158
URI: http://eprints.soton.ac.uk/id/eprint/437158
ISSN: 0167-8655
PURE UUID: cb3a32f3-7a6d-47bc-bc6a-8673fab0c343
Catalogue record
Date deposited: 20 Jan 2020 17:31
Last modified: 17 Mar 2024 05:13
Export record
Altmetrics
Contributors
Author:
Donya Rahmani
Author:
Damien Fay
Author:
Akiko Takeda
Author:
Mahesan Niranjan
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics