The University of Southampton
University of Southampton Institutional Repository

Monte Carlo methods for adaptive sparse approximations of time-series

Monte Carlo methods for adaptive sparse approximations of time-series
Monte Carlo methods for adaptive sparse approximations of time-series
This paper deals with adaptive sparse approximations of time-series. The work is based on a Bayesian specification of the shift-invariant sparse coding model. To learn approximations for a particular class of signals, two different learning strategies are discussed. The first method uses a gradient optimization technique commonly employed in sparse coding problems.

The other method is novel in this context and is based on a sampling estimate. To approximate the gradient in the first approach we compare two Monte Carlo estimation techniques, Gibbs sampling and a novel importance sampling method. The second approach is based on a direct sample estimate and uses an extension of the Gibbs sampler used with the first approach. Both approaches allow the specification of different prior distributions and we here introduce a novel mixture prior based on a modified Rayleigh distribution. Experiments demonstrate that all Gibbs sampler based methods show comparable performance.

The importance sampler was found to work nearly as well as the Gibbs sampler on smaller problems in terms of estimating the model parameters, however, the method performed substantially worse on estimating the sparse coefficients. For large problems we found that the combination of a subset selection heuristic with the Gibbs sampling approaches can outperform previous suggested methods.

In addition, the methods studied here are flexible and allow the incorporation of additional prior knowledge, such as the nonnegativity of the approximation coefficients, which was found to offer additional benefits where applicable.
importance and gibbs sampling, monte carlo approximation, sparse approximation, time-series modeling
1053-587X
4474-4481
Blumensath, Thomas
470d9055-0373-457e-bf80-4389f8ec4ead
Davies, Mike E.
9ca3625e-5b14-4f1f-90ac-1af468f521ae
Blumensath, Thomas
470d9055-0373-457e-bf80-4389f8ec4ead
Davies, Mike E.
9ca3625e-5b14-4f1f-90ac-1af468f521ae

Blumensath, Thomas and Davies, Mike E. (2007) Monte Carlo methods for adaptive sparse approximations of time-series. IEEE Transactions on Signal Processing, 55 (9), 4474-4481. (doi:10.1109/TSP.2007.896242).

Record type: Article

Abstract

This paper deals with adaptive sparse approximations of time-series. The work is based on a Bayesian specification of the shift-invariant sparse coding model. To learn approximations for a particular class of signals, two different learning strategies are discussed. The first method uses a gradient optimization technique commonly employed in sparse coding problems.

The other method is novel in this context and is based on a sampling estimate. To approximate the gradient in the first approach we compare two Monte Carlo estimation techniques, Gibbs sampling and a novel importance sampling method. The second approach is based on a direct sample estimate and uses an extension of the Gibbs sampler used with the first approach. Both approaches allow the specification of different prior distributions and we here introduce a novel mixture prior based on a modified Rayleigh distribution. Experiments demonstrate that all Gibbs sampler based methods show comparable performance.

The importance sampler was found to work nearly as well as the Gibbs sampler on smaller problems in terms of estimating the model parameters, however, the method performed substantially worse on estimating the sparse coefficients. For large problems we found that the combination of a subset selection heuristic with the Gibbs sampling approaches can outperform previous suggested methods.

In addition, the methods studied here are flexible and allow the incorporation of additional prior knowledge, such as the nonnegativity of the approximation coefficients, which was found to offer additional benefits where applicable.

Text
BDMC07.pdf - Other
Download (712kB)

More information

Published date: September 2007
Keywords: importance and gibbs sampling, monte carlo approximation, sparse approximation, time-series modeling
Organisations: Signal Processing & Control Grp

Identifiers

Local EPrints ID: 142527
URI: https://eprints.soton.ac.uk/id/eprint/142527
ISSN: 1053-587X
PURE UUID: 35223720-14ef-49f0-8433-000dcfa95ab7

Catalogue record

Date deposited: 31 Mar 2010 15:58
Last modified: 04 Nov 2019 21:05

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×