A fast importance sampling algorithm for unsupervised learning of over-complete dictionaries

Blumensath, T. and Davies, M.E. (2005) A fast importance sampling algorithm for unsupervised learning of over-complete dictionaries. IEEE International Conference on Acoustics, Speech and Signal Processing, Philadelphia, United States. 18 - 23 Mar 2005. pp. 213-216 .

Record type: Conference or Workshop Item (Paper)

Abstract

We use Bayesian statistics to study the dictionary learning problem in which an over-complete generative signal model has to be adapted for optimally sparse signal representations. With such a formulation we develop a stochastic gradient learning algorithm based on Importance Sampling techniques to minimise the negative marginal log-likelihood. As this likelihood is not available analytically, approximations have to be utilised. The Importance Sampling Monte Carlo marginalisation proposed here improves on previous methods and addresses three main issues: 1) bias of the gradient estimate; 2) multi-modality of the distribution to be approximated; and 3) computational efficiency.
Experimental results show the advantages of the new method when compared to previous techniques. The gained efficiency allows the treatment of large scale problems in a statistically sound framework as demonstrated here by the extraction of individual piano notes from a polyphonic piano recording.

This record has no associated files available for download.