A fast importance sampling algorithm for unsupervised learning of over-complete dictionaries

Blumensath, T. and Davies, M.E. (2005) A fast importance sampling algorithm for unsupervised learning of over-complete dictionaries. In, IEEE International Conference on Acoustics, Speech and Signal Processing, Philadelphia, US, 18 - 23 Mar 2005. Institute of Electrical and Electronics Engineers , 213-216.


Full text not available from this repository.


We use Bayesian statistics to study the dictionary learning problem
in which an over-complete generative signal model has to be
adapted for optimally sparse signal representations. With such a
formulation we develop a stochastic gradient learning algorithm
based on Importance Sampling techniques to minimise the negative
marginal log-likelihood. As this likelihood is not available
analytically, approximations have to be utilised. The Importance
Sampling Monte Carlo marginalisation proposed here improves on
previous methods and addresses three main issues: 1) bias of the
gradient estimate; 2) multi-modality of the distribution to be approximated;
and 3) computational efficiency. Experimental results
show the advantages of the new method when compared to previous
techniques. The gained efficiency allows the treatment of large
scale problems in a statistically sound framework as demonstrated
here by the extraction of individual piano notes from a polyphonic
piano recording

Item Type: Conference or Workshop Item (Paper)
Related URLs:
Subjects: Q Science > QA Mathematics
Q Science > QC Physics
Divisions : University Structure - Pre August 2011 > School of Mathematics
Faculty of Engineering and the Environment > Institute of Sound and Vibration Research > Signal Processing & Control Research Group
ePrint ID: 151935
Accepted Date and Publication Date:
March 2005Published
Date Deposited: 01 Jul 2010 10:42
Last Modified: 31 Mar 2016 13:24
URI: http://eprints.soton.ac.uk/id/eprint/151935

Actions (login required)

View Item View Item