The University of Southampton
University of Southampton Institutional Repository

Audio-visual tracking by density approximation in a sequential Bayesian filtering framework

Audio-visual tracking by density approximation in a sequential Bayesian filtering framework
Audio-visual tracking by density approximation in a sequential Bayesian filtering framework

This paper proposes a novel audio-visual tracking approach that exploits constructively audio and visual modalities in order to estimate trajectories of multiple people in a joint state space. The tracking problem is modeled using a sequential Bayesian filtering framework. Within this framework, we propose to represent the posterior density with a Gaussian Mixture Model (GMM). To ensure that a GMM representation can be retained sequentially over time, the predictive density is approximated by a GMM using the Unscented Transform. While a density interpolation technique is introduced to obtain a continuous representation of the observation likelihood, which is also a GMM. Furthermore, to prevent the number of mixtures from growing exponentially over time, a density approximation based on the Expectation Maximization (EM) algorithm is applied, resulting in a compact GMM representation of the posterior density. Recordings using a camcorder and microphone array are used to evaluate the proposed approach, demonstrating significant improvements in tracking performance of the proposed audio-visual approach compared to two benchmark visual trackers.

Audio-visual systems, Bayes methods, Machine vision, Motion estimation, Speech processing
71-75
Institute of Electrical and Electronics Engineers Inc.
Gebru, Israel D.
de707c41-541c-4b5b-b4c6-d63486b18684
Evers, Christine
93090c84-e984-4cc3-9363-fbf3f3639c4b
Naylor, Patrick A.
13079486-664a-414c-a1a2-01a30bf0997b
Horaud, Radu
86f58721-f833-4200-a8ce-959babf9c522
Gebru, Israel D.
de707c41-541c-4b5b-b4c6-d63486b18684
Evers, Christine
93090c84-e984-4cc3-9363-fbf3f3639c4b
Naylor, Patrick A.
13079486-664a-414c-a1a2-01a30bf0997b
Horaud, Radu
86f58721-f833-4200-a8ce-959babf9c522

Gebru, Israel D., Evers, Christine, Naylor, Patrick A. and Horaud, Radu (2017) Audio-visual tracking by density approximation in a sequential Bayesian filtering framework. In 2017 Hands-Free Speech Communications and Microphone Arrays, HSCMA 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. pp. 71-75 . (doi:10.1109/HSCMA.2017.7895564).

Record type: Conference or Workshop Item (Paper)

Abstract

This paper proposes a novel audio-visual tracking approach that exploits constructively audio and visual modalities in order to estimate trajectories of multiple people in a joint state space. The tracking problem is modeled using a sequential Bayesian filtering framework. Within this framework, we propose to represent the posterior density with a Gaussian Mixture Model (GMM). To ensure that a GMM representation can be retained sequentially over time, the predictive density is approximated by a GMM using the Unscented Transform. While a density interpolation technique is introduced to obtain a continuous representation of the observation likelihood, which is also a GMM. Furthermore, to prevent the number of mixtures from growing exponentially over time, a density approximation based on the Expectation Maximization (EM) algorithm is applied, resulting in a compact GMM representation of the posterior density. Recordings using a camcorder and microphone array are used to evaluate the proposed approach, demonstrating significant improvements in tracking performance of the proposed audio-visual approach compared to two benchmark visual trackers.

Full text not available from this repository.

More information

Published date: 10 April 2017
Venue - Dates: 2017 Hands-Free Speech Communications and Microphone Arrays, HSCMA 2017, , San Francisco, United States, 2017-03-01 - 2017-03-03
Keywords: Audio-visual systems, Bayes methods, Machine vision, Motion estimation, Speech processing

Identifiers

Local EPrints ID: 445098
URI: http://eprints.soton.ac.uk/id/eprint/445098
PURE UUID: 7872dfe0-1ca4-4209-9782-d89c9b0fba71
ORCID for Christine Evers: ORCID iD orcid.org/0000-0003-0757-5504

Catalogue record

Date deposited: 19 Nov 2020 17:32
Last modified: 19 Nov 2020 17:32

Export record

Altmetrics

Contributors

Author: Israel D. Gebru
Author: Christine Evers ORCID iD
Author: Patrick A. Naylor
Author: Radu Horaud

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×