The University of Southampton
University of Southampton Institutional Repository

An audio-visual method for room boundary estimation and material recognition

An audio-visual method for room boundary estimation and material recognition
An audio-visual method for room boundary estimation and material recognition
In applications such as virtual and augmented reality, a plausible and coherent audio-visual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audio-visual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a vision-based method. Results show 89 % and 80 % agreement between the estimated and reference room volumes and materials, respectively. © 2018 Association for Computing Machinery.
Acoustic wave absorption, Architectural acoustics, Augmented reality, Learning systems, Reverberation, Acoustic absorption coefficients, Audio-visual, Boundary estimation, Machine learning methods, Material recognition, Scene understanding, Virtual and augmented reality, Vision-based methods, Audio acoustics
3-9
ACM
Remaggi, L.
c74406cb-15d2-4575-b086-97b55421649e
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Jackson, P.J.B.
8ffa8744-bd52-4f35-9632-d5c73dc92b4b
Hilton, A.
12782a55-4c4d-4dfb-a690-62505f6665db
Remaggi, L.
c74406cb-15d2-4575-b086-97b55421649e
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Jackson, P.J.B.
8ffa8744-bd52-4f35-9632-d5c73dc92b4b
Hilton, A.
12782a55-4c4d-4dfb-a690-62505f6665db

Remaggi, L., Kim, H., Jackson, P.J.B. and Hilton, A. (2018) An audio-visual method for room boundary estimation and material recognition. In AVSU'18: Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia. ACM. pp. 3-9 . (doi:10.1145/3264869.3264876).

Record type: Conference or Workshop Item (Paper)

Abstract

In applications such as virtual and augmented reality, a plausible and coherent audio-visual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audio-visual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a vision-based method. Results show 89 % and 80 % agreement between the estimated and reference room volumes and materials, respectively. © 2018 Association for Computing Machinery.

Full text not available from this repository.

More information

Published date: October 2018
Additional Information: cited By 0
Venue - Dates: ACM Mulrtimedia Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, 2018-10-10
Keywords: Acoustic wave absorption, Architectural acoustics, Augmented reality, Learning systems, Reverberation, Acoustic absorption coefficients, Audio-visual, Boundary estimation, Machine learning methods, Material recognition, Scene understanding, Virtual and augmented reality, Vision-based methods, Audio acoustics

Identifiers

Local EPrints ID: 440624
URI: http://eprints.soton.ac.uk/id/eprint/440624
PURE UUID: fd0f0579-c049-4a51-82f1-f0bcb3f48c8e
ORCID for H. Kim: ORCID iD orcid.org/0000-0003-4907-0491

Catalogue record

Date deposited: 12 May 2020 16:46
Last modified: 13 May 2020 00:59

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×