An audio-visual method for room boundary estimation and material recognition
An audio-visual method for room boundary estimation and material recognition
In applications such as virtual and augmented reality, a plausible and coherent audio-visual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audio-visual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a vision-based method. Results show 89 % and 80 % agreement between the estimated and reference room volumes and materials, respectively. © 2018 Association for Computing Machinery.
Acoustic wave absorption, Architectural acoustics, Augmented reality, Learning systems, Reverberation, Acoustic absorption coefficients, Audio-visual, Boundary estimation, Machine learning methods, Material recognition, Scene understanding, Virtual and augmented reality, Vision-based methods, Audio acoustics
3-9
Association for Computing Machinery
Remaggi, L.
c74406cb-15d2-4575-b086-97b55421649e
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Jackson, P.J.B.
8ffa8744-bd52-4f35-9632-d5c73dc92b4b
Hilton, A.
12782a55-4c4d-4dfb-a690-62505f6665db
October 2018
Remaggi, L.
c74406cb-15d2-4575-b086-97b55421649e
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Jackson, P.J.B.
8ffa8744-bd52-4f35-9632-d5c73dc92b4b
Hilton, A.
12782a55-4c4d-4dfb-a690-62505f6665db
Remaggi, L., Kim, H., Jackson, P.J.B. and Hilton, A.
(2018)
An audio-visual method for room boundary estimation and material recognition.
In AVSU'18: Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia.
Association for Computing Machinery.
.
(doi:10.1145/3264869.3264876).
Record type:
Conference or Workshop Item
(Paper)
Abstract
In applications such as virtual and augmented reality, a plausible and coherent audio-visual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audio-visual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a vision-based method. Results show 89 % and 80 % agreement between the estimated and reference room volumes and materials, respectively. © 2018 Association for Computing Machinery.
This record has no associated files available for download.
More information
Published date: October 2018
Additional Information:
cited By 0
Venue - Dates:
ACM Mulrtimedia Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, 2018-10-10
Keywords:
Acoustic wave absorption, Architectural acoustics, Augmented reality, Learning systems, Reverberation, Acoustic absorption coefficients, Audio-visual, Boundary estimation, Machine learning methods, Material recognition, Scene understanding, Virtual and augmented reality, Vision-based methods, Audio acoustics
Identifiers
Local EPrints ID: 440624
URI: http://eprints.soton.ac.uk/id/eprint/440624
PURE UUID: fd0f0579-c049-4a51-82f1-f0bcb3f48c8e
Catalogue record
Date deposited: 12 May 2020 16:46
Last modified: 17 Mar 2024 04:01
Export record
Altmetrics
Contributors
Author:
L. Remaggi
Author:
H. Kim
Author:
P.J.B. Jackson
Author:
A. Hilton
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics