Material recognition for immersive interactions in virtual/augmented reality

Heng, Yuwen, Dasmahapatra, Srinandan and Kim, Hansung (2023) Material recognition for immersive interactions in virtual/augmented reality. In Proceedings of the 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023. IEEE. pp. 577-578 . (doi:10.1109/VRW58643.2023.00131).

Record type: Conference or Workshop Item (Paper)

Abstract

To provide an immersive experience in a mirrored virtual world such as spatially synchronised audio, visualisation of reproduced real-world scenes and haptic sensing, it is necessary to know the materials of the object surface which provides the optical and acous-tic properties for the rendering engine. We focus on identifying materials from real-world images to reproduce more realistic and plausible virtual environments. To cope with considerable variation in material, we propose the DPT architecture which dynamically decides the dependency on different patch resolutions. We evaluate the benefits of learning from multiple patch resolutions on LMD and OpenSurfaces datasets.

This record has no associated files available for download.

More information

Published date: 1 May 2023

Additional Information: Funding Information: This work was partially supported by the EPSRC Programme Grant Immersive Audio-Visual 3D Scene Reproduction (EP/V03538X/1) and partially by the Korea Institute of Science and Technology (KIST) Institutional Program (Project No. 2E31591).

Venue - Dates: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023, , Shanghai, China, 2023-03-25 - 2023-03-29

Keywords: artificial intelligence, computer vision-scene understanding, computing methodologies, human-centered computing-Human computer in-teraction (HCI), interaction paradigms-virtual reality