The University of Southampton
University of Southampton Institutional Repository

Multimodal visual data registration for web-based visualization in media production

Multimodal visual data registration for web-based visualization in media production
Multimodal visual data registration for web-based visualization in media production
Recent developments of video and sensing technology have led to large volumes of digital media data. Current media production relies on videos from the principal camera together with a wide variety of heterogeneous source of supporting data [photos, light detection and ranging point clouds, witness video camera, high dynamic range imaging, and depth imagery]. Registration of visual data acquired from various 2D and 3D sensing modalities is challenging because current matching and registration methods are not appropriate due to differences in structure, format, and noise characteristics for multimodal data. A combined 2D/3D visualization of this registered data allows an integrated overview of the entire data set. For such a visualization, a Web-based context presents several advantages. In this paper, we propose a unified framework for registration and visualization of this type of visual media data. A new feature description and matching method is proposed, adaptively considering local geometry, semiglobal geometry, and color information in the scene for more robust registration. The resulting registered 2D/3D multimodal visual data are too large to be downloaded and viewed directly via the Web browser, while maintaining an acceptable user experience. Thus, we employ hierarchical techniques for compression and restructuring to enable efficient transmission and visualization over the Web, leading to interactive visualization as registered point clouds, 2D images, and videos in the browser, improving on the current state-of-the-art techniques for Web-based visualization of big media data. This is the first unified 3D Web-based visualization of multimodal visual media production data sets. The proposed pipeline is tested on big multimodal data set typical of film and broadcast production, which are made publicly available. The
proposed feature description method shows two times higher precision of feature matching and more stable registration performance than existing 3D feature descriptors.
863 - 877
Kim, Hansung
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Evans, Alun
e346913a-22f1-4bdf-808d-01f084ec887f
Blat, Josep
25b3e3d8-930e-4d0a-82c3-1d98c0aa38dc
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Kim, Hansung
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Evans, Alun
e346913a-22f1-4bdf-808d-01f084ec887f
Blat, Josep
25b3e3d8-930e-4d0a-82c3-1d98c0aa38dc
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db

Kim, Hansung, Evans, Alun, Blat, Josep and Hilton, Adrian (2018) Multimodal visual data registration for web-based visualization in media production. IEEE Transactions on Circuits and Systems for Video Technology, 28 (4), 863 - 877. (doi:10.1109/TCSVT.2016.2642825).

Record type: Article

Abstract

Recent developments of video and sensing technology have led to large volumes of digital media data. Current media production relies on videos from the principal camera together with a wide variety of heterogeneous source of supporting data [photos, light detection and ranging point clouds, witness video camera, high dynamic range imaging, and depth imagery]. Registration of visual data acquired from various 2D and 3D sensing modalities is challenging because current matching and registration methods are not appropriate due to differences in structure, format, and noise characteristics for multimodal data. A combined 2D/3D visualization of this registered data allows an integrated overview of the entire data set. For such a visualization, a Web-based context presents several advantages. In this paper, we propose a unified framework for registration and visualization of this type of visual media data. A new feature description and matching method is proposed, adaptively considering local geometry, semiglobal geometry, and color information in the scene for more robust registration. The resulting registered 2D/3D multimodal visual data are too large to be downloaded and viewed directly via the Web browser, while maintaining an acceptable user experience. Thus, we employ hierarchical techniques for compression and restructuring to enable efficient transmission and visualization over the Web, leading to interactive visualization as registered point clouds, 2D images, and videos in the browser, improving on the current state-of-the-art techniques for Web-based visualization of big media data. This is the first unified 3D Web-based visualization of multimodal visual media production data sets. The proposed pipeline is tested on big multimodal data set typical of film and broadcast production, which are made publicly available. The
proposed feature description method shows two times higher precision of feature matching and more stable registration performance than existing 3D feature descriptors.

Full text not available from this repository.

More information

Accepted/In Press date: 7 December 2016
e-pub ahead of print date: 21 December 2016
Published date: April 2018

Identifiers

Local EPrints ID: 438833
URI: http://eprints.soton.ac.uk/id/eprint/438833
PURE UUID: 8dbe2a17-b22f-42f2-aa16-74d884d2b7d5
ORCID for Hansung Kim: ORCID iD orcid.org/0000-0003-4907-0491

Catalogue record

Date deposited: 25 Mar 2020 17:31
Last modified: 07 Oct 2020 02:27

Export record

Altmetrics

Contributors

Author: Hansung Kim ORCID iD
Author: Alun Evans
Author: Josep Blat
Author: Adrian Hilton

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×