The University of Southampton
University of Southampton Institutional Repository

Jigsaw: multi-modal big data management in digital film production

Jigsaw: multi-modal big data management in digital film production
Jigsaw: multi-modal big data management in digital film production
Modern digital film production uses large quantities of data captured on-set, such as videos, digital photographs, LIDAR scans, spherical photography and many other sources to create the final film frames. The processing and management of this massive amount of heterogeneous data consumes enormous resources. We propose an integrated pipeline for 2D/3D data registration aimed at film production, based around the prototype application Jigsaw. It allows users to efficiently manage and process various data types from digital photographs to 3D point clouds. A key step in the use of multi-modal 2D/3D data for content production is the registration into a common coordinate frame (match moving). 3D geometric information is reconstructed from 2D data and registered to the reference 3D models using 3D feature matching [Kim and Hilton 2014]. We present several highly efficient and robust approaches to this problem. Additionally, we have developed and integrated a fast algorithm for incremental marginal covariance calculation [Ila et al. 2015]. This allows us to estimate and visualize the 3D reconstruction error directly on-set, where insufficient coverage or other problems can be addressed right away. We describe the fast hybrid multi-core and GPU accelerated techniques that let us run these algorithms on a laptop. Jigsaw has been used and evaluated in several major digital film productions and significantly reduced the time and work required to manage and process on-set data.
Computer graphics, Information management, Interactive computer graphics, Photography, Three dimensional computer graphics, 3D reconstruction, Content production, Data registration, Digital photographs, Feature matching, Geometric information, Heterogeneous data, Robust approaches, Big data
Pabst, S.
6c05cee4-d655-4bc5-beb5-81ebbf148d92
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Polok, L.
76afc50a-4c97-4449-8d89-15b88df23a1c
Ila, V.
699c4e52-9c1d-4bf2-b64d-dae91e5046df
Waine, T.
50857fda-9423-40e3-b32c-c778828c3cea
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Clifford, J.
96e180b6-6c69-403e-bf7d-a156f20838da
Pabst, S.
6c05cee4-d655-4bc5-beb5-81ebbf148d92
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Polok, L.
76afc50a-4c97-4449-8d89-15b88df23a1c
Ila, V.
699c4e52-9c1d-4bf2-b64d-dae91e5046df
Waine, T.
50857fda-9423-40e3-b32c-c778828c3cea
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Clifford, J.
96e180b6-6c69-403e-bf7d-a156f20838da

Pabst, S., Kim, H., Polok, L., Ila, V., Waine, T., Hilton, Adrian and Clifford, J. (2015) Jigsaw: multi-modal big data management in digital film production. Special Interest Group on Computer Graphics and Interactive Techniques Conference: SIGGRAPH'15, United States. (doi:10.1145/2787626.2792617).

Record type: Conference or Workshop Item (Paper)

Abstract

Modern digital film production uses large quantities of data captured on-set, such as videos, digital photographs, LIDAR scans, spherical photography and many other sources to create the final film frames. The processing and management of this massive amount of heterogeneous data consumes enormous resources. We propose an integrated pipeline for 2D/3D data registration aimed at film production, based around the prototype application Jigsaw. It allows users to efficiently manage and process various data types from digital photographs to 3D point clouds. A key step in the use of multi-modal 2D/3D data for content production is the registration into a common coordinate frame (match moving). 3D geometric information is reconstructed from 2D data and registered to the reference 3D models using 3D feature matching [Kim and Hilton 2014]. We present several highly efficient and robust approaches to this problem. Additionally, we have developed and integrated a fast algorithm for incremental marginal covariance calculation [Ila et al. 2015]. This allows us to estimate and visualize the 3D reconstruction error directly on-set, where insufficient coverage or other problems can be addressed right away. We describe the fast hybrid multi-core and GPU accelerated techniques that let us run these algorithms on a laptop. Jigsaw has been used and evaluated in several major digital film productions and significantly reduced the time and work required to manage and process on-set data.

Full text not available from this repository.

More information

Published date: July 2015
Venue - Dates: Special Interest Group on Computer Graphics and Interactive Techniques Conference: SIGGRAPH'15, United States, 2015-07-24
Keywords: Computer graphics, Information management, Interactive computer graphics, Photography, Three dimensional computer graphics, 3D reconstruction, Content production, Data registration, Digital photographs, Feature matching, Geometric information, Heterogeneous data, Robust approaches, Big data

Identifiers

Local EPrints ID: 440920
URI: http://eprints.soton.ac.uk/id/eprint/440920
PURE UUID: 865a64f3-e4cc-4930-824e-f0e729dc7add
ORCID for H. Kim: ORCID iD orcid.org/0000-0003-4907-0491

Catalogue record

Date deposited: 22 May 2020 16:38
Last modified: 23 May 2020 00:47

Export record

Altmetrics

Contributors

Author: S. Pabst
Author: H. Kim ORCID iD
Author: L. Polok
Author: V. Ila
Author: T. Waine
Author: Adrian Hilton
Author: J. Clifford

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×