The University of Southampton
University of Southampton Institutional Repository

Temporally coherent general dynamic scene reconstruction

Temporally coherent general dynamic scene reconstruction
Temporally coherent general dynamic scene reconstruction

Existing techniques for dynamic scene reconstruction from multiple wide-baseline cameras primarily focus on reconstruction in controlled environments, with fixed calibrated cameras and strong prior constraints. This paper introduces a general approach to obtain a 4D representation of complex dynamic scenes from multi-view wide-baseline static or moving cameras without prior knowledge of the scene structure, appearance, or illumination. Contributions of the work are: an automatic method for initial coarse reconstruction to initialize joint estimation; sparse-to-dense temporal correspondence integrated with joint multi-view segmentation and reconstruction to introduce temporal coherence; and a general robust approach for joint segmentation refinement and dense reconstruction of dynamic scenes by introducing shape constraint. Comparison with state-of-the-art approaches on a variety of complex indoor and outdoor scenes, demonstrates improved accuracy in both multi-view segmentation and dense reconstruction. This paper demonstrates unsupervised reconstruction of complete temporally coherent 4D scene models with improved non-rigid object segmentation and shape reconstruction and its application to various applications such as free-view rendering and virtual reality.

3D, Dynamic 4D reconstruction, Dynamic scenes, Reconstruction, Segmentation, Temporal coherence
Mustafa, Armin
29037014-ab45-4368-81e3-6b698e9bbbd0
Volino, Marco
6c0974af-5978-457c-bb6b-319f4da9a96b
Kim, Hansung
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Guillemaut, Jean-Yves
fcaefbd8-8ba1-4c43-978c-0b5d432a3285
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Mustafa, Armin
29037014-ab45-4368-81e3-6b698e9bbbd0
Volino, Marco
6c0974af-5978-457c-bb6b-319f4da9a96b
Kim, Hansung
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Guillemaut, Jean-Yves
fcaefbd8-8ba1-4c43-978c-0b5d432a3285
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db

Mustafa, Armin, Volino, Marco, Kim, Hansung, Guillemaut, Jean-Yves and Hilton, Adrian (2020) Temporally coherent general dynamic scene reconstruction. International Journal of Computer Vision. (doi:10.1007/s11263-020-01367-2).

Record type: Article

Abstract

Existing techniques for dynamic scene reconstruction from multiple wide-baseline cameras primarily focus on reconstruction in controlled environments, with fixed calibrated cameras and strong prior constraints. This paper introduces a general approach to obtain a 4D representation of complex dynamic scenes from multi-view wide-baseline static or moving cameras without prior knowledge of the scene structure, appearance, or illumination. Contributions of the work are: an automatic method for initial coarse reconstruction to initialize joint estimation; sparse-to-dense temporal correspondence integrated with joint multi-view segmentation and reconstruction to introduce temporal coherence; and a general robust approach for joint segmentation refinement and dense reconstruction of dynamic scenes by introducing shape constraint. Comparison with state-of-the-art approaches on a variety of complex indoor and outdoor scenes, demonstrates improved accuracy in both multi-view segmentation and dense reconstruction. This paper demonstrates unsupervised reconstruction of complete temporally coherent 4D scene models with improved non-rigid object segmentation and shape reconstruction and its application to various applications such as free-view rendering and virtual reality.

Text
Mustafa 2020 Article Temporally Coherent General Dynamic - Version of Record
Available under License Creative Commons Attribution.
Download (7MB)

More information

Accepted/In Press date: 4 August 2020
e-pub ahead of print date: 18 August 2020
Keywords: 3D, Dynamic 4D reconstruction, Dynamic scenes, Reconstruction, Segmentation, Temporal coherence

Identifiers

Local EPrints ID: 445057
URI: http://eprints.soton.ac.uk/id/eprint/445057
PURE UUID: c2341a4a-092a-46d1-b4e1-e56979ae958c
ORCID for Hansung Kim: ORCID iD orcid.org/0000-0003-4907-0491

Catalogue record

Date deposited: 18 Nov 2020 17:33
Last modified: 17 Mar 2024 04:01

Export record

Altmetrics

Contributors

Author: Armin Mustafa
Author: Marco Volino
Author: Hansung Kim ORCID iD
Author: Jean-Yves Guillemaut
Author: Adrian Hilton

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×