The University of Southampton
University of Southampton Institutional Repository

Wide-baseline multi-view video segmentation for 3D reconstruction

Wide-baseline multi-view video segmentation for 3D reconstruction
Wide-baseline multi-view video segmentation for 3D reconstruction
Obtaining a foreground silhouette across multiple views is one of the fundamental steps in 3D reconstruction. In this paper we present a novel video segmentation approach, to obtain a foreground silhouette, for scenes captured by a wide-baseline camera rig given a sparse manual interaction in a single view. The algorithm is based on trimap propagation, a framework used in video matting. Bayesian inference coupled with camera calibration information are used to spatio-temporally propagate high confidence trimap labels across the multi-view video to obtain coarse silhouettes which are later refined using a matting algorithm. Recent techniques have been developed for foreground segmentation, based on image matting, in multiple views but they are limited to narrow baseline with low foreground variation. The proposed wide-baseline silhouette propagation is robust to inter-view foreground appearance changes, shadows and similarity in foreground/background appearance. The approach has demonstrated good performance in silhouette estimation for views up to 180° baseline (opposing views). The segmentation technique has been fully integrated in a multi-view reconstruction pipeline. The results obtained demonstrate the suitability of the technique for multi-view reconstruction with wide-baseline camera set-ups and natural background.
3D reconstruction, Bayesian inference, Camera calibration, Foreground segmentation, Foreground/background, Fully integrated, High confidence, Image matting, Manual interaction, Multi-view reconstruction, Multi-views, Multiple views, Multiview video, Natural backgrounds, Segmentation, Segmentation techniques, Set-ups, Silhouette, Video matting, Video segmentation, Wide baselines, Algorithms, Bayesian networks, Cameras, Image segmentation, Inference engines, Video signal processing, Three dimensional
13-18
Sarim, M.
1242380f-3b23-491b-ab75-85daccc93f92
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Guillemaut, J.-Y.
fcaefbd8-8ba1-4c43-978c-0b5d432a3285
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Takai, T.
53894c03-6042-4bc8-86fb-692161fd22bb
Sarim, M.
1242380f-3b23-491b-ab75-85daccc93f92
Hilton, Adrian
12782a55-4c4d-4dfb-a690-62505f6665db
Guillemaut, J.-Y.
fcaefbd8-8ba1-4c43-978c-0b5d432a3285
Kim, H.
2c7c135c-f00b-4409-acb2-85b3a9e8225f
Takai, T.
53894c03-6042-4bc8-86fb-692161fd22bb

Sarim, M., Hilton, Adrian, Guillemaut, J.-Y., Kim, H. and Takai, T. (2010) Wide-baseline multi-view video segmentation for 3D reconstruction. ACM Workshop on 3D Video Processing, Italy. 30 Sep 2010. pp. 13-18 . (doi:10.1145/1877791.1877795).

Record type: Conference or Workshop Item (Paper)

Abstract

Obtaining a foreground silhouette across multiple views is one of the fundamental steps in 3D reconstruction. In this paper we present a novel video segmentation approach, to obtain a foreground silhouette, for scenes captured by a wide-baseline camera rig given a sparse manual interaction in a single view. The algorithm is based on trimap propagation, a framework used in video matting. Bayesian inference coupled with camera calibration information are used to spatio-temporally propagate high confidence trimap labels across the multi-view video to obtain coarse silhouettes which are later refined using a matting algorithm. Recent techniques have been developed for foreground segmentation, based on image matting, in multiple views but they are limited to narrow baseline with low foreground variation. The proposed wide-baseline silhouette propagation is robust to inter-view foreground appearance changes, shadows and similarity in foreground/background appearance. The approach has demonstrated good performance in silhouette estimation for views up to 180° baseline (opposing views). The segmentation technique has been fully integrated in a multi-view reconstruction pipeline. The results obtained demonstrate the suitability of the technique for multi-view reconstruction with wide-baseline camera set-ups and natural background.

Full text not available from this repository.

More information

Published date: 2010
Venue - Dates: ACM Workshop on 3D Video Processing, Italy, 2010-09-30 - 2010-09-30
Keywords: 3D reconstruction, Bayesian inference, Camera calibration, Foreground segmentation, Foreground/background, Fully integrated, High confidence, Image matting, Manual interaction, Multi-view reconstruction, Multi-views, Multiple views, Multiview video, Natural backgrounds, Segmentation, Segmentation techniques, Set-ups, Silhouette, Video matting, Video segmentation, Wide baselines, Algorithms, Bayesian networks, Cameras, Image segmentation, Inference engines, Video signal processing, Three dimensional

Identifiers

Local EPrints ID: 440578
URI: http://eprints.soton.ac.uk/id/eprint/440578
PURE UUID: 7643379d-f16c-4120-9ff4-b1006fd178e5
ORCID for H. Kim: ORCID iD orcid.org/0000-0003-4907-0491

Catalogue record

Date deposited: 07 May 2020 16:37
Last modified: 23 May 2020 00:47

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×