The University of Southampton
University of Southampton Institutional Repository

Markerless Real-Time Garment Retexturing From Monocular 3D Reconstruction

Markerless Real-Time Garment Retexturing From Monocular 3D Reconstruction
Markerless Real-Time Garment Retexturing From Monocular 3D Reconstruction
We present a fusion of augmented reality (AR) and virtual try on (VTO) that incorporates sparse 3D point recovery by exploiting distance constraints based on 2D point correspondences between a deformed texture in monocular video and a reference texture which is derived from the start of the sequence by face detection aided segmentation. A hierarchical and multi-resolution surface reconstruction approach is proposed, employing thin-plate splines, cloth modeling, and patch tessellation. Our method attempts to accurately recover a rectangular surface from a deformed arbitrarily shaped texture. We also propose a hue-based method for segmenting garment cloth and printed texture. The methods are demonstrated in an AR framework for real-time visualization of a virtual garment worn in a real scene. Real-time AR cloth retexturing from monocular vision is a state-of-the-art field. Previous work lacks realism and accuracy, only recovering the 2D cloth layout and lacks robustness, requiring a special T-shirt color and simple texture along with lab hardware. Our approach alleviates these limitations. We design a practical approach which considers a typical consumer environment with a mid-range PC and webcam. Our results are convincing and photorealistic with robustness to arbitrary T-shirts, subjects, and backgrounds. Future work will focus on extending our global model and quantitative analysis.
Cushen, George
52f73d41-3ae0-4c11-a50a-86e782c03745
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12
Cushen, George
52f73d41-3ae0-4c11-a50a-86e782c03745
Nixon, Mark
2b5b9804-5a81-462a-82e6-92ee5fa74e12

Cushen, George and Nixon, Mark (2011) Markerless Real-Time Garment Retexturing From Monocular 3D Reconstruction. IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia.

Record type: Conference or Workshop Item (Paper)

Abstract

We present a fusion of augmented reality (AR) and virtual try on (VTO) that incorporates sparse 3D point recovery by exploiting distance constraints based on 2D point correspondences between a deformed texture in monocular video and a reference texture which is derived from the start of the sequence by face detection aided segmentation. A hierarchical and multi-resolution surface reconstruction approach is proposed, employing thin-plate splines, cloth modeling, and patch tessellation. Our method attempts to accurately recover a rectangular surface from a deformed arbitrarily shaped texture. We also propose a hue-based method for segmenting garment cloth and printed texture. The methods are demonstrated in an AR framework for real-time visualization of a virtual garment worn in a real scene. Real-time AR cloth retexturing from monocular vision is a state-of-the-art field. Previous work lacks realism and accuracy, only recovering the 2D cloth layout and lacks robustness, requiring a special T-shirt color and simple texture along with lab hardware. Our approach alleviates these limitations. We design a practical approach which considers a typical consumer environment with a mid-range PC and webcam. Our results are convincing and photorealistic with robustness to arbitrary T-shirts, subjects, and backgrounds. Future work will focus on extending our global model and quantitative analysis.

Text
ICSIPA2011-Cushen-IEEEcertified.pdf - Version of Record
Download (2MB)

More information

Published date: November 2011
Venue - Dates: IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, 2011-11-01
Organisations: Vision, Learning and Control

Identifiers

Local EPrints ID: 273103
URI: http://eprints.soton.ac.uk/id/eprint/273103
PURE UUID: 239584cf-7c4a-474b-9f96-722bb27de9a5
ORCID for Mark Nixon: ORCID iD orcid.org/0000-0002-9174-5934

Catalogue record

Date deposited: 15 Jan 2012 12:20
Last modified: 15 Mar 2024 02:35

Export record

Contributors

Author: George Cushen
Author: Mark Nixon ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×