Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds

Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a variety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac motion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications involving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art techniques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spatial resolution is ∼1.8×1.8×10mm³). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysis.

Cardiac mesh reconstruction, Cardiac surface reconstruction, Contours to mesh reconstruction, Deep learning, Graph convolutional network

10.1016/j.media.2021.102228

1361-8415

Chen, Xiang

2bddccea-9285-45d3-ba1f-046aa5c32d09

Ravikumar, Nishant

e31cd3f7-b112-4078-b5fc-0cfc005a061a

Xia, Yan

e6c0b611-427b-4871-86f1-406efee13bb5

Attar, Rahman

f5efd538-042a-4647-9d46-1370d3049b72

Diaz-Pinto, Andres

a4243590-1301-4860-bcd2-adff059abc07

Piechnik, Stefan K.

7de3d548-ca5a-40cb-a52b-c53d2dd2278a

Neubauer, Stefan

c8a34156-a4ed-4dfe-97cb-4f47627d927d

Petersen, Steffen E.

04f2ce88-790d-48dc-baac-cbe0946dd928

Frangi, Alejandro F.

35127be7-3586-4fff-a4a2-bb79cc228141

et al.

December 2021

Chen, Xiang

2bddccea-9285-45d3-ba1f-046aa5c32d09

Ravikumar, Nishant

e31cd3f7-b112-4078-b5fc-0cfc005a061a

Xia, Yan

e6c0b611-427b-4871-86f1-406efee13bb5

Attar, Rahman

f5efd538-042a-4647-9d46-1370d3049b72

Diaz-Pinto, Andres

a4243590-1301-4860-bcd2-adff059abc07

Piechnik, Stefan K.

7de3d548-ca5a-40cb-a52b-c53d2dd2278a

Neubauer, Stefan

c8a34156-a4ed-4dfe-97cb-4f47627d927d

Petersen, Steffen E.

04f2ce88-790d-48dc-baac-cbe0946dd928

Frangi, Alejandro F.

35127be7-3586-4fff-a4a2-bb79cc228141

Chen, Xiang, Ravikumar, Nishant, Xia, Yan, Attar, Rahman and Frangi, Alejandro F. , et al. (2021) Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds. Medical Image Analysis, 74 (12), [102228]. (doi:10.1016/j.media.2021.102228).

Record type: Article

Abstract

Text

Shape_registration_with_learned_deformations - Version of Record

Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB)

More information

Published date: December 2021

Additional Information: Funding Information: The Royal Academy of Engineering supports AFF through a Chair in Emerging Technologies (CiET1819∖19) and the MedIAN Network (EP/N026993/1) funded by the Engineering and Physical Sciences Research Council (EPSRC). This research was conducted using the UKBB resource under access application 11350. SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans (www.bhf.org.uk; PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Cardiovascular Biomedical Research Centre at Barts. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence. Funding Information: The Royal Academy of Engineering supports AFF through a Chair in Emerging Technologies (CiET181919) and the MedIAN Network (EP/N026993/1) funded by the Engineering and Physical Sciences Research Council (EPSRC) . This research was conducted using the UKBB resource under access application 11350. SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans ( www.bhf.org.uk ; PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Cardiovascular Biomedical Research Centre at Barts. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence. Publisher Copyright: © 2021 The Author(s)

Keywords: Cardiac mesh reconstruction, Cardiac surface reconstruction, Contours to mesh reconstruction, Deep learning, Graph convolutional network

Learn more about Vision, Learning and Control research