Modelling of 3D placental cell features using deep learning

Abstract

Serial block-face scanning electron microscopy (SBFSEM) is a well-established technique for producing a sequential series of high-resolution images of a material, from which three-dimensional structures can be determined. This is achieved by incorporating a diamond knife microtome within a scanning electron microscope (SEM). The result is a "stack" of sequential SEM images that correspond to the 3D volume of the sample. Typically, a SBFSEM stack might include ~500-2000 SEM images, each with ~10nm spatial resolution in x and y, and separated in height by ~50nm in z.
In our work imaging human placental tissue using SBFSEM, we currently employ human labelling of features within the SEM image stacks, which allows 3D reconstruction and analysis. Whilst human labelling can be very accurate, the amount of time needed to label a single feature in a SBFSEM stack can be several months, and hence there is a practical limit to how many image stacks can be analysed. There is therefore a clear need for alternative methods for analysis that do not require human labelling.
Here, for the first time, we use a neural network to generate an SEM image when we provide the overall shape of the structures within the placental tissue. This is based on recent work in the field of generative neural networks that can be used to generate “fake” images of human faces [1] (also see the interactive website [2]). The analogy is that we are using a neural network to generate “fake” SEM images of placenta cells.
This process can be repeated for any 3D cell shape, and the results so far are shown to be statistically consistent with experimental data. This early result therefore offers the tantalising prospect of using a generative neural network for data-driven modelling of 3D cell biology.
The work presented here takes several novel steps. Firstly, we use a custom-written algorithm to label the boundary of the placenta cell in every SEM image in a stack. For each image, this boundary is used to make a “mask” that is labelled green outside the cell and red inside the cell. Secondly, we train a neural network to transform a green/red mask into a “fake” SEM image, meaning we can therefore generate an SEM image for any placental cell shape. Thirdly, we extend this capability into 3D, and use the neural network to generate a “fake” stack of SEM images for any chosen 3D placental cell shape. As a demonstration of a potential application of this approach, we label the position of generated blood cells in each of the generated SEM images in the “fake” stack, and our next aim is to visualise a predicted 3D capillary network for a placental villi.
This work builds upon our previous work [3] (presented at BIOSTEC 2020), where we demonstrated that a neural network can be used to automatically label features of placenta cell types, such as the endothelial cells and pericytes. This was achieved through training a neural network on SEM images of placenta cells that had the features of interest already labelled by a human.
[1] Karras, T. et al, “A style-based generator architecture for generative adversarial networks”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 4401-4410.
[2] https://thispersondoesnotexist.com/
[3] Mackay B.S. et al, “Deep Learning for the Automated Feature Labelling of 3-Dimensional Imaged Placenta”, BIOSTEC 2020, doi:10.1007/978-3-030-72379-8_6

More information

Identifiers

ORCID for Benjamin Mills: orcid.org/0000-0002-1784-1012

ORCID for James Grant-Jacob: orcid.org/0000-0002-4270-4247

ORCID for Benita, Scout MacKay: orcid.org/0000-0003-2050-8912

ORCID for Rohan Lewis: orcid.org/0000-0003-4044-9104

ORCID for Bram Sengers: orcid.org/0000-0001-5859-6984

Catalogue record

Export record