Generating 3-dimensional images of pollen grains from their scattering patterns using deep learning

Abstract

Pollen grains can be found in a variety of shapes (e.g., spheroidal), and range in geometrical size from a few microns to ~100 microns1. Healthwise, pollen can lead to hay fever, with individuals being susceptible to different types of pollen. Additionally, pollen can be an indicator of the state of the environment2, with a pollen’s hydration level, for example, affecting its morphology3. Therefore, being able to 3D image individual pollen grains throughout the country in real-time would be invaluable for both hay fever mitigation and for environmental scientists. For this, a small footprint, low-cost sensor would likely need to be developed. Here, we use lensless sensing4 to produce scattering patterns of pollen grains, and then use a deep learning neural network5 to transform these 2D scattering patterns into 3D reconstructed images of pollen grains. We illuminated pollen with a laser to obtain scattering patterns from different pollen grains within a laser beam focus. For each pollen grain, we also obtained 3D Z-stack images using a microscope. Since only one scattering pattern was obtained for each pollen grain, but 2D image slices vary within the 3D Z-stack, for a neural network to transform the same 2D scattering pattern into different 2D images, additional information is needed. Thus, the position in the Z-axis information was encoded into the scattering pattern image via values in the green and blue channels. 3D images of pollen were then constructed using a stack of 2D images that were generated by the neural network.
1 Halbritter, H. Biotech. Histochem. 73, 137–143 (1998)
2 Ejsmond, M. J. et al. Ecosphere 2, art117 (2011)
3 Pacini, E. et al. Protoplasma 228, 73–77 (2006)
4 Grant-Jacob, J. A. et al. Environ. Res. Commun. 2, 075005 (2020)
5 Isola, P. et al. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 5967–5976 (IEEE, 2017)

More information

Identifiers

ORCID for James Grant-Jacob: orcid.org/0000-0002-4270-4247

ORCID for Matthew Praeger: orcid.org/0000-0002-5814-6155

ORCID for R.W. Eason: orcid.org/0000-0001-9704-2204

ORCID for Benjamin Mills: orcid.org/0000-0002-1784-1012

Catalogue record

Export record