The University of Southampton
University of Southampton Institutional Repository

A foundation model for generalizable disease detection from retinal images

A foundation model for generalizable disease detection from retinal images
A foundation model for generalizable disease detection from retinal images

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Humans, Artificial Intelligence, Eye Diseases/complications, Heart Failure/complications, Myocardial Infarction/complications, Retina/diagnostic imaging, Supervised Machine Learning
0028-0836
156-163
Zhou, Yukun
df670730-c245-42ee-b7f9-cafa0b99432a
Chia, Mark A.
90d497ba-4034-4bb2-9cc5-836fe361f1d1
Wagner, Siegfried K.
726fcef3-87d7-4416-84ed-bd8d309dc3fc
Ayhan, Murat S.
58b34820-4a70-4078-8e7f-b28671f4ff1f
Williamson, Dominic J.
3a034212-b48c-4663-befc-ca9627baaa68
Struyven, Robbert R.
692e8884-f0c8-4f40-bbe2-303da4bdb5cb
Liu, Timing
8d606f8d-af9c-4438-bf64-b72b12c0ea63
Xu, Moucheng
91eaa8d2-beac-48f1-b475-3f92f398d887
Lozano, Mateo G.
57280bb1-d96e-4da6-ab3a-821ac3a7d952
Woodward-Court, Peter
923230ea-5771-4f71-baa3-513c7cffe7a6
Kihara, Yuka
4b0d3e77-6b5e-4303-b6a6-1f3595c13ad5
Allen, Naomi
1d4110d6-f1c5-43c8-9113-549a17340a5a
Gallacher, John E.J.
e05d736b-eb13-4f6b-b8ca-6f417d60a6a7
Littlejohns, Thomas
ea8614d6-9dcd-4a9f-adfc-16cdc7942ab3
Aslam, Tariq
eb99b291-2d05-4ba5-9a3e-8bf8d232adcb
Bishop, Paul
78947692-320c-41b7-a30d-a4aee5835e8d
Black, Graeme
23862d1e-19b9-4ec9-b8ca-b09e4dc42d57
Sergouniotis, Panagiotis
d9e3116d-beff-4259-bbb3-e5ef7539b725
Atan, Denize
ee27dcf4-671c-4c31-b521-99f9db4dee21
Dick, Andrew D.
3663310f-576f-459a-aa3e-7a249ab33e28
Williams, Cathy
0c609b70-3206-4414-bd13-a045f664194c
Barman, Sarah
d5a5203f-d607-4091-9341-32bc5e2ce29a
Barrett, Jenny H.
a363c0de-1578-4768-bc91-77c41b1443fa
Mackie, Sarah
52fb2df7-467c-49ca-9890-aebc3455ab8f
Braithwaite, Tasanee
89e1e305-7312-466a-bdb9-c1202474fcf1
Carare, Roxana O.
0478c197-b0c1-4206-acae-54e88c8f21fa
Ennis, Sarah
7b57f188-9d91-4beb-b217-09856146f1e9
Gibson, Jane
855033a6-38f3-4853-8f60-d7d4561226ae
Lotery, Andrew J.
5ecc2d2d-d0b4-468f-ad2c-df7156f8e514
Self, Jay
0f6efc58-ae24-4667-b8d6-6fafa849e389
Chakravarthy, Usha
2c06cdaf-94c0-409a-8eff-2f624c120a5e
Hogg, Ruth E.
ce459eb4-c6d9-40e4-8539-ebafab779a2a
Paterson, Euan
e009cf42-41ce-4668-a247-51ae92d69ecb
Woodside, Jayne
8ed81a6d-4e4f-4ee4-ac15-7b9b6b18d528
Peto, Tunde
e5511bbd-2ef8-4465-a2b3-46c8cc3ce63e
Mckay, Gareth
1fd4e9c2-f5b6-4491-a735-67ef35aa1c33
Mcguinness, Bernadette
a4d6fd5f-41c5-4444-a692-931729948f6a
Foster, Paul J.
d58cb374-4307-4319-8e9f-f0ac6092e347
Balaskas, Konstantinos
6a559a09-c161-435a-aa2c-b9f0f3e1521a
Khawaja, Anthony P.
ac15cbf1-b21c-4760-80d9-d8ff7ce51e03
Pontikos, Nikolas
5a964059-ef8e-4b83-8a5a-6d832839e0f2
Rahi, Jugnoo S.
243a4ce6-5961-4215-bf07-91d56aefb32d
Lascaratos, Gerassimos
c96af728-2bc1-419c-9512-d18dcec37ecf
Patel, Praveen J.
91baf7b2-de1c-40aa-b3f4-7cc7007220f2
Chan, Michelle
c57c833d-6a9d-4a60-a06d-000a27e0cfab
Chua, Sharon Y.L.
806116ec-8f79-41cb-b52b-0ad40dcaf587
Day, Alexander
a0fcd347-ce82-4e58-ac80-c05f93f28fd9
Desai, Parul
2cec4ddd-d533-4188-933b-2b0b24ea0807
Morgan, James E.
391ed7a6-fadb-4127-86e6-404f80596b08
Stratton, Irene
772f25b9-23c0-4240-a3f6-1e76b03b172f
UK Biobank Eye and Vision Consortium
Zhou, Yukun
df670730-c245-42ee-b7f9-cafa0b99432a
Chia, Mark A.
90d497ba-4034-4bb2-9cc5-836fe361f1d1
Wagner, Siegfried K.
726fcef3-87d7-4416-84ed-bd8d309dc3fc
Ayhan, Murat S.
58b34820-4a70-4078-8e7f-b28671f4ff1f
Williamson, Dominic J.
3a034212-b48c-4663-befc-ca9627baaa68
Struyven, Robbert R.
692e8884-f0c8-4f40-bbe2-303da4bdb5cb
Liu, Timing
8d606f8d-af9c-4438-bf64-b72b12c0ea63
Xu, Moucheng
91eaa8d2-beac-48f1-b475-3f92f398d887
Lozano, Mateo G.
57280bb1-d96e-4da6-ab3a-821ac3a7d952
Woodward-Court, Peter
923230ea-5771-4f71-baa3-513c7cffe7a6
Kihara, Yuka
4b0d3e77-6b5e-4303-b6a6-1f3595c13ad5
Allen, Naomi
1d4110d6-f1c5-43c8-9113-549a17340a5a
Gallacher, John E.J.
e05d736b-eb13-4f6b-b8ca-6f417d60a6a7
Littlejohns, Thomas
ea8614d6-9dcd-4a9f-adfc-16cdc7942ab3
Aslam, Tariq
eb99b291-2d05-4ba5-9a3e-8bf8d232adcb
Bishop, Paul
78947692-320c-41b7-a30d-a4aee5835e8d
Black, Graeme
23862d1e-19b9-4ec9-b8ca-b09e4dc42d57
Sergouniotis, Panagiotis
d9e3116d-beff-4259-bbb3-e5ef7539b725
Atan, Denize
ee27dcf4-671c-4c31-b521-99f9db4dee21
Dick, Andrew D.
3663310f-576f-459a-aa3e-7a249ab33e28
Williams, Cathy
0c609b70-3206-4414-bd13-a045f664194c
Barman, Sarah
d5a5203f-d607-4091-9341-32bc5e2ce29a
Barrett, Jenny H.
a363c0de-1578-4768-bc91-77c41b1443fa
Mackie, Sarah
52fb2df7-467c-49ca-9890-aebc3455ab8f
Braithwaite, Tasanee
89e1e305-7312-466a-bdb9-c1202474fcf1
Carare, Roxana O.
0478c197-b0c1-4206-acae-54e88c8f21fa
Ennis, Sarah
7b57f188-9d91-4beb-b217-09856146f1e9
Gibson, Jane
855033a6-38f3-4853-8f60-d7d4561226ae
Lotery, Andrew J.
5ecc2d2d-d0b4-468f-ad2c-df7156f8e514
Self, Jay
0f6efc58-ae24-4667-b8d6-6fafa849e389
Chakravarthy, Usha
2c06cdaf-94c0-409a-8eff-2f624c120a5e
Hogg, Ruth E.
ce459eb4-c6d9-40e4-8539-ebafab779a2a
Paterson, Euan
e009cf42-41ce-4668-a247-51ae92d69ecb
Woodside, Jayne
8ed81a6d-4e4f-4ee4-ac15-7b9b6b18d528
Peto, Tunde
e5511bbd-2ef8-4465-a2b3-46c8cc3ce63e
Mckay, Gareth
1fd4e9c2-f5b6-4491-a735-67ef35aa1c33
Mcguinness, Bernadette
a4d6fd5f-41c5-4444-a692-931729948f6a
Foster, Paul J.
d58cb374-4307-4319-8e9f-f0ac6092e347
Balaskas, Konstantinos
6a559a09-c161-435a-aa2c-b9f0f3e1521a
Khawaja, Anthony P.
ac15cbf1-b21c-4760-80d9-d8ff7ce51e03
Pontikos, Nikolas
5a964059-ef8e-4b83-8a5a-6d832839e0f2
Rahi, Jugnoo S.
243a4ce6-5961-4215-bf07-91d56aefb32d
Lascaratos, Gerassimos
c96af728-2bc1-419c-9512-d18dcec37ecf
Patel, Praveen J.
91baf7b2-de1c-40aa-b3f4-7cc7007220f2
Chan, Michelle
c57c833d-6a9d-4a60-a06d-000a27e0cfab
Chua, Sharon Y.L.
806116ec-8f79-41cb-b52b-0ad40dcaf587
Day, Alexander
a0fcd347-ce82-4e58-ac80-c05f93f28fd9
Desai, Parul
2cec4ddd-d533-4188-933b-2b0b24ea0807
Morgan, James E.
391ed7a6-fadb-4127-86e6-404f80596b08
Stratton, Irene
772f25b9-23c0-4240-a3f6-1e76b03b172f

Zhou, Yukun, Chia, Mark A. and Wagner, Siegfried K. , UK Biobank Eye and Vision Consortium (2023) A foundation model for generalizable disease detection from retinal images. Nature, 622 (7981), 156-163. (doi:10.1038/s41586-023-06555-x).

Record type: Article

Abstract

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Text
s41586-023-06555-x - Version of Record
Available under License Creative Commons Attribution.
Download (22MB)

More information

Accepted/In Press date: 18 August 2023
e-pub ahead of print date: 13 September 2023
Published date: 5 October 2023
Additional Information: Funding Information: We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.
Keywords: Humans, Artificial Intelligence, Eye Diseases/complications, Heart Failure/complications, Myocardial Infarction/complications, Retina/diagnostic imaging, Supervised Machine Learning

Identifiers

Local EPrints ID: 484186
URI: http://eprints.soton.ac.uk/id/eprint/484186
ISSN: 0028-0836
PURE UUID: 89ccc4e8-7cab-42b8-bf3b-deebc40d6a68
ORCID for Roxana O. Carare: ORCID iD orcid.org/0000-0001-6458-3776
ORCID for Sarah Ennis: ORCID iD orcid.org/0000-0003-2648-0869
ORCID for Jane Gibson: ORCID iD orcid.org/0000-0002-0973-8285
ORCID for Andrew J. Lotery: ORCID iD orcid.org/0000-0001-5541-4305
ORCID for Jay Self: ORCID iD orcid.org/0000-0002-1030-9963
ORCID for Irene Stratton: ORCID iD orcid.org/0000-0003-1172-7865

Catalogue record

Date deposited: 12 Nov 2023 07:05
Last modified: 29 May 2024 02:01

Export record

Altmetrics

Contributors

Author: Yukun Zhou
Author: Mark A. Chia
Author: Siegfried K. Wagner
Author: Murat S. Ayhan
Author: Dominic J. Williamson
Author: Robbert R. Struyven
Author: Timing Liu
Author: Moucheng Xu
Author: Mateo G. Lozano
Author: Peter Woodward-Court
Author: Yuka Kihara
Author: Naomi Allen
Author: John E.J. Gallacher
Author: Thomas Littlejohns
Author: Tariq Aslam
Author: Paul Bishop
Author: Graeme Black
Author: Panagiotis Sergouniotis
Author: Denize Atan
Author: Andrew D. Dick
Author: Cathy Williams
Author: Sarah Barman
Author: Jenny H. Barrett
Author: Sarah Mackie
Author: Tasanee Braithwaite
Author: Sarah Ennis ORCID iD
Author: Jane Gibson ORCID iD
Author: Jay Self ORCID iD
Author: Usha Chakravarthy
Author: Ruth E. Hogg
Author: Euan Paterson
Author: Jayne Woodside
Author: Tunde Peto
Author: Gareth Mckay
Author: Bernadette Mcguinness
Author: Paul J. Foster
Author: Konstantinos Balaskas
Author: Anthony P. Khawaja
Author: Nikolas Pontikos
Author: Jugnoo S. Rahi
Author: Gerassimos Lascaratos
Author: Praveen J. Patel
Author: Michelle Chan
Author: Sharon Y.L. Chua
Author: Alexander Day
Author: Parul Desai
Author: James E. Morgan
Author: Irene Stratton ORCID iD
Corporate Author: UK Biobank Eye and Vision Consortium

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×