Exploring the role of ChatGPT in rapid intervention text development
Exploring the role of ChatGPT in rapid intervention text development
Background: there have been successful applications of AI to answering health-related questions, which suggests a potential role for AI in assisting with development of intervention text. This paper explores how ChatGPT might be used to support the rapid development of intervention text.
Methods: three case studies are presented. In the first case study, ChatGPT (using GPT-4) was asked to generate sleep advice for adolescents. In case study two, ChatGPT (using GPT-3) was asked to optimise advice for people experiencing homelessness on staying hydrated in extreme heat. Case study three asked ChatGPT using GPT-3 and GPT-4 to optimise an information sheet for participation in a study developing an intervention for maternal blood pressure. Outputs were evaluated by the researchers who developed the text, and in case studies two and three were shown to public and patient contributors for feedback.
Results: ChatGPT was able to generate informative advice about sleep in case study one and was able to accurately summarise information in case studies two and three. In all three cases, elements or aspects were omitted that were included in the researcher-generated text that was based on behaviour change theory, evidence and input from public and patient contributors. However, in case study three, feedback from public contributors suggested ChatGPTs outputs were preferred to the original, although the outputs omitted information and were not at the requested accessible reading level.
Conclusions: ChatGPT was able to accurately generate and summarise health information. However, this information typically excluded core behaviour change techniques and was sometimes inappropriate for the target users. There is likely to be a valuable role for generative AI in the intervention development process, but this will need to be combined with detailed scrutiny and input from researchers and public contributors.
ChatGPT, intervention development, AI, behaviour change
Bowers, Hannah
c81d418d-3cd7-4da5-bd09-0eee862bd49f
Ochieng, Cynthia
a379f0df-49ab-417a-bb24-2d868ef6b123
Bennett, Sarah E.
409e3275-0202-46b1-9535-086e42b2f76c
Denford, Sarah
8970b5a7-8cad-4356-ad0e-88297b67db37
Johnston, Milly
497110b3-d467-4ce7-9c75-6cfbefd8a05a
Yardley, Lucy
64be42c4-511d-484d-abaa-f8813452a22e
Bowers, Hannah
c81d418d-3cd7-4da5-bd09-0eee862bd49f
Ochieng, Cynthia
a379f0df-49ab-417a-bb24-2d868ef6b123
Bennett, Sarah E.
409e3275-0202-46b1-9535-086e42b2f76c
Denford, Sarah
8970b5a7-8cad-4356-ad0e-88297b67db37
Johnston, Milly
497110b3-d467-4ce7-9c75-6cfbefd8a05a
Yardley, Lucy
64be42c4-511d-484d-abaa-f8813452a22e
Bowers, Hannah, Ochieng, Cynthia, Bennett, Sarah E., Denford, Sarah, Johnston, Milly and Yardley, Lucy
(2023)
Exploring the role of ChatGPT in rapid intervention text development.
F1000 Research, 12 (1395).
(doi:10.12688/f1000research.140708.1).
Abstract
Background: there have been successful applications of AI to answering health-related questions, which suggests a potential role for AI in assisting with development of intervention text. This paper explores how ChatGPT might be used to support the rapid development of intervention text.
Methods: three case studies are presented. In the first case study, ChatGPT (using GPT-4) was asked to generate sleep advice for adolescents. In case study two, ChatGPT (using GPT-3) was asked to optimise advice for people experiencing homelessness on staying hydrated in extreme heat. Case study three asked ChatGPT using GPT-3 and GPT-4 to optimise an information sheet for participation in a study developing an intervention for maternal blood pressure. Outputs were evaluated by the researchers who developed the text, and in case studies two and three were shown to public and patient contributors for feedback.
Results: ChatGPT was able to generate informative advice about sleep in case study one and was able to accurately summarise information in case studies two and three. In all three cases, elements or aspects were omitted that were included in the researcher-generated text that was based on behaviour change theory, evidence and input from public and patient contributors. However, in case study three, feedback from public contributors suggested ChatGPTs outputs were preferred to the original, although the outputs omitted information and were not at the requested accessible reading level.
Conclusions: ChatGPT was able to accurately generate and summarise health information. However, this information typically excluded core behaviour change techniques and was sometimes inappropriate for the target users. There is likely to be a valuable role for generative AI in the intervention development process, but this will need to be combined with detailed scrutiny and input from researchers and public contributors.
Text
4472cd83-b10e-4cf2-b4fc-9b53ecbc3c40_140708_-_hannah_bowers
- Author's Original
More information
e-pub ahead of print date: 23 October 2023
Keywords:
ChatGPT, intervention development, AI, behaviour change
Identifiers
Local EPrints ID: 483308
URI: http://eprints.soton.ac.uk/id/eprint/483308
ISSN: 2046-1402
PURE UUID: b9940398-44c9-4197-9fd6-cf9c49fadff0
Catalogue record
Date deposited: 27 Oct 2023 16:47
Last modified: 18 Mar 2024 02:50
Export record
Altmetrics
Contributors
Author:
Cynthia Ochieng
Author:
Sarah E. Bennett
Author:
Sarah Denford
Author:
Milly Johnston
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics
