The University of Southampton
University of Southampton Institutional Repository

Exploring the role of ChatGPT in rapid intervention text development

Exploring the role of ChatGPT in rapid intervention text development
Exploring the role of ChatGPT in rapid intervention text development
Background: there have been successful applications of AI to answering health-related questions, which suggests a potential role for AI in assisting with development of intervention text. This paper explores how ChatGPT might be used to support the rapid development of intervention text.

Methods: three case studies are presented. In the first case study, ChatGPT (using GPT-4) was asked to generate sleep advice for adolescents. In case study two, ChatGPT (using GPT-3) was asked to optimise advice for people experiencing homelessness on staying hydrated in extreme heat. Case study three asked ChatGPT using GPT-3 and GPT-4 to optimise an information sheet for participation in a study developing an intervention for maternal blood pressure. Outputs were evaluated by the researchers who developed the text, and in case studies two and three were shown to public and patient contributors for feedback.

Results: ChatGPT was able to generate informative advice about sleep in case study one and was able to accurately summarise information in case studies two and three. In all three cases, elements or aspects were omitted that were included in the researcher-generated text that was based on behaviour change theory, evidence and input from public and patient contributors. However, in case study three, feedback from public contributors suggested ChatGPTs outputs were preferred to the original, although the outputs omitted information and were not at the requested accessible reading level.

Conclusions: ChatGPT was able to accurately generate and summarise health information. However, this information typically excluded core behaviour change techniques and was sometimes inappropriate for the target users. There is likely to be a valuable role for generative AI in the intervention development process, but this will need to be combined with detailed scrutiny and input from researchers and public contributors.
ChatGPT, intervention development, AI, behaviour change
2046-1402
Bowers, Hannah
c81d418d-3cd7-4da5-bd09-0eee862bd49f
Ochieng, Cynthia
a379f0df-49ab-417a-bb24-2d868ef6b123
Bennett, Sarah E.
409e3275-0202-46b1-9535-086e42b2f76c
Denford, Sarah
8970b5a7-8cad-4356-ad0e-88297b67db37
Johnston, Milly
497110b3-d467-4ce7-9c75-6cfbefd8a05a
Yardley, Lucy
64be42c4-511d-484d-abaa-f8813452a22e
Bowers, Hannah
c81d418d-3cd7-4da5-bd09-0eee862bd49f
Ochieng, Cynthia
a379f0df-49ab-417a-bb24-2d868ef6b123
Bennett, Sarah E.
409e3275-0202-46b1-9535-086e42b2f76c
Denford, Sarah
8970b5a7-8cad-4356-ad0e-88297b67db37
Johnston, Milly
497110b3-d467-4ce7-9c75-6cfbefd8a05a
Yardley, Lucy
64be42c4-511d-484d-abaa-f8813452a22e

Bowers, Hannah, Ochieng, Cynthia, Bennett, Sarah E., Denford, Sarah, Johnston, Milly and Yardley, Lucy (2023) Exploring the role of ChatGPT in rapid intervention text development. F1000 Research, 12 (1395). (doi:10.12688/f1000research.140708.1).

Record type: Article

Abstract

Background: there have been successful applications of AI to answering health-related questions, which suggests a potential role for AI in assisting with development of intervention text. This paper explores how ChatGPT might be used to support the rapid development of intervention text.

Methods: three case studies are presented. In the first case study, ChatGPT (using GPT-4) was asked to generate sleep advice for adolescents. In case study two, ChatGPT (using GPT-3) was asked to optimise advice for people experiencing homelessness on staying hydrated in extreme heat. Case study three asked ChatGPT using GPT-3 and GPT-4 to optimise an information sheet for participation in a study developing an intervention for maternal blood pressure. Outputs were evaluated by the researchers who developed the text, and in case studies two and three were shown to public and patient contributors for feedback.

Results: ChatGPT was able to generate informative advice about sleep in case study one and was able to accurately summarise information in case studies two and three. In all three cases, elements or aspects were omitted that were included in the researcher-generated text that was based on behaviour change theory, evidence and input from public and patient contributors. However, in case study three, feedback from public contributors suggested ChatGPTs outputs were preferred to the original, although the outputs omitted information and were not at the requested accessible reading level.

Conclusions: ChatGPT was able to accurately generate and summarise health information. However, this information typically excluded core behaviour change techniques and was sometimes inappropriate for the target users. There is likely to be a valuable role for generative AI in the intervention development process, but this will need to be combined with detailed scrutiny and input from researchers and public contributors.

Text
4472cd83-b10e-4cf2-b4fc-9b53ecbc3c40_140708_-_hannah_bowers - Author's Original
Available under License Creative Commons Attribution.
Download (195kB)

More information

e-pub ahead of print date: 23 October 2023
Keywords: ChatGPT, intervention development, AI, behaviour change

Identifiers

Local EPrints ID: 483308
URI: http://eprints.soton.ac.uk/id/eprint/483308
ISSN: 2046-1402
PURE UUID: b9940398-44c9-4197-9fd6-cf9c49fadff0
ORCID for Hannah Bowers: ORCID iD orcid.org/0000-0002-1996-6652
ORCID for Lucy Yardley: ORCID iD orcid.org/0000-0002-3853-883X

Catalogue record

Date deposited: 27 Oct 2023 16:47
Last modified: 12 Jun 2024 01:53

Export record

Altmetrics

Contributors

Author: Hannah Bowers ORCID iD
Author: Cynthia Ochieng
Author: Sarah E. Bennett
Author: Sarah Denford
Author: Milly Johnston
Author: Lucy Yardley ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×