Using large language models for narrative analysis: a novel application of generative AI
Using large language models for narrative analysis: a novel application of generative AI
This study, a collaboration between the University of Southampton and Ipsos UK, aimed to develop and test a novel method for analysing qualitative data using generative artificial intelligence (AI). It compared large language model (LLM)-conducted analysis with human analysis of the same qualitative data, explored optimisation of LLMs for narrative analysis and evaluated their benefits and drawbacks. Using existing data, 138 short stories written by young people (aged 13-25 years) about social media, identity formation and food choices were analysed separately three times: by human researchers, and by two different LLMs (Claude and GPT-o1). The method was developed iteratively, combining Ipsos’ artificial intelligence (AI) expertise and tools with researchers’ qualitative analysis expertise. Claude and GPT-o1 each conducted a narrative analysis of all 138 stories using the same analytic steps as the human researchers. Findings between the humans and both LLMs were then compared. Both LLMs quickly and successfully conducted a narrative analysis of the stories. Their findings were comparable to those of the human researchers and were judged by the researchers to be credible and thorough. Beyond replication, the LLMs provided additional insights into the data that enhanced the human analysis. This study highlights the significant potential benefits of LLMs to the field of qualitative research and proposes that LLMs could one day be seen as valuable tools for strengthening research quality and increasing efficiency. Additionally, this study discusses ethical concerns surrounding responsible AI use in research and proposes a framework for using LLMs in qualitative analysis.
Adolescent health, Artificial intelligence, Health psychology, Large language models, Narrative analysis, Story completion
Jenner, Sarah
6de57ea6-89f7-4bed-8e76-bad5ed5957e8
Raidos, Dimitris
76a99fdc-c111-4e7c-b9e1-124bc80a6092
Anderson, Emma
374e6b5f-424b-4649-af32-0fbcb4d353fa
Fleetwood, Stella
fd5d8df1-15ed-4c8b-8d84-80e085375b6d
Ainsworth, Ben
b02d78c3-aa8b-462d-a534-31f1bf164f81
Fox, Kerry
5013960a-14c4-48e3-9bd5-9f4a4eba0773
Kreppner, Jana
6a5f447e-1cfe-4654-95b4-e6f89b0275d6
Barker, Mary
374310ad-d308-44af-b6da-515bf5d2d6d2
5 May 2025
Jenner, Sarah
6de57ea6-89f7-4bed-8e76-bad5ed5957e8
Raidos, Dimitris
76a99fdc-c111-4e7c-b9e1-124bc80a6092
Anderson, Emma
374e6b5f-424b-4649-af32-0fbcb4d353fa
Fleetwood, Stella
fd5d8df1-15ed-4c8b-8d84-80e085375b6d
Ainsworth, Ben
b02d78c3-aa8b-462d-a534-31f1bf164f81
Fox, Kerry
5013960a-14c4-48e3-9bd5-9f4a4eba0773
Kreppner, Jana
6a5f447e-1cfe-4654-95b4-e6f89b0275d6
Barker, Mary
374310ad-d308-44af-b6da-515bf5d2d6d2
Jenner, Sarah, Raidos, Dimitris, Anderson, Emma, Fleetwood, Stella, Ainsworth, Ben, Fox, Kerry, Kreppner, Jana and Barker, Mary
(2025)
Using large language models for narrative analysis: a novel application of generative AI.
Methods in Psychology, 12, [100183].
(doi:10.1016/j.metip.2025.100183).
Abstract
This study, a collaboration between the University of Southampton and Ipsos UK, aimed to develop and test a novel method for analysing qualitative data using generative artificial intelligence (AI). It compared large language model (LLM)-conducted analysis with human analysis of the same qualitative data, explored optimisation of LLMs for narrative analysis and evaluated their benefits and drawbacks. Using existing data, 138 short stories written by young people (aged 13-25 years) about social media, identity formation and food choices were analysed separately three times: by human researchers, and by two different LLMs (Claude and GPT-o1). The method was developed iteratively, combining Ipsos’ artificial intelligence (AI) expertise and tools with researchers’ qualitative analysis expertise. Claude and GPT-o1 each conducted a narrative analysis of all 138 stories using the same analytic steps as the human researchers. Findings between the humans and both LLMs were then compared. Both LLMs quickly and successfully conducted a narrative analysis of the stories. Their findings were comparable to those of the human researchers and were judged by the researchers to be credible and thorough. Beyond replication, the LLMs provided additional insights into the data that enhanced the human analysis. This study highlights the significant potential benefits of LLMs to the field of qualitative research and proposes that LLMs could one day be seen as valuable tools for strengthening research quality and increasing efficiency. Additionally, this study discusses ethical concerns surrounding responsible AI use in research and proposes a framework for using LLMs in qualitative analysis.
Text
Jenner_et_al._Accepted_manuscript
- Accepted Manuscript
Text
1-s2.0-S2590260125000098-main
- Version of Record
More information
Accepted/In Press date: 10 April 2025
e-pub ahead of print date: 11 April 2025
Published date: 5 May 2025
Keywords:
Adolescent health, Artificial intelligence, Health psychology, Large language models, Narrative analysis, Story completion
Identifiers
Local EPrints ID: 501671
URI: http://eprints.soton.ac.uk/id/eprint/501671
PURE UUID: def25a2b-fe2f-402b-bdfc-8f69921c45c7
Catalogue record
Date deposited: 05 Jun 2025 16:50
Last modified: 11 Sep 2025 03:13
Export record
Altmetrics
Contributors
Author:
Sarah Jenner
Author:
Dimitris Raidos
Author:
Emma Anderson
Author:
Stella Fleetwood
Author:
Ben Ainsworth
Author:
Kerry Fox
Author:
Jana Kreppner
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics