The University of Southampton
University of Southampton Institutional Repository

Responsible use of language AI in healthcare: towards evaluating and regulating large language models for multilingual communication in the NHS

Responsible use of language AI in healthcare: towards evaluating and regulating large language models for multilingual communication in the NHS
Responsible use of language AI in healthcare: towards evaluating and regulating large language models for multilingual communication in the NHS
Shortages of professional interpreters in healthcare can create communication risks for patients with limited English proficiency, while also leading practitioners to rely on ad hoc alternatives, including family members, untrained interpreters, or increasingly, machine translation and large language model (LLM)-based tools. Yet despite growing informal use of such technologies, there remains a significant gap between technical research on language AI, sociotechnical understandings of multilingual communication, and the policy and regulatory frameworks needed to govern safe deployment in public-sector settings. This work addresses that gap through a sociotechnical and empirically informed examination of responsible language AI use in healthcare contexts, with the intention of studying this in the National Health Service (NHS) in the future.

Our work explores the ethical, technical and regulatory challenges associated with using machine translation and LLMs to support multilingual communication when professional interpreters are unavailable. It focuses particularly on questions of translation quality, clinical risk, accountability, and appropriateness of use across different healthcare settings, including general practice, accident and emergency, and pharmacy interactions. Rather than treating machine translation as a substitute for professional interpreters, the work investigates conditions under which its use may, or may not, be justified as a risk-managed communication support tool.

We report findings from our pilot study, which used 10 healthcare case studies designed by medical professionals to generate simulated multilingual conversations with Google Translate and OpenAI GPT-5 for English–Ukrainian and English–Russian language pairs. Initial results suggest high translation quality and accuracy for these simulated conversational scenarios, differing from findings in earlier studies focused on medical text translation. We discuss factors that may influence these outcomes, including language pair characteristics, domain specificity, conversational structure, and model choice.

Building on these findings, the project will propose a framework for responsible use of language AI in healthcare that integrates technical evaluation, including BLEU and assessment of clinically significant errors, with broader governance considerations, including accountability pathways, trust, and institutional variation. In doing so, the paper contributes to debates on ethical aspects of AI in translation and interpreting, the role of LLMs in multilingual communication, and the responsible deployment of language AI in the public sector. It argues that evidence-based regulation, rather than unregulated adoption or blanket prohibition, is necessary to support safer and more equitable multilingual healthcare communication.
Antiufieieva, Yuliia
89075d6f-3114-418a-95e4-b63d8a7e1f8a
Wilde, Adriana Gabriela
4f9174fe-482a-4114-8e81-79b835946224
Antiufieieva, Yuliia
89075d6f-3114-418a-95e4-b63d8a7e1f8a
Wilde, Adriana Gabriela
4f9174fe-482a-4114-8e81-79b835946224

Antiufieieva, Yuliia and Wilde, Adriana Gabriela (2026) Responsible use of language AI in healthcare: towards evaluating and regulating large language models for multilingual communication in the NHS. CONVERGENCE 2026: Human-AI Integration for Multilingual and Accessible Communication., University of Surrey, Guildford, United Kingdom. 17 - 19 Jun 2026.

Record type: Conference or Workshop Item (Paper)

Abstract

Shortages of professional interpreters in healthcare can create communication risks for patients with limited English proficiency, while also leading practitioners to rely on ad hoc alternatives, including family members, untrained interpreters, or increasingly, machine translation and large language model (LLM)-based tools. Yet despite growing informal use of such technologies, there remains a significant gap between technical research on language AI, sociotechnical understandings of multilingual communication, and the policy and regulatory frameworks needed to govern safe deployment in public-sector settings. This work addresses that gap through a sociotechnical and empirically informed examination of responsible language AI use in healthcare contexts, with the intention of studying this in the National Health Service (NHS) in the future.

Our work explores the ethical, technical and regulatory challenges associated with using machine translation and LLMs to support multilingual communication when professional interpreters are unavailable. It focuses particularly on questions of translation quality, clinical risk, accountability, and appropriateness of use across different healthcare settings, including general practice, accident and emergency, and pharmacy interactions. Rather than treating machine translation as a substitute for professional interpreters, the work investigates conditions under which its use may, or may not, be justified as a risk-managed communication support tool.

We report findings from our pilot study, which used 10 healthcare case studies designed by medical professionals to generate simulated multilingual conversations with Google Translate and OpenAI GPT-5 for English–Ukrainian and English–Russian language pairs. Initial results suggest high translation quality and accuracy for these simulated conversational scenarios, differing from findings in earlier studies focused on medical text translation. We discuss factors that may influence these outcomes, including language pair characteristics, domain specificity, conversational structure, and model choice.

Building on these findings, the project will propose a framework for responsible use of language AI in healthcare that integrates technical evaluation, including BLEU and assessment of clinically significant errors, with broader governance considerations, including accountability pathways, trust, and institutional variation. In doing so, the paper contributes to debates on ethical aspects of AI in translation and interpreting, the role of LLMs in multilingual communication, and the responsible deployment of language AI in the public sector. It argues that evidence-based regulation, rather than unregulated adoption or blanket prohibition, is necessary to support safer and more equitable multilingual healthcare communication.

This record has no associated files available for download.

More information

Accepted/In Press date: 1 May 2026
Published date: 17 June 2026
Venue - Dates: CONVERGENCE 2026: Human-AI Integration for Multilingual and Accessible Communication., University of Surrey, Guildford, United Kingdom, 2026-06-17 - 2026-06-19

Identifiers

Local EPrints ID: 513411
URI: https://eprints.soton.ac.uk/id/eprint/513411
PURE UUID: 327b3968-e95c-47e7-ac70-8f2bb8e3b21e
ORCID for Adriana Gabriela Wilde: ORCID iD orcid.org/0000-0002-1684-1539

Catalogue record

Date deposited: 18 Aug 2026 16:40
Last modified: 15 Oct 2026 01:54

Export record

Contributors

Author: Yuliia Antiufieieva
Author: Adriana Gabriela Wilde ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×