The University of Southampton
University of Southampton Institutional Repository

Pilot testing of the Research Impact Assessment Framework

Pilot testing of the Research Impact Assessment Framework
Pilot testing of the Research Impact Assessment Framework

Traditional academic impact frameworks and metrics have been criticised, because they fail to assess the 'real-world' value of research. The Research Impact Assessment Framework was developed to complement and extend existing impact assessment frameworks by focusing on health system needs and priorities. This paper reports on piloting of the Research Impact Assessment Framework for feasibility and utility with researchers working in health and research organisations in Sydney, Australia. The Framework's Research Environment domain was assessed through a bespoke employee survey, and the Alignment and Influence of Research domain through independent review of case studies. Using synthetic data, the outputs from the two domains were combined to compare institutions. The tools tested in this pilot are feasible to use in assessing an organisation's capacity to produce impactful research that is aligned with identified health, social and economic priorities. The case study reviews illustrated the challenge of determining potential versus realised impact. This study identified that implementation of the Research Impact Assessment Framework will be enabled by use of existing routinely collected data, such as annual employee surveys, and the use of artificial intelligence tools to reduce administrative burden and ensure consistency of outputs.

governance, health policy, health system, impact, investment decisions, performance and evaluation, research funding, research impact, translation, value
Ward, Robyn L.
d5d8f55c-61d5-4c5e-bd04-183ab4e195e5
Nutbeam, Don
dbd2271d-7e7c-40cf-9c8e-663d01580e47
Mijnhardt, Wilfred
d1bf2eab-ffa5-4c6d-be61-26c3465c34e0
Nelson, Philip
5c6f5cc9-ea52-4fe2-9edf-05d696b0c1a9
Todd, Angela L.
600f6c29-657a-46b0-990d-4f301966d4d0
Richards, Janine
ff059d33-31f1-45fc-9fc2-110cb0a6ec3d
Jackson, Kirsten
768f5aba-be4a-4976-bf7d-8f5a6c26eee2
Khan, Nadia N.
609226ea-7e58-4a23-9a2a-6e135d9d1fcc
Chung, Sean
a110249f-47bf-45f7-beac-12a9d93cb9a7
Ward, Robyn L.
d5d8f55c-61d5-4c5e-bd04-183ab4e195e5
Nutbeam, Don
dbd2271d-7e7c-40cf-9c8e-663d01580e47
Mijnhardt, Wilfred
d1bf2eab-ffa5-4c6d-be61-26c3465c34e0
Nelson, Philip
5c6f5cc9-ea52-4fe2-9edf-05d696b0c1a9
Todd, Angela L.
600f6c29-657a-46b0-990d-4f301966d4d0
Richards, Janine
ff059d33-31f1-45fc-9fc2-110cb0a6ec3d
Jackson, Kirsten
768f5aba-be4a-4976-bf7d-8f5a6c26eee2
Khan, Nadia N.
609226ea-7e58-4a23-9a2a-6e135d9d1fcc
Chung, Sean
a110249f-47bf-45f7-beac-12a9d93cb9a7

Ward, Robyn L., Nutbeam, Don, Mijnhardt, Wilfred, Nelson, Philip, Todd, Angela L., Richards, Janine, Jackson, Kirsten, Khan, Nadia N. and Chung, Sean (2025) Pilot testing of the Research Impact Assessment Framework. Australian Health Review, 49 (6), [AH25214]. (doi:10.1071/AH25214).

Record type: Article

Abstract

Traditional academic impact frameworks and metrics have been criticised, because they fail to assess the 'real-world' value of research. The Research Impact Assessment Framework was developed to complement and extend existing impact assessment frameworks by focusing on health system needs and priorities. This paper reports on piloting of the Research Impact Assessment Framework for feasibility and utility with researchers working in health and research organisations in Sydney, Australia. The Framework's Research Environment domain was assessed through a bespoke employee survey, and the Alignment and Influence of Research domain through independent review of case studies. Using synthetic data, the outputs from the two domains were combined to compare institutions. The tools tested in this pilot are feasible to use in assessing an organisation's capacity to produce impactful research that is aligned with identified health, social and economic priorities. The case study reviews illustrated the challenge of determining potential versus realised impact. This study identified that implementation of the Research Impact Assessment Framework will be enabled by use of existing routinely collected data, such as annual employee surveys, and the use of artificial intelligence tools to reduce administrative burden and ensure consistency of outputs.

Text
ah25214 - Version of Record
Download (859kB)

More information

Accepted/In Press date: 5 September 2025
e-pub ahead of print date: 14 October 2025
Published date: 4 December 2025
Keywords: governance, health policy, health system, impact, investment decisions, performance and evaluation, research funding, research impact, translation, value

Identifiers

Local EPrints ID: 507278
URI: http://eprints.soton.ac.uk/id/eprint/507278
PURE UUID: b0960248-1c7b-4d13-ab5e-33d1543355a7
ORCID for Philip Nelson: ORCID iD orcid.org/0000-0002-9563-3235

Catalogue record

Date deposited: 03 Dec 2025 17:30
Last modified: 06 Dec 2025 02:31

Export record

Altmetrics

Contributors

Author: Robyn L. Ward
Author: Don Nutbeam
Author: Wilfred Mijnhardt
Author: Philip Nelson ORCID iD
Author: Angela L. Todd
Author: Janine Richards
Author: Kirsten Jackson
Author: Nadia N. Khan
Author: Sean Chung

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×