Peer review of health research funding proposals: a systematic map and systematic review of innovations for effectiveness and efficiency

Abstract

Objective

To investigate methods and processes for timely, efficient and good quality peer review of research funding proposals in health.


Methods

A two-stage evidence synthesis: (1) a systematic map to describe the key characteristics of the evidence base, followed by (2) a systematic review of the studies stakeholders prioritised as relevant from the map on the effectiveness and efficiency of peer review ‘innovations’. Standard processes included literature searching, duplicate inclusion criteria screening, study keyword coding, data extraction, critical appraisal and study synthesis.


Results

A total of 83 studies from 15 countries were included in the systematic map. The evidence base is diverse, investigating many aspects of the systems for, and processes of, peer review. The systematic review included eight studies from Australia, Canada, and the USA, evaluating a broad range of peer review innovations. These studies showed that simplifying the process by shortening proposal forms, using smaller reviewer panels, or expediting processes can speed up the review process and reduce costs, but this might come at the expense of peer review quality, a key aspect that has not been assessed. Virtual peer review using videoconferencing or teleconferencing appears promising for reducing costs by avoiding the need for reviewers to travel, but again any consequences for quality have not been adequately assessed.


Conclusions

There is increasing international research activity into the peer review of health research funding. The studies reviewed had methodological limitations and variable generalisability to research funders. Given these limitations it is not currently possible to recommend immediate implementation of these innovations. However, many appear promising based on existing evidence, and could be adapted as necessary by funders and evaluated. Where feasible, experimental evaluation, including randomised controlled trials, should be conducted, evaluating impact on effectiveness, efficiency and quality.

More information

Identifiers

ORCID for Jonathan Shepherd: orcid.org/0000-0003-1682-4330

ORCID for Geoffrey Frampton: orcid.org/0000-0003-2005-0497

ORCID for Karen Pickett: orcid.org/0000-0002-8631-6465

ORCID for Jeremy Wyatt: orcid.org/0000-0001-7008-1473

Catalogue record

Export record

Altmetrics