The University of Southampton
University of Southampton Institutional Repository

The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis

The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis
The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis
BACKGROUND: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncertainties around the company's model and their implications for decision-making. The ERG reports are a central component of the evidence considered by the NICE Technology Appraisal Committees (ACs) in their deliberations.

OBJECTIVE: The aim of this research was to develop an understanding of the number and type of exploratory analyses undertaken by the ERGs within the STA process and to understand how these analyses are used by the NICE ACs in their decision-making.

METHODS: The 100 most recently completed STAs with published guidance were selected for inclusion in the analysis. The documents considered were ERG reports, clarification letters, the first appraisal consultation document and the final appraisal determination. Over 400 documents were assessed in this study. The categories of types of exploratory analyses included fixing errors, fixing violations, addressing matters of judgement and the ERG-preferred base case. A content analysis of documents (documentary analysis) was undertaken to identify and extract relevant data, and narrative synthesis was then used to rationalise and present these data.

RESULTS: The level and type of detail in ERG reports and clarification letters varied considerably. The vast majority (93%) of ERG reports reported one or more exploratory analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category 'matters of judgement', which was reported in 83 (89%) reports. The category 'ERG base-case/preferred analysis' was reported in 45 (48%) reports, the category 'fixing errors' was reported in 33 (35%) reports and the category 'fixing violations' was reported in 17 (18%) reports. The exploratory analyses performed were the result of issues raised by an ERG in its critique of the submitted economic evidence. These analyses had more influence on recommendations earlier in the STA process than later on in the process.

LIMITATIONS: The descriptions of analyses undertaken were often highly specific to a particular STA and could be inconsistent across ERG reports and thus difficult to interpret.

CONCLUSIONS: Evidence Review Groups frequently conduct exploratory analyses to test or improve the economic evaluations submitted by companies as part of the STA process. ERG exploratory analyses often have an influence on the recommendations produced by the ACs.

FUTURE WORK: More in-depth analysis is needed to understand how ERGs make decisions regarding which exploratory analyses should be undertaken. More research is also needed to fully understand which types of exploratory analyses are most useful to ACs in their decision-making.

FUNDING: The National Institute for Health Research Health Technology Assessment programme.
1366-5278
Kaltenthaler, Eva
ce3c498d-a652-494f-a0a8-56f7cfd0b162
Carroll, Christopher
73b828e7-ad05-42f3-8878-87c5168e8b9c
Hill-McManus, Daniel
1c4481a7-a4e0-489f-9420-0af0fc3395c3
Scope, Alison
ae59f074-f374-47fc-996e-a911373add8a
Holmes, Michael
0728404e-9cbc-4ddc-8224-0503ea919edc
Rice, Stephen
536a9393-7785-42d8-aa04-f00ceacb076b
Rose, Micah
f6deee44-f21f-4d14-90a9-f7a449d0adba
Tappenden, Paul
aa3fcbbb-cf8c-4c59-a7de-4ab870d689fe
Woolacott, Nerys
51a97637-05a7-49ec-9209-993ee8d6ae4c
Kaltenthaler, Eva
ce3c498d-a652-494f-a0a8-56f7cfd0b162
Carroll, Christopher
73b828e7-ad05-42f3-8878-87c5168e8b9c
Hill-McManus, Daniel
1c4481a7-a4e0-489f-9420-0af0fc3395c3
Scope, Alison
ae59f074-f374-47fc-996e-a911373add8a
Holmes, Michael
0728404e-9cbc-4ddc-8224-0503ea919edc
Rice, Stephen
536a9393-7785-42d8-aa04-f00ceacb076b
Rose, Micah
f6deee44-f21f-4d14-90a9-f7a449d0adba
Tappenden, Paul
aa3fcbbb-cf8c-4c59-a7de-4ab870d689fe
Woolacott, Nerys
51a97637-05a7-49ec-9209-993ee8d6ae4c

Kaltenthaler, Eva, Carroll, Christopher, Hill-McManus, Daniel, Scope, Alison, Holmes, Michael, Rice, Stephen, Rose, Micah, Tappenden, Paul and Woolacott, Nerys (2016) The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis. Health Technology Assessment, 20 (26). (doi:10.3310/hta20260).

Record type: Article

Abstract

BACKGROUND: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncertainties around the company's model and their implications for decision-making. The ERG reports are a central component of the evidence considered by the NICE Technology Appraisal Committees (ACs) in their deliberations.

OBJECTIVE: The aim of this research was to develop an understanding of the number and type of exploratory analyses undertaken by the ERGs within the STA process and to understand how these analyses are used by the NICE ACs in their decision-making.

METHODS: The 100 most recently completed STAs with published guidance were selected for inclusion in the analysis. The documents considered were ERG reports, clarification letters, the first appraisal consultation document and the final appraisal determination. Over 400 documents were assessed in this study. The categories of types of exploratory analyses included fixing errors, fixing violations, addressing matters of judgement and the ERG-preferred base case. A content analysis of documents (documentary analysis) was undertaken to identify and extract relevant data, and narrative synthesis was then used to rationalise and present these data.

RESULTS: The level and type of detail in ERG reports and clarification letters varied considerably. The vast majority (93%) of ERG reports reported one or more exploratory analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category 'matters of judgement', which was reported in 83 (89%) reports. The category 'ERG base-case/preferred analysis' was reported in 45 (48%) reports, the category 'fixing errors' was reported in 33 (35%) reports and the category 'fixing violations' was reported in 17 (18%) reports. The exploratory analyses performed were the result of issues raised by an ERG in its critique of the submitted economic evidence. These analyses had more influence on recommendations earlier in the STA process than later on in the process.

LIMITATIONS: The descriptions of analyses undertaken were often highly specific to a particular STA and could be inconsistent across ERG reports and thus difficult to interpret.

CONCLUSIONS: Evidence Review Groups frequently conduct exploratory analyses to test or improve the economic evaluations submitted by companies as part of the STA process. ERG exploratory analyses often have an influence on the recommendations produced by the ACs.

FUTURE WORK: More in-depth analysis is needed to understand how ERGs make decisions regarding which exploratory analyses should be undertaken. More research is also needed to fully understand which types of exploratory analyses are most useful to ACs in their decision-making.

FUNDING: The National Institute for Health Research Health Technology Assessment programme.

This record has no associated files available for download.

More information

e-pub ahead of print date: April 2016
Published date: April 2016

Identifiers

Local EPrints ID: 412818
URI: http://eprints.soton.ac.uk/id/eprint/412818
ISSN: 1366-5278
PURE UUID: cf29ddb8-25d4-4832-9151-56d9dbf57683

Catalogue record

Date deposited: 02 Aug 2017 16:30
Last modified: 10 May 2024 16:56

Export record

Altmetrics

Contributors

Author: Eva Kaltenthaler
Author: Christopher Carroll
Author: Daniel Hill-McManus
Author: Alison Scope
Author: Michael Holmes
Author: Stephen Rice
Author: Micah Rose
Author: Paul Tappenden
Author: Nerys Woolacott

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×