Barriers and facilitators to model replication within health economics
Barriers and facilitators to model replication within health economics
BACKGROUND: Model replication is important because it enables researchers to check research integrity and transparency and, potentially, to inform the model conceptualization process when developing a new or updated model.
OBJECTIVE: The aim of this study was to evaluate the replicability of published decision analytic models and to identify the barriers and facilitators to replication.
METHODS: Replication attempts of 5 published economic modeling studies were made. The replications were conducted using only publicly available information within the manuscripts and supplementary materials. The replicator attempted to reproduce the key results detailed in the paper, for example, the total cost, total outcomes, and if applicable, incremental cost-effectiveness ratio reported. Although a replication attempt was not explicitly defined as a success or failure, the replicated results were compared for percentage difference to the original results.
RESULTS: In conducting the replication attempts, common barriers and facilitators emerged. For most case studies, the replicator needed to make additional assumptions when recreating the model. This was often exacerbated by conflicting information being presented in the text and the tables. Across the case studies, the variation between original and replicated results ranged from -4.54% to 108.00% for costs and -3.81% to 0.40% for outcomes.
CONCLUSION: This study demonstrates that although models may appear to be comprehensively reported, it is often not enough to facilitate a precise replication. Further work is needed to understand how to improve model transparency and in turn increase the chances of replication, thus ensuring future usability.
Cost-Benefit Analysis, Decision Making, Economics, Medical, Humans, Models, Economic, Reproducibility of Results
1018-1025
McManus, Emma
f04f4622-5b27-41f6-ae69-c4a24b9b87f5
Turner, David
407ea6bc-44cc-4afd-927d-ea953190e60a
Gray, Ewan
640f76cd-997b-4a8d-8690-081ffdcd7d01
Khawar, Haseeb
e610bf9d-8faf-4422-8abd-6516c034f4a9
Okoli, Toochukwu
c320b4c2-1638-4230-aaf4-96b58f82a894
Sach, Tracey
5c09256f-ebed-4d14-853a-181f6c92d6f2
16 July 2019
McManus, Emma
f04f4622-5b27-41f6-ae69-c4a24b9b87f5
Turner, David
407ea6bc-44cc-4afd-927d-ea953190e60a
Gray, Ewan
640f76cd-997b-4a8d-8690-081ffdcd7d01
Khawar, Haseeb
e610bf9d-8faf-4422-8abd-6516c034f4a9
Okoli, Toochukwu
c320b4c2-1638-4230-aaf4-96b58f82a894
Sach, Tracey
5c09256f-ebed-4d14-853a-181f6c92d6f2
McManus, Emma, Turner, David, Gray, Ewan, Khawar, Haseeb, Okoli, Toochukwu and Sach, Tracey
(2019)
Barriers and facilitators to model replication within health economics.
Value in Health, 22 (9), .
(doi:10.1016/j.jval.2019.04.1928).
Abstract
BACKGROUND: Model replication is important because it enables researchers to check research integrity and transparency and, potentially, to inform the model conceptualization process when developing a new or updated model.
OBJECTIVE: The aim of this study was to evaluate the replicability of published decision analytic models and to identify the barriers and facilitators to replication.
METHODS: Replication attempts of 5 published economic modeling studies were made. The replications were conducted using only publicly available information within the manuscripts and supplementary materials. The replicator attempted to reproduce the key results detailed in the paper, for example, the total cost, total outcomes, and if applicable, incremental cost-effectiveness ratio reported. Although a replication attempt was not explicitly defined as a success or failure, the replicated results were compared for percentage difference to the original results.
RESULTS: In conducting the replication attempts, common barriers and facilitators emerged. For most case studies, the replicator needed to make additional assumptions when recreating the model. This was often exacerbated by conflicting information being presented in the text and the tables. Across the case studies, the variation between original and replicated results ranged from -4.54% to 108.00% for costs and -3.81% to 0.40% for outcomes.
CONCLUSION: This study demonstrates that although models may appear to be comprehensively reported, it is often not enough to facilitate a precise replication. Further work is needed to understand how to improve model transparency and in turn increase the chances of replication, thus ensuring future usability.
This record has no associated files available for download.
More information
Published date: 16 July 2019
Additional Information:
Copyright © 2019 ISPOR–The Professional Society for Health Economics and Outcomes Research. Published by Elsevier Inc. All rights reserved.
Keywords:
Cost-Benefit Analysis, Decision Making, Economics, Medical, Humans, Models, Economic, Reproducibility of Results
Identifiers
Local EPrints ID: 480881
URI: http://eprints.soton.ac.uk/id/eprint/480881
ISSN: 1098-3015
PURE UUID: 1ab55563-da30-41f1-a586-32a524c352f6
Catalogue record
Date deposited: 10 Aug 2023 16:41
Last modified: 17 Mar 2024 04:20
Export record
Altmetrics
Contributors
Author:
Emma McManus
Author:
David Turner
Author:
Ewan Gray
Author:
Haseeb Khawar
Author:
Toochukwu Okoli
Author:
Tracey Sach
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics