Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank
Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank
Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.
Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.
Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).
Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB’s developers.
AC1, Meta-analysis, Quality assessment, Systematic review
981-993
Margulis, Andrea V.
fcfc9ec7-09a0-403c-816d-91080ab312e1
Pladevall, Manel
47941c64-421e-41c2-b361-e547fe1c2d94
Riera-guardia, Nuria
7076a770-4fae-4a17-933d-3ad180042c2f
Varas-Lorenzo, Cristina
a4a931ab-98f3-4d37-a1ce-899b335b5150
Hazell, Lorna
1c9036d8-13c0-4fe1-88be-9a926dc003b5
Berkman, Nancy D.
da9ed613-980a-438a-9ceb-5afdc3bba8d2
Viswanathan, Meera
7e982e66-b4a3-4526-a272-bad8ede9d5ca
Perez-Gutthann, Susana
32401fa5-7336-49d5-ac93-bc5e455866ce
1 January 2014
Margulis, Andrea V.
fcfc9ec7-09a0-403c-816d-91080ab312e1
Pladevall, Manel
47941c64-421e-41c2-b361-e547fe1c2d94
Riera-guardia, Nuria
7076a770-4fae-4a17-933d-3ad180042c2f
Varas-Lorenzo, Cristina
a4a931ab-98f3-4d37-a1ce-899b335b5150
Hazell, Lorna
1c9036d8-13c0-4fe1-88be-9a926dc003b5
Berkman, Nancy D.
da9ed613-980a-438a-9ceb-5afdc3bba8d2
Viswanathan, Meera
7e982e66-b4a3-4526-a272-bad8ede9d5ca
Perez-Gutthann, Susana
32401fa5-7336-49d5-ac93-bc5e455866ce
Margulis, Andrea V., Pladevall, Manel, Riera-guardia, Nuria, Varas-Lorenzo, Cristina, Hazell, Lorna, Berkman, Nancy D., Viswanathan, Meera and Perez-Gutthann, Susana
(2014)
Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank.
Clinical Epidemiology, 6, .
(doi:10.2147/CLEP.S66677).
Abstract
Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.
Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.
Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).
Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB’s developers.
This record has no associated files available for download.
More information
Published date: 1 January 2014
Keywords:
AC1, Meta-analysis, Quality assessment, Systematic review
Identifiers
Local EPrints ID: 439197
URI: http://eprints.soton.ac.uk/id/eprint/439197
ISSN: 1179-1349
PURE UUID: b3fc1b0d-fdcb-4414-b8b6-275cff14fd6c
Catalogue record
Date deposited: 06 Apr 2020 16:36
Last modified: 16 Mar 2024 06:53
Export record
Altmetrics
Contributors
Author:
Andrea V. Margulis
Author:
Manel Pladevall
Author:
Nuria Riera-guardia
Author:
Cristina Varas-Lorenzo
Author:
Nancy D. Berkman
Author:
Meera Viswanathan
Author:
Susana Perez-Gutthann
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics