The University of Southampton
University of Southampton Institutional Repository

Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank

Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank
Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank

Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.

Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.

Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).

Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB’s developers.

AC1, Meta-analysis, Quality assessment, Systematic review
1179-1349
981-993
Margulis, Andrea V.
fcfc9ec7-09a0-403c-816d-91080ab312e1
Pladevall, Manel
47941c64-421e-41c2-b361-e547fe1c2d94
Riera-guardia, Nuria
7076a770-4fae-4a17-933d-3ad180042c2f
Varas-Lorenzo, Cristina
a4a931ab-98f3-4d37-a1ce-899b335b5150
Hazell, Lorna
1c9036d8-13c0-4fe1-88be-9a926dc003b5
Berkman, Nancy D.
da9ed613-980a-438a-9ceb-5afdc3bba8d2
Viswanathan, Meera
7e982e66-b4a3-4526-a272-bad8ede9d5ca
Perez-Gutthann, Susana
32401fa5-7336-49d5-ac93-bc5e455866ce
Margulis, Andrea V.
fcfc9ec7-09a0-403c-816d-91080ab312e1
Pladevall, Manel
47941c64-421e-41c2-b361-e547fe1c2d94
Riera-guardia, Nuria
7076a770-4fae-4a17-933d-3ad180042c2f
Varas-Lorenzo, Cristina
a4a931ab-98f3-4d37-a1ce-899b335b5150
Hazell, Lorna
1c9036d8-13c0-4fe1-88be-9a926dc003b5
Berkman, Nancy D.
da9ed613-980a-438a-9ceb-5afdc3bba8d2
Viswanathan, Meera
7e982e66-b4a3-4526-a272-bad8ede9d5ca
Perez-Gutthann, Susana
32401fa5-7336-49d5-ac93-bc5e455866ce

Margulis, Andrea V., Pladevall, Manel, Riera-guardia, Nuria, Varas-Lorenzo, Cristina, Hazell, Lorna, Berkman, Nancy D., Viswanathan, Meera and Perez-Gutthann, Susana (2014) Quality assessment of observational studies in a drug-safety systematic review, Comparison of two tools: The Newcastle-Ottawa scale and the RTI item bank. Clinical Epidemiology, 6, 981-993. (doi:10.2147/CLEP.S66677).

Record type: Article

Abstract

Methods: We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.

Background: The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.

Results: The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).

Conclusion: The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB’s developers.

This record has no associated files available for download.

More information

Published date: 1 January 2014
Keywords: AC1, Meta-analysis, Quality assessment, Systematic review

Identifiers

Local EPrints ID: 439197
URI: http://eprints.soton.ac.uk/id/eprint/439197
ISSN: 1179-1349
PURE UUID: b3fc1b0d-fdcb-4414-b8b6-275cff14fd6c
ORCID for Lorna Hazell: ORCID iD orcid.org/0000-0002-5962-0648

Catalogue record

Date deposited: 06 Apr 2020 16:36
Last modified: 16 Mar 2024 06:53

Export record

Altmetrics

Contributors

Author: Andrea V. Margulis
Author: Manel Pladevall
Author: Nuria Riera-guardia
Author: Cristina Varas-Lorenzo
Author: Lorna Hazell ORCID iD
Author: Nancy D. Berkman
Author: Meera Viswanathan
Author: Susana Perez-Gutthann

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×