The University of Southampton
University of Southampton Institutional Repository

The REal Life EVidence AssessmeNt Tool (RELEVANT): Development of a novel quality assurance asset to rate observational comparative effectiveness research studies

The REal Life EVidence AssessmeNt Tool (RELEVANT): Development of a novel quality assurance asset to rate observational comparative effectiveness research studies
The REal Life EVidence AssessmeNt Tool (RELEVANT): Development of a novel quality assurance asset to rate observational comparative effectiveness research studies

Background: Evidence from observational comparative effectiveness research (CER) is ranked below that from randomized controlled trials in traditional evidence hierarchies. However, asthma observational CER studies represent an important complementary evidence source answering different research questions and are particularly valuable in guiding clinical decision making in real-life patient and practice settings. Tools are required to assist in quality appraisal of observational CER to enable identification of and confidence in high-quality CER evidence to inform guideline development. Methods: The REal Life EVidence AssessmeNt Tool (RELEVANT) was developed through a step-wise approach. We conducted an iterative refinement of the tool based on Task Force member expertise and feedback from pilot testing the tool until reaching adequate inter-rater agreement percentages. Two distinct pilots were conducted - the first involving six members of the Respiratory Effectiveness Group (REG) and European Academy of Allergy and Clinical Immunology (EAACI) joint Task Force for quality appraisal of observational asthma CER; the second involving 22 members of REG and EAACI membership. The final tool consists of 21 quality sub-items distributed across seven methodology domains: Background, Design, Measures, Analysis, Results, Discussion/Interpretation, and Conflict of Interest. Eleven of these sub-items are considered critical and named "primary sub-items". Results: Following the second pilot, RELEVANT showed inter-rater agreement ≥ 70% for 94% of all primary and 93% for all secondary sub-items tested across three rater groups. For observational CER to be classified as sufficiently high quality for future guideline consideration, all RELEVANT primary sub-items must be fulfilled. The ten secondary sub-items further qualify the relative strengths and weaknesses of the published CER evidence. RELEVANT could also be applicable to general quality appraisal of observational CER across other medical specialties. Conclusions: RELEVANT is the first quality checklist to assist in the appraisal of published observational CER developed through iterative feedback derived from pilot implementation and inter-rater agreement evaluation. Developed for a REG-EAACI Task Force quality appraisal of recent asthma CER, RELEVANT also has wider utility to support appraisal of CER literature in general (including pre-publication). It may also assist in manuscript development and in educating relevant stakeholders about key quality markers in observational CER.

Assessment tool, Asthma, Comparative effectiveness research (CER), Observational studies, Quality
Campbell, Jonathan D.
a17142a5-833b-4da0-9b3d-b55ef9e77f03
Perry, Robert
34194792-4c05-4b3b-a8f5-29892295608f
Papadopoulos, Nikolaos G.
acf133b9-9a22-4637-a7f3-d67f509d9353
Krishnan, Jerry
738f057a-5ab2-4a6a-b4aa-bba676d37428
Brusselle, Guy
7b8ee2aa-06d6-437c-8591-b09be09fed7b
Chisholm, Alison
b9c6e5d9-a4f3-4d05-aaa4-a1435fdc7771
Bjermer, Leif
b1880328-5814-4334-b922-4ed3332b7bae
Thomas, Michael
997c78e0-3849-4ce8-b1bc-86ebbdee3953
Van Ganse, Eric
1b5ed09b-c6fd-49df-a284-3cda931a37d7
Van Den Berge, Maarten
998f7d7f-9878-4bf9-8734-dcfe65aff04e
Quint, Jennifer
22800655-8987-4464-8385-7f08860c92f9
Price, David
8c7cc7b9-d207-4364-9902-0a2a765bd450
Roche, Nicolas
59e3804d-1f4f-48f4-abbc-37c44279b126
Campbell, Jonathan D.
a17142a5-833b-4da0-9b3d-b55ef9e77f03
Perry, Robert
34194792-4c05-4b3b-a8f5-29892295608f
Papadopoulos, Nikolaos G.
acf133b9-9a22-4637-a7f3-d67f509d9353
Krishnan, Jerry
738f057a-5ab2-4a6a-b4aa-bba676d37428
Brusselle, Guy
7b8ee2aa-06d6-437c-8591-b09be09fed7b
Chisholm, Alison
b9c6e5d9-a4f3-4d05-aaa4-a1435fdc7771
Bjermer, Leif
b1880328-5814-4334-b922-4ed3332b7bae
Thomas, Michael
997c78e0-3849-4ce8-b1bc-86ebbdee3953
Van Ganse, Eric
1b5ed09b-c6fd-49df-a284-3cda931a37d7
Van Den Berge, Maarten
998f7d7f-9878-4bf9-8734-dcfe65aff04e
Quint, Jennifer
22800655-8987-4464-8385-7f08860c92f9
Price, David
8c7cc7b9-d207-4364-9902-0a2a765bd450
Roche, Nicolas
59e3804d-1f4f-48f4-abbc-37c44279b126

Campbell, Jonathan D., Perry, Robert, Papadopoulos, Nikolaos G., Krishnan, Jerry, Brusselle, Guy, Chisholm, Alison, Bjermer, Leif, Thomas, Michael, Van Ganse, Eric, Van Den Berge, Maarten, Quint, Jennifer, Price, David and Roche, Nicolas (2019) The REal Life EVidence AssessmeNt Tool (RELEVANT): Development of a novel quality assurance asset to rate observational comparative effectiveness research studies. Clinical and Translational Allergy, 9 (1). (doi:10.1186/s13601-019-0256-9).

Record type: Article

Abstract

Background: Evidence from observational comparative effectiveness research (CER) is ranked below that from randomized controlled trials in traditional evidence hierarchies. However, asthma observational CER studies represent an important complementary evidence source answering different research questions and are particularly valuable in guiding clinical decision making in real-life patient and practice settings. Tools are required to assist in quality appraisal of observational CER to enable identification of and confidence in high-quality CER evidence to inform guideline development. Methods: The REal Life EVidence AssessmeNt Tool (RELEVANT) was developed through a step-wise approach. We conducted an iterative refinement of the tool based on Task Force member expertise and feedback from pilot testing the tool until reaching adequate inter-rater agreement percentages. Two distinct pilots were conducted - the first involving six members of the Respiratory Effectiveness Group (REG) and European Academy of Allergy and Clinical Immunology (EAACI) joint Task Force for quality appraisal of observational asthma CER; the second involving 22 members of REG and EAACI membership. The final tool consists of 21 quality sub-items distributed across seven methodology domains: Background, Design, Measures, Analysis, Results, Discussion/Interpretation, and Conflict of Interest. Eleven of these sub-items are considered critical and named "primary sub-items". Results: Following the second pilot, RELEVANT showed inter-rater agreement ≥ 70% for 94% of all primary and 93% for all secondary sub-items tested across three rater groups. For observational CER to be classified as sufficiently high quality for future guideline consideration, all RELEVANT primary sub-items must be fulfilled. The ten secondary sub-items further qualify the relative strengths and weaknesses of the published CER evidence. RELEVANT could also be applicable to general quality appraisal of observational CER across other medical specialties. Conclusions: RELEVANT is the first quality checklist to assist in the appraisal of published observational CER developed through iterative feedback derived from pilot implementation and inter-rater agreement evaluation. Developed for a REG-EAACI Task Force quality appraisal of recent asthma CER, RELEVANT also has wider utility to support appraisal of CER literature in general (including pre-publication). It may also assist in manuscript development and in educating relevant stakeholders about key quality markers in observational CER.

Text
s13601-019-0256-9 - Version of Record
Available under License Creative Commons Attribution.
Download (901kB)

More information

Accepted/In Press date: 20 February 2019
Published date: 27 March 2019
Keywords: Assessment tool, Asthma, Comparative effectiveness research (CER), Observational studies, Quality

Identifiers

Local EPrints ID: 430072
URI: https://eprints.soton.ac.uk/id/eprint/430072
PURE UUID: aa13a88c-d44f-42db-8731-f95b29b61c95

Catalogue record

Date deposited: 11 Apr 2019 16:30
Last modified: 18 Jul 2019 10:16

Export record

Altmetrics

Contributors

Author: Jonathan D. Campbell
Author: Robert Perry
Author: Nikolaos G. Papadopoulos
Author: Jerry Krishnan
Author: Guy Brusselle
Author: Alison Chisholm
Author: Leif Bjermer
Author: Michael Thomas
Author: Eric Van Ganse
Author: Maarten Van Den Berge
Author: Jennifer Quint
Author: David Price
Author: Nicolas Roche

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×