The University of Southampton
University of Southampton Institutional Repository

Do the findings of case series studies vary significantly according to methodological characteristics?

Do the findings of case series studies vary significantly according to methodological characteristics?
Do the findings of case series studies vary significantly according to methodological characteristics?
OBJECTIVES: To review the use of case series in National Institute for Clinical Excellence (NICE) Health Technology Assessment (HTA) reports, to review systematically the methodological literature for papers relating to the validity of aspects of case series design, and to investigate characteristics and findings of case series using examples from the UK's Health Technology Assessment programme.
DATA SOURCES: Electronic databases. NICE website. Reports produced as part of the UK's HTA programme.
REVIEW METHODS: NICE HTAs that used information from case series studies were obtained from the NICE website and a range of quality criteria applied. Searches of electronic databases, handsearched journals and the bibliographies of papers were made in order to find studies that assessed aspects of case series design, analysis or quality in relation to study validity. Hypotheses relating to the design of case series studies were developed and empirically investigated using four case examples from existing reports produced as part of the UK's HTA programme (functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic angina). Analysis was undertaken comparing studies within each review.
RESULTS: There was no consensus on which case series to include in HTAs, how to use them or how to assess their quality, despite them being used in 30% of NICE HTAs. No previous studies empirically investigating methodological characteristics of case series were found. However, it is possible that the search strategy failed to find relevant studies. Poor reporting of case series characteristics severely constrained analysis and there were insufficient data to investigate all the hypotheses. Findings were not consistent across the different topics and were subject to considerable uncertainty. All the examples in our analysis were surgical interventions, which are prone to additional confounding factors due to difficulties of standardisation compared with drug treatment. Our findings may not be generalisable outside the interventions studied. The case series reports included generally exhibited poor reporting of methodological characteristics. This constrained our analysis. The use of several methods of analysis has led to apparently discrepant results. Given the number of analysis performed, the usual level of significance (p = 0.05) should be viewed with caution. The most important limitation of this study is the small number of cases on which the findings are based. The results are therefore tentative and should be viewed with caution.
CONCLUSIONS: Case series are incorporated in a significant proportion of health technology assessments. Quality criteria have been used to appraise the quality of case series and decide on their inclusion in reviews of studies using this design. In this small series of case studies drawn from HTAs carried out for the NHS HTA programme, little evidence was found to support the use of many of the factors included in quality assessment tools. Importantly, no relationship was found between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, such as blinding of outcome measurement, but these were not shown consistently across the small number of examples studied. This study is based on a very small sample of studies and should therefore be considered as exploratory. Further investigation of the relationship between methodological features and outcome is justified given the frequency of use of case series in health technology assessments. Further research into the methodological features of case series and their outcome is justified in a wider sample of technologies and larger sets of case series. Value of information analyses including case series could be explored. Further exploration of the differences between case series and randomised controlled trial results, preferably using registry or comprehensive case series data, would be valuable. [References: 370]
Academies and Institutes Angina Pectoris Angioplasty,Balloon Chronic Disease Coronary Artery Bypass Databases,Bibliographic Electric Stimulation Endoscopy Great Britain Humans Low Back Pain medline Methods Methods. Nasal Polyps Outcome Assessment (Health Care) Reproducibility of Results Research Design Standards Surgery Technology Assessment,Biomedical Therapy
1366-5278
Dalziel, K.
a63f012b-052c-4e76-842f-e063c8a21ff3
Round, A.
951bf70f-06bd-4769-ba02-6693af15c39e
Stein, K.
dba3ca57-81c5-4172-a80e-2b38f61a7cc1
Garside, R.
7178cdef-fe0c-4bba-92f8-8c23d1f50386
Castelnuovo, E.
94e87c73-3d71-40e0-b451-2cfd95270489
Payne, L.
862f8fcf-711d-4146-a723-a9109339c70a
Dalziel, K.
a63f012b-052c-4e76-842f-e063c8a21ff3
Round, A.
951bf70f-06bd-4769-ba02-6693af15c39e
Stein, K.
dba3ca57-81c5-4172-a80e-2b38f61a7cc1
Garside, R.
7178cdef-fe0c-4bba-92f8-8c23d1f50386
Castelnuovo, E.
94e87c73-3d71-40e0-b451-2cfd95270489
Payne, L.
862f8fcf-711d-4146-a723-a9109339c70a

Dalziel, K., Round, A., Stein, K., Garside, R., Castelnuovo, E. and Payne, L. (2005) Do the findings of case series studies vary significantly according to methodological characteristics? Health Technology Assessment, 9 (2). (doi:10.3310/hta9020).

Record type: Article

Abstract

OBJECTIVES: To review the use of case series in National Institute for Clinical Excellence (NICE) Health Technology Assessment (HTA) reports, to review systematically the methodological literature for papers relating to the validity of aspects of case series design, and to investigate characteristics and findings of case series using examples from the UK's Health Technology Assessment programme.
DATA SOURCES: Electronic databases. NICE website. Reports produced as part of the UK's HTA programme.
REVIEW METHODS: NICE HTAs that used information from case series studies were obtained from the NICE website and a range of quality criteria applied. Searches of electronic databases, handsearched journals and the bibliographies of papers were made in order to find studies that assessed aspects of case series design, analysis or quality in relation to study validity. Hypotheses relating to the design of case series studies were developed and empirically investigated using four case examples from existing reports produced as part of the UK's HTA programme (functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic angina). Analysis was undertaken comparing studies within each review.
RESULTS: There was no consensus on which case series to include in HTAs, how to use them or how to assess their quality, despite them being used in 30% of NICE HTAs. No previous studies empirically investigating methodological characteristics of case series were found. However, it is possible that the search strategy failed to find relevant studies. Poor reporting of case series characteristics severely constrained analysis and there were insufficient data to investigate all the hypotheses. Findings were not consistent across the different topics and were subject to considerable uncertainty. All the examples in our analysis were surgical interventions, which are prone to additional confounding factors due to difficulties of standardisation compared with drug treatment. Our findings may not be generalisable outside the interventions studied. The case series reports included generally exhibited poor reporting of methodological characteristics. This constrained our analysis. The use of several methods of analysis has led to apparently discrepant results. Given the number of analysis performed, the usual level of significance (p = 0.05) should be viewed with caution. The most important limitation of this study is the small number of cases on which the findings are based. The results are therefore tentative and should be viewed with caution.
CONCLUSIONS: Case series are incorporated in a significant proportion of health technology assessments. Quality criteria have been used to appraise the quality of case series and decide on their inclusion in reviews of studies using this design. In this small series of case studies drawn from HTAs carried out for the NHS HTA programme, little evidence was found to support the use of many of the factors included in quality assessment tools. Importantly, no relationship was found between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, such as blinding of outcome measurement, but these were not shown consistently across the small number of examples studied. This study is based on a very small sample of studies and should therefore be considered as exploratory. Further investigation of the relationship between methodological features and outcome is justified given the frequency of use of case series in health technology assessments. Further research into the methodological features of case series and their outcome is justified in a wider sample of technologies and larger sets of case series. Value of information analyses including case series could be explored. Further exploration of the differences between case series and randomised controlled trial results, preferably using registry or comprehensive case series data, would be valuable. [References: 370]

Full text not available from this repository.

More information

Published date: January 2005
Additional Information: cao, 9706284 Case Reports. Journal Article. Review English included in summary CV for PD RefMgr field[16]: England
Keywords: Academies and Institutes Angina Pectoris Angioplasty,Balloon Chronic Disease Coronary Artery Bypass Databases,Bibliographic Electric Stimulation Endoscopy Great Britain Humans Low Back Pain medline Methods Methods. Nasal Polyps Outcome Assessment (Health Care) Reproducibility of Results Research Design Standards Surgery Technology Assessment,Biomedical Therapy

Identifiers

Local EPrints ID: 415429
URI: https://eprints.soton.ac.uk/id/eprint/415429
ISSN: 1366-5278
PURE UUID: a039fcde-9dcb-443f-9600-c745d4ce2569
ORCID for L. Payne: ORCID iD orcid.org/0000-0002-6594-5668

Catalogue record

Date deposited: 09 Nov 2017 17:30
Last modified: 14 Mar 2019 01:45

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×