Potential use of routine databases in health technology assessment

Raftery, J., Roderick, P. and Stevens, A. (2005) Potential use of routine databases in health technology assessment. Health Technology Assessment, 9 (20), 1-106.

Record type: Article

Abstract

Introduction
This report defines health technology assessment to include the investigation of (i) effectiveness, (ii) diffusion and equity and (iii) cost – all as applied to the range of health technologies (HTs) including pharmaceuticals, devices, procedures and settings. Key characteristics of routine data are regular collection, standard definitions, obligatory completion and representative coverage.
Aims
The aims of this study were to:
1. develop criteria for classifying databases in relation to their potential use in HT assessment 2. list the databases of relevance in the UK 3. apply the criteria for classifying databases to that list 4. explore the extent to which prioritised databases could pick up those HTs being assessed by the National Coordinating Centre for Health Technology Assessment (NCCHTA) 5. investigate the extent to which these databases have been used in HT assessment 6. explore the degree to which databases, so used, have been validated 7. estimate the cost of the prioritised databases 8. make suggestions for facilitating the use of routine data for HT assessment.
Methods
A ‘first principles’ examination of the data necessary for each type of HT assessment was central to aim 1, supplemented by literature searches and a historical review.
A long list (aim 2) was developed using selected literature and by networking with people with relevant experience.
The principal investigators applied the criteria to the long list (aim 3) using annotations of each. Comments of the ‘keepers’ of the prioritised databases were incorporated.
For aim 4, details of 161 topics funded by the NHS R&D Health Technology Assessment (HTA) programme were reviewed iteratively by the principal investigators.
Uses of databases in HT assessments (aim 5) were identified by literature searches, which included the title of each prioritised database as a keyword. Annual reports of databases were examined and ‘keepers’ queried. Each identified use was checked by the three principal investigators.
The validity of each database (aim 6) was assessed using criteria based on a literature search and involvement by the authors in a national academic network. The ‘keepers’ of databases were queried.
The costs of databases (aim 7) were established from annual reports, enquiries to ‘keepers’ of databases and ‘guesstimates‘ based on cost per record.
The proposals under aim 8 were based on the above and discussion between authors.
Results
To be of value in HT assessment, databases must at least identify a well-defined HT. Additional dimensions depend on the type of HT assessment. For assessing effectiveness, equity and diffusion, routine databases were classified into three broad groups:
* group I databases, identifying both HTs and health states * group II databases, identifying the HTs, but not a health state * group III databases, identifying health states, but not an HT.
Group I datasets were disaggregated into clinical registries, clinical administrative databases and population-oriented databases. Group III were disaggregated into adverse event reporting, confidential enquiries, disease-only registers and health surveys.
Databases in group I can be used not only to assess effectiveness but also to assess diffusion and equity. Databases in group II can only assess diffusion. Group III has restricted scope for assessing HTs, except for analysis of adverse events.
For use in costing, databases need to include unit costs or prices. Some databases included unit cost as well as a specific HT.
A long list of around 270 databases was identified at the level of the UK, England and Wales or England (over 1000 including Scotland, Wales and Northern Ireland).
Allocation of these to the above groups identified around 60 databases with some potential for HT assessment, roughly half to group I. Eighteen clinical registers were identified as having the greatest potential although the clinical administrative datasets had potential mainly owing to their inclusion of a wide range of technologies. Only two databases were identified that could be directly used in costing.
The review of the potential capture of HTs prioritised by the UK’s NHS R&D HTA programme showed that only 10% would be captured in these databases, mainly drugs prescribed in primary care.
The review of the use of routine databases in any form of HT assessment indicated that clinical registers were mainly used for national comparative audit. Some databases have only been used in annual reports, usually time trend analysis. A few peer-reviewed papers used a clinical register to assess the effectiveness of a technology, particularly those with relatively simple outcomes (conceptions from in vitro fertilisation or graft failure in organ transplants). The authorship of such studies suggests that accessibility is a barrier to using most databases.
Clinical administrative databases (group Ib) have been mainly used to build population needs indices and performance indicators.
A review of the validity of used databases showed that although internal consistency checks were common, relatively few had any form of external audit. Some comparative audit databases have data scrutinised by participating units. Issues around coverage and coding have, in general, received little attention.
NHS funding of databases has been mainly for ‘Central Returns’ for management purposes, which excludes those databases with the greatest potential for HT assessment. Funding for these was various, but some are unfunded, relying on goodwill. The estimated total cost of databases in group I plus selected databases from groups II and III has been estimated at £50 million or around 0.1% of annual NHS spend. A few databases with limited potential for health technology assessment account for the bulk of spending.
Conclusions and recommendations for further research
Proposals for policy include clarification of responsibility for the strategic development of databases, improved resourcing, and issues around coding, confidentiality, ownership and access, maintenance of clinical support, optimal use of information technology, filling gaps and remedying deficiencies.
Recommendations for researchers include closer policy links between routine data and R&D, and selective investment in the more promising databases. Recommended research topics include optimal capture and coding of the range of HTs, international comparisons of the role, funding and use of routine data in healthcare systems and use of routine databases in trials and in modelling. Independent evaluations are recommended for information strategies (such as those around the NSFs and various collaborations) and for electronic patient and health records.