The University of Southampton
University of Southampton Institutional Repository

European network for Health Technology Assessment Joint Action (EUnetHTA JA): a process evaluation performed by questionnaires and documentary analysis

European network for Health Technology Assessment Joint Action (EUnetHTA JA): a process evaluation performed by questionnaires and documentary analysis
European network for Health Technology Assessment Joint Action (EUnetHTA JA): a process evaluation performed by questionnaires and documentary analysis
Background: The European network for Health Technology Assessment Joint Action (EUnetHTA JA) project’s overarching objective was to ‘establish an effective and sustainable HTA [Health technology assessment] collaboration in Europe that brings added value at the regional, national and European level’. Specific objectives were to develop a strategy and business model for sustainable European collaboration on HTA, develop HTA tools and methods and promote good practice in HTA methods and processes. We describe activities performed on behalf of the National Institute for Health Research HTA programme; evaluating the project processes and developing a data set for a registry of planned clinical studies of relevance to public funders.

Methods: Annual self-completion online questionnaires were sent to project participants and external stakeholders to identify their views about the project processes. Documentary review was undertaken at the project end on the final technical reports from the work packages to examine whether or not their deliverables had been achieved. The project’s impact was assessed by whether or not the deliverables were produced, the objectives met and additional ‘added value’ generated. The project’s effectiveness was evaluated by its processes, communication, administration, workings of individual work packages and involvement of external stakeholders. A two-stage Delphi exercise was undertaken to identify the data elements that should be included in a registry of planned clinical studies of relevance to public funders.
The data set was validated by an efficacy testing exercise.

Results and discussion: High response rates were achieved for the questionnaires sent to project participants and this was attributed to the evidence-based strategy implemented. Response rates to questionnaires sent to external stakeholders were disappointingly lower. Most of the high-level objectives were achieved, although applying the developed tools in practice will be implemented in the European network for Health Technology Assessment Joint Action 2 (EUnetHTA JA2). Most work packages produced their planned deliverables. Networking emerged as one of the main benefits of the project and face-to-face meetings were important. However, the overarching objective did not appear to have been met because there will be a follow-up EUnetHTA JA2 project (reliant on project funding) before the
establishment of any permanent network. Twelve organisations from three continents participated in the Delphi exercise to develop the data set. It was demonstrated that a registry for matching pragmatic
clinical studies under consideration by funders could be built on a very small data set. This would include 10 unique items, of which five are required to describe a study and the rest are metadata. In the test sample the data set with an appropriate matching rule was able to deliver a sensitivity of between 50% and 100% and a specificity of between 43% and 86% for matching different elements.

Conclusions: A number of recommendations have been made for the next EUnetHTA JA2 project and its evaluation. This included that the evaluation of the EUnetHTA JA2 project should extend beyond the end of the project to allow assessment of its impact; that the quality, usability and cost-effectiveness of tools in ‘real-world HTA practice’ should be assessed and tangible benefits of international networking should be evaluated. It is worth proceeding to develop a database registry aimed at identifying trials
in development based on the data set developed.


Funding: The study was funded by the National Institute for Health Research Health Technology Assessment programme (50%) and the European Union Commission (50%).
1366-5278
37
NIHR Journals Library
Woodford Guegan, Eleanor
c1e11e35-e05b-484e-a206-bc3b6db546db
Cook, Andrew
ab9c7bb3-974a-4db9-b3c2-9942988005d5
Woodford Guegan, Eleanor
c1e11e35-e05b-484e-a206-bc3b6db546db
Cook, Andrew
ab9c7bb3-974a-4db9-b3c2-9942988005d5

Woodford Guegan, Eleanor and Cook, Andrew (2014) European network for Health Technology Assessment Joint Action (EUnetHTA JA): a process evaluation performed by questionnaires and documentary analysis (Health Technology Assessment, , (doi:10.3310/hta18370), 37, 37) Southampton, GB. NIHR Journals Library 296pp.

Record type: Monograph (Project Report)

Abstract

Background: The European network for Health Technology Assessment Joint Action (EUnetHTA JA) project’s overarching objective was to ‘establish an effective and sustainable HTA [Health technology assessment] collaboration in Europe that brings added value at the regional, national and European level’. Specific objectives were to develop a strategy and business model for sustainable European collaboration on HTA, develop HTA tools and methods and promote good practice in HTA methods and processes. We describe activities performed on behalf of the National Institute for Health Research HTA programme; evaluating the project processes and developing a data set for a registry of planned clinical studies of relevance to public funders.

Methods: Annual self-completion online questionnaires were sent to project participants and external stakeholders to identify their views about the project processes. Documentary review was undertaken at the project end on the final technical reports from the work packages to examine whether or not their deliverables had been achieved. The project’s impact was assessed by whether or not the deliverables were produced, the objectives met and additional ‘added value’ generated. The project’s effectiveness was evaluated by its processes, communication, administration, workings of individual work packages and involvement of external stakeholders. A two-stage Delphi exercise was undertaken to identify the data elements that should be included in a registry of planned clinical studies of relevance to public funders.
The data set was validated by an efficacy testing exercise.

Results and discussion: High response rates were achieved for the questionnaires sent to project participants and this was attributed to the evidence-based strategy implemented. Response rates to questionnaires sent to external stakeholders were disappointingly lower. Most of the high-level objectives were achieved, although applying the developed tools in practice will be implemented in the European network for Health Technology Assessment Joint Action 2 (EUnetHTA JA2). Most work packages produced their planned deliverables. Networking emerged as one of the main benefits of the project and face-to-face meetings were important. However, the overarching objective did not appear to have been met because there will be a follow-up EUnetHTA JA2 project (reliant on project funding) before the
establishment of any permanent network. Twelve organisations from three continents participated in the Delphi exercise to develop the data set. It was demonstrated that a registry for matching pragmatic
clinical studies under consideration by funders could be built on a very small data set. This would include 10 unique items, of which five are required to describe a study and the rest are metadata. In the test sample the data set with an appropriate matching rule was able to deliver a sensitivity of between 50% and 100% and a specificity of between 43% and 86% for matching different elements.

Conclusions: A number of recommendations have been made for the next EUnetHTA JA2 project and its evaluation. This included that the evaluation of the EUnetHTA JA2 project should extend beyond the end of the project to allow assessment of its impact; that the quality, usability and cost-effectiveness of tools in ‘real-world HTA practice’ should be assessed and tangible benefits of international networking should be evaluated. It is worth proceeding to develop a database registry aimed at identifying trials
in development based on the data set developed.


Funding: The study was funded by the National Institute for Health Research Health Technology Assessment programme (50%) and the European Union Commission (50%).

Text
FullReport-hta18370.pdf - Other
Download (44MB)
Archive
HTA1837-05-52-03.zip - Other
Download (86MB)

More information

Published date: June 2014
Organisations: Faculty of Medicine

Identifiers

Local EPrints ID: 366250
URI: http://eprints.soton.ac.uk/id/eprint/366250
ISSN: 1366-5278
PURE UUID: fb257c9e-0fc5-4898-b2c3-ce082d528439
ORCID for Eleanor Woodford Guegan: ORCID iD orcid.org/0000-0001-7225-6249
ORCID for Andrew Cook: ORCID iD orcid.org/0000-0002-6680-439X

Catalogue record

Date deposited: 24 Jun 2014 09:36
Last modified: 18 Feb 2021 17:07

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×