The University of Southampton
University of Southampton Institutional Repository

Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study

Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study
Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study
Background
External pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.

Methods
We use a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes. We present the impact of increasing the pilot sample size on the precision and bias of these estimates, and predicted power under three realistic scenarios. We also illustrate the consequences of using a confidence interval argument to inflate estimates so the required power is achieved with a pre-specified level of confidence. We limit our attention to external pilot and feasibility studies prior to a two-parallel-balanced-group superiority RCT.

Results
For normally distributed outcomes, the relative gain in precision of the pooled standard deviation (SD p ) is less than 10% (for each five subjects added per group) once the total sample size is 70. For true proportions between 0.1 and 0.5, we find the gain in precision for each five subjects added to the pilot sample is less than 5% once the sample size is 60. Adjusting the required sample sizes for the imprecision in the pilot study estimates can result in excessively large definitive RCTs and also requires a pilot sample size of 60 to 90 for the true effect sizes considered here.

Conclusions
We recommend that an external pilot study has at least 70 measured subjects (35 per group) when estimating the SD p for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates.
1745-6215
1-13
Teare, M. Dawn
a1107526-ed90-40bf-b11c-010260bef7cb
Dimairo, Munyaradzi
a13f902b-b486-46f8-b7c4-3cdcfe2860eb
Shephard, Neil
b593f04f-79c0-4824-8fac-36538611194e
Hayman, Alex
02957d25-5262-49bc-a5e7-066c266a2300
Whitehead, Amy
7bd4e1d1-078b-4f2b-bfc9-ed44ba0a195a
Walters, Stephen J.
aed7a87c-2c83-4a28-a68f-54bfecb3fa53
Teare, M. Dawn
a1107526-ed90-40bf-b11c-010260bef7cb
Dimairo, Munyaradzi
a13f902b-b486-46f8-b7c4-3cdcfe2860eb
Shephard, Neil
b593f04f-79c0-4824-8fac-36538611194e
Hayman, Alex
02957d25-5262-49bc-a5e7-066c266a2300
Whitehead, Amy
7bd4e1d1-078b-4f2b-bfc9-ed44ba0a195a
Walters, Stephen J.
aed7a87c-2c83-4a28-a68f-54bfecb3fa53

Teare, M. Dawn, Dimairo, Munyaradzi, Shephard, Neil, Hayman, Alex, Whitehead, Amy and Walters, Stephen J. (2014) Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials, 15 (264), 1-13. (doi:10.1186/1745-6215-15-264).

Record type: Article

Abstract

Background
External pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.

Methods
We use a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes. We present the impact of increasing the pilot sample size on the precision and bias of these estimates, and predicted power under three realistic scenarios. We also illustrate the consequences of using a confidence interval argument to inflate estimates so the required power is achieved with a pre-specified level of confidence. We limit our attention to external pilot and feasibility studies prior to a two-parallel-balanced-group superiority RCT.

Results
For normally distributed outcomes, the relative gain in precision of the pooled standard deviation (SD p ) is less than 10% (for each five subjects added per group) once the total sample size is 70. For true proportions between 0.1 and 0.5, we find the gain in precision for each five subjects added to the pilot sample is less than 5% once the sample size is 60. Adjusting the required sample sizes for the imprecision in the pilot study estimates can result in excessively large definitive RCTs and also requires a pilot sample size of 60 to 90 for the true effect sizes considered here.

Conclusions
We recommend that an external pilot study has at least 70 measured subjects (35 per group) when estimating the SD p for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates.

Text
1745-6215-15-264 - Version of Record
Available under License Creative Commons Attribution.
Download (1MB)

More information

Published date: July 2014

Identifiers

Local EPrints ID: 421838
URI: https://eprints.soton.ac.uk/id/eprint/421838
ISSN: 1745-6215
PURE UUID: 395ec25d-2cdd-4bfa-9967-4e8dcef8348e

Catalogue record

Date deposited: 29 Jun 2018 16:30
Last modified: 18 Jul 2019 17:13

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×