A note on permutation tests
A note on permutation tests
Permutation tests were first introduced in Eden and Yates (1933), Fisher (1935) and Pitman (1937a, 1937b, 1938), and are popular nowadays due to several nice properties they possess and the cheap availability of computation power of modern computers. In this paper, we demonstrate potential power loss of permutation tests using the prototype permutation test for comparing two populations that may possibly be different only in locations. Specifically, we show that the reference distribution used for this permutation test depends on the true value of the unknown parameter that is being tested and this may reduce the power considerably in comparison with the standard parametric test especially for small sample sizes. The observations made for this particular permutation test in this paper is applicable to numerous other permutation tests and so users should be aware of this potential power loss of permutation tests for small sample sizes.
University of Southampton
Liu, Wei
b64150aa-d935-4209-804d-24c1b97e024a
Bretz, Frank
aada8c67-05d5-4a06-ad97-92a24c6f1d6a
Dette, Holger
8c7b1c2e-3adc-45df-acfc-9e76509a228e
2020
Liu, Wei
b64150aa-d935-4209-804d-24c1b97e024a
Bretz, Frank
aada8c67-05d5-4a06-ad97-92a24c6f1d6a
Dette, Holger
8c7b1c2e-3adc-45df-acfc-9e76509a228e
Liu, Wei, Bretz, Frank and Dette, Holger
(2020)
A note on permutation tests
University of Southampton
17pp.
Record type:
Monograph
(Working Paper)
Abstract
Permutation tests were first introduced in Eden and Yates (1933), Fisher (1935) and Pitman (1937a, 1937b, 1938), and are popular nowadays due to several nice properties they possess and the cheap availability of computation power of modern computers. In this paper, we demonstrate potential power loss of permutation tests using the prototype permutation test for comparing two populations that may possibly be different only in locations. Specifically, we show that the reference distribution used for this permutation test depends on the true value of the unknown parameter that is being tested and this may reduce the power considerably in comparison with the standard parametric test especially for small sample sizes. The observations made for this particular permutation test in this paper is applicable to numerous other permutation tests and so users should be aware of this potential power loss of permutation tests for small sample sizes.
Text
PURE-2020-4-29
- Author's Original
More information
Published date: 2020
Identifiers
Local EPrints ID: 439748
URI: http://eprints.soton.ac.uk/id/eprint/439748
PURE UUID: f5329763-f749-4592-adb6-3aba977eb5db
Catalogue record
Date deposited: 01 May 2020 16:36
Last modified: 17 Mar 2024 02:37
Export record
Contributors
Author:
Frank Bretz
Author:
Holger Dette
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics
