A note on the use of multiple comparison scenario techniques in education and practice
A note on the use of multiple comparison scenario techniques in education and practice
Our main aim in this paper is to highlight current practice and education in multiple scenario comparison within DES experimentation and to illustrate the possible benefits of employing false discovery rate (FDR) control as opposed to strict family-wise error rate (FWER) control when comparing large numbers of DES experimentation scenarios in an exploratory manner. We present the results of a small survey into the current practice of scenario analysis by simulation practitioners and academics. Although the survey was small, the results indicated that the range of scenarios used in DES studies may prohibit the use of FWER control methods such as the Bonferroni Correction referred to in DES textbooks. Furthermore, 80% of our sample were not familiar with multiple comparison control. We provide a practical example of the FDR in action and argue that it is preferable to employ FDR instead of no multiple comparison control in exploratory style studies
978-1-4577-2108-3
3899-3909
Hoad, Kathryn A.
86cad5e0-46c9-4ec3-bd74-859270234241
Monks, Thomas
fece343c-106d-461d-a1dd-71c1772627ca
December 2011
Hoad, Kathryn A.
86cad5e0-46c9-4ec3-bd74-859270234241
Monks, Thomas
fece343c-106d-461d-a1dd-71c1772627ca
Hoad, Kathryn A. and Monks, Thomas
(2011)
A note on the use of multiple comparison scenario techniques in education and practice.
Winter Simulation Conference 2011, Phoenix, United States.
11 - 14 Dec 2011.
.
(doi:10.1109/WSC.2011.6148081).
Record type:
Conference or Workshop Item
(Paper)
Abstract
Our main aim in this paper is to highlight current practice and education in multiple scenario comparison within DES experimentation and to illustrate the possible benefits of employing false discovery rate (FDR) control as opposed to strict family-wise error rate (FWER) control when comparing large numbers of DES experimentation scenarios in an exploratory manner. We present the results of a small survey into the current practice of scenario analysis by simulation practitioners and academics. Although the survey was small, the results indicated that the range of scenarios used in DES studies may prohibit the use of FWER control methods such as the Bonferroni Correction referred to in DES textbooks. Furthermore, 80% of our sample were not familiar with multiple comparison control. We provide a practical example of the FDR in action and argue that it is preferable to employ FDR instead of no multiple comparison control in exploratory style studies
Other
HOADMONKS_WSC2011.PDF
- Author's Original
More information
Published date: December 2011
Venue - Dates:
Winter Simulation Conference 2011, Phoenix, United States, 2011-12-11 - 2011-12-14
Organisations:
Faculty of Health Sciences
Identifiers
Local EPrints ID: 367147
URI: http://eprints.soton.ac.uk/id/eprint/367147
ISBN: 978-1-4577-2108-3
PURE UUID: 6e7ba045-3374-42c0-a732-88bb9df00fce
Catalogue record
Date deposited: 24 Jul 2014 13:33
Last modified: 14 Mar 2024 17:24
Export record
Altmetrics
Contributors
Author:
Kathryn A. Hoad
Author:
Thomas Monks
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics