Characterising the convergence of a stochastic simulation model using the bootstrap method

Abstract

Accepting that all models are wrong, whilst some are useful (G.E.P. Box), this paper will describe how the bootstrap technique has been applied to forecasting for Rolls-Royce, in order to develop a more robust decision support framework that can deliver on the promise of being more useful.
Practically speaking, the computational cost of large simulation studies may seem prohibitive, but large enough samples are often required to ensure both the accuracy and precision necessary for unambiguous experimentation. In order to appropriately prioritise opportunities, it is crucially important that an analyst can distinguish the sample differences that are due to experimental changes, rather than imprecise estimates.
To illustrate this issue, the bootstrap method has been applied to a complex model of engine maintenance operations to show a clear relationship between sample size, computational power, and precision. As example, all metric means in this case study were shown to have reached an acceptable level of precision within 1024 runs, or one hour of computation time, however, one hour per trial is considered impractically slow with respect to an experimental study requiring upwards of 350 trials. In showing that 25 of 198 tracked metric values require more than 1024 iterations to achieve acceptable levels of precision, this paper illustrates the need to trade off computation time against computation power, thus enabling the sorts of experimentation that usefully prioritise the exploitation of profitable opportunities.

More information

Identifiers

Catalogue record

Export record