Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs
Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs
In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of these stakeholders to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.
Energy studies, Sample size, Statistical power, Statistical significance, Study design
1-8
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b
1 January 2020
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b
Anderson, Ben, Rushby, Tom, Bahaj, Abubakr and James, Patrick
(2020)
Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs.
Energy Research & Social Science, 59, , [101260].
(doi:10.1016/j.erss.2019.101260).
Abstract
In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of these stakeholders to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.
Text
Working Paper
- Author's Original
Restricted to Registered users only
Request a copy
Text
Ensuring statistics have power
- Version of Record
More information
Accepted/In Press date: 15 August 2019
e-pub ahead of print date: 21 September 2019
Published date: 1 January 2020
Keywords:
Energy studies, Sample size, Statistical power, Statistical significance, Study design
Identifiers
Local EPrints ID: 434972
URI: http://eprints.soton.ac.uk/id/eprint/434972
ISSN: 2214-6296
PURE UUID: 42920b21-1491-4568-b40c-09fbc8c9853a
Catalogue record
Date deposited: 17 Oct 2019 16:30
Last modified: 17 Mar 2024 02:39
Export record
Altmetrics
Contributors
Author:
Tom Rushby
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics