The University of Southampton
University of Southampton Institutional Repository

We got the power: Clarifying the role of statistical power and statistical significance in study design, inference and decision making in energy demand reduction studies

We got the power: Clarifying the role of statistical power and statistical significance in study design, inference and decision making in energy demand reduction studies
We got the power: Clarifying the role of statistical power and statistical significance in study design, inference and decision making in energy demand reduction studies
In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of them to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.
Universty of Southampton
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b

Anderson, Ben, Rushby, Tom, Bahaj, Abubakr and James, Patrick (2019) We got the power: Clarifying the role of statistical power and statistical significance in study design, inference and decision making in energy demand reduction studies Southampton. Universty of Southampton 21pp.

Record type: Monograph (Working Paper)

Abstract

In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of them to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.

Text
Working Paper - Author's Original
Available under License Creative Commons Attribution.
Download (646kB)

More information

Published date: 2019

Identifiers

Local EPrints ID: 428946
URI: https://eprints.soton.ac.uk/id/eprint/428946
PURE UUID: 7c8e7e48-83cd-4565-8bb1-5c5d4f48bed7
ORCID for Ben Anderson: ORCID iD orcid.org/0000-0003-2092-4406
ORCID for Tom Rushby: ORCID iD orcid.org/0000-0002-3686-5140
ORCID for Abubakr Bahaj: ORCID iD orcid.org/0000-0002-0043-6045

Catalogue record

Date deposited: 15 Mar 2019 17:30
Last modified: 16 Mar 2019 01:39

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×