The University of Southampton
University of Southampton Institutional Repository

Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs

Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs
Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs
In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of these stakeholders to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.
2214-6296
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Rushby, Tom
bdb7715f-0331-491c-a9dd-5835f30b0bf8
Bahaj, Abubakr
a64074cc-2b6e-43df-adac-a8437e7f1b37
James, Patrick
da0be14a-aa63-46a7-8646-a37f9a02a71b

Anderson, Ben, Rushby, Tom, Bahaj, Abubakr and James, Patrick (2020) Ensuring statistics have power: guidance for designing, reporting and acting on electricity demand reduction and behaviour change programs. Energy Research & Social Science, 59. (doi:10.1016/j.erss.2019.101260).

Record type: Article

Abstract

In this paper we address ongoing confusion over the meaning of statistical significance and statistical power in energy efficiency and energy demand reduction intervention studies. We discuss the role of these concepts in designing studies, in deciding what can be inferred from the results and thus what course of subsequent action to take. We do this using a worked example of a study of Heat Pump demand response in New Zealand to show how to appropriately size experimental and observational studies, the consequences this has for subsequent data analysis and the decisions that can then be taken. The paper then provides two sets of recommendations. The first focuses on the uncontroversial but seemingly ignorable issue of statistical power analysis and sample design, something regularly omitted in the energy studies literature. The second focuses on how to report energy demand reduction study or trial results, make inferences and take commercial or policy-oriented decisions in a contextually appropriate way. The paper therefore offers guidance to researchers tasked with designing and assessing such studies; project managers who need to understand what can count as evidence, for what purpose and in what context and decision makers who need to make defensible commercial or policy decisions based on that evidence. The paper therefore helps all of these stakeholders to distinguish the search for statistical significance from the requirement for actionable evidence and so avoid throwing the substantive baby out with the p-value bathwater.

Text
Working Paper - Author's Original
Restricted to Registered users only
Available under License Creative Commons Attribution.
Download (646kB)
Request a copy
Text
Ensuring statistics have power - Version of Record
Available under License Creative Commons Attribution.
Download (780kB)

More information

Accepted/In Press date: 15 August 2019
e-pub ahead of print date: 21 September 2019
Published date: 1 January 2020

Identifiers

Local EPrints ID: 434972
URI: https://eprints.soton.ac.uk/id/eprint/434972
ISSN: 2214-6296
PURE UUID: 42920b21-1491-4568-b40c-09fbc8c9853a
ORCID for Ben Anderson: ORCID iD orcid.org/0000-0003-2092-4406
ORCID for Tom Rushby: ORCID iD orcid.org/0000-0002-3686-5140
ORCID for Abubakr Bahaj: ORCID iD orcid.org/0000-0002-0043-6045
ORCID for Patrick James: ORCID iD orcid.org/0000-0002-2694-7054

Catalogue record

Date deposited: 17 Oct 2019 16:30
Last modified: 12 Nov 2019 02:01

Export record

Altmetrics

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×