Increasing response rates and improving research design: Learnings from the Smart Energy Research Lab in the United Kingdom
Increasing response rates and improving research design: Learnings from the Smart Energy Research Lab in the United Kingdom
Obtaining high-resolution energy consumption data from a large, representative sample of homes is critical for research, but low response rates, sample bias and high recruitment costs form substantial barriers. The widespread installation of smart meters offers a novel route to access such data, but in countries like Great Britain (GB) consent is required from each household; a real barrier to large-scale sampling. In this paper we show how certain study design choices can impact the response rate for energy studies requesting access to half-hourly smart meter data and (optional) survey completion. We used a randomised control trial (RCT) with a 3×2×2 factorial design; 3 (including none) incentive groups ×2 message content/structures ×2 ‘push-to-web’ treatment groups. Up to 4 mailings (letters) were sent to 18,000 addresses, recruiting 1711 participants (9.5% response rate) in England and Wales. The most effective strategy offered a conditional £5 voucher and postal response options in multiple mailings (compared to only once in the push-to-web approach, although at the expense of far fewer online signups). Motivational headlines and message structure were also found to be influential. Reminders increased response but a 4th mailing was not cost effective. Our results and recommendations can be used to help future energy studies to achieve greater response rates and improved representation. UK-based researchers can apply to use our longitudinal smart meter and contextual datasets.
Energy survey, Incentive, Push-to-web, Randomized control trial, Recruitment, Reminder, Response rate, Smart meter, Web push
Webborn, Ellen
f210ccbe-36c5-4b88-9651-43246eba8157
McKenna, Eoghan
6b4322f5-c975-4a03-a332-3d199df865c3
Elam, Simon
28497ead-5c4d-45a4-b0ce-79aea15cb5cf
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Cooper, Adam
bc52a8bc-eb6d-410c-990b-a705fee6322d
Oreszczyn, Tadj
cd7ee772-c397-41af-877b-74d2f2e795d9
January 2022
Webborn, Ellen
f210ccbe-36c5-4b88-9651-43246eba8157
McKenna, Eoghan
6b4322f5-c975-4a03-a332-3d199df865c3
Elam, Simon
28497ead-5c4d-45a4-b0ce-79aea15cb5cf
Anderson, Ben
01e98bbd-b402-48b0-b83e-142341a39b2d
Cooper, Adam
bc52a8bc-eb6d-410c-990b-a705fee6322d
Oreszczyn, Tadj
cd7ee772-c397-41af-877b-74d2f2e795d9
Webborn, Ellen, McKenna, Eoghan, Elam, Simon, Anderson, Ben, Cooper, Adam and Oreszczyn, Tadj
(2022)
Increasing response rates and improving research design: Learnings from the Smart Energy Research Lab in the United Kingdom.
Energy Research & Social Science, 83, [102312].
(doi:10.31219/osf.io/f82b7).
Abstract
Obtaining high-resolution energy consumption data from a large, representative sample of homes is critical for research, but low response rates, sample bias and high recruitment costs form substantial barriers. The widespread installation of smart meters offers a novel route to access such data, but in countries like Great Britain (GB) consent is required from each household; a real barrier to large-scale sampling. In this paper we show how certain study design choices can impact the response rate for energy studies requesting access to half-hourly smart meter data and (optional) survey completion. We used a randomised control trial (RCT) with a 3×2×2 factorial design; 3 (including none) incentive groups ×2 message content/structures ×2 ‘push-to-web’ treatment groups. Up to 4 mailings (letters) were sent to 18,000 addresses, recruiting 1711 participants (9.5% response rate) in England and Wales. The most effective strategy offered a conditional £5 voucher and postal response options in multiple mailings (compared to only once in the push-to-web approach, although at the expense of far fewer online signups). Motivational headlines and message structure were also found to be influential. Reminders increased response but a 4th mailing was not cost effective. Our results and recommendations can be used to help future energy studies to achieve greater response rates and improved representation. UK-based researchers can apply to use our longitudinal smart meter and contextual datasets.
Text
Webborn_et_al_Increasing_Rersponse_Rates_ePrints
- Author's Original
Restricted to Registered users only
Request a copy
More information
Submitted date: 18 May 2021
Accepted/In Press date: 9 September 2021
Published date: January 2022
Additional Information:
Funding Information:
This work has been funded by EPSRC through grant number EP/P032761/1. There are over 30 individuals across 8 organisations in the SERL Consortium (University College London, the University of Essex (UK Data Archive), the University of Edinburgh, Cardiff University, Loughborough University, Leeds Beckett University, the University of Southampton and the Energy Saving Trust) who have contributed to the development of SERL and thus the content of this paper. Particular thanks go to the SERL technical team at the UK Data Archive: Darren Bell, Deirdre Lungley, Martin Randall and Jacob Joy and to James O'Toole at UCL. Ipsos MORI contributed to the research design and mailing development and carried out the fieldwork.
Funding Information:
This work has been funded by EPSRC through grant number EP/P032761/1 . There are over 30 individuals across 8 organisations in the SERL Consortium (University College London, the University of Essex (UK Data Archive), the University of Edinburgh, Cardiff University, Loughborough University, Leeds Beckett University, the University of Southampton and the Energy Saving Trust) who have contributed to the development of SERL and thus the content of this paper. Particular thanks go to the SERL technical team at the UK Data Archive: Darren Bell, Deirdre Lungley, Martin Randall and Jacob Joy and to James O’Toole at UCL. Ipsos MORI contributed to the research design and mailing development and carried out the fieldwork.
Publisher Copyright:
© 2021 Elsevier Ltd
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
Keywords:
Energy survey, Incentive, Push-to-web, Randomized control trial, Recruitment, Reminder, Response rate, Smart meter, Web push
Identifiers
Local EPrints ID: 449347
URI: http://eprints.soton.ac.uk/id/eprint/449347
ISSN: 2214-6296
PURE UUID: 1edc84b9-dcf7-4cfc-a36a-748e5af184e4
Catalogue record
Date deposited: 25 May 2021 16:35
Last modified: 17 Mar 2024 12:46
Export record
Altmetrics
Contributors
Author:
Ellen Webborn
Author:
Eoghan McKenna
Author:
Simon Elam
Author:
Adam Cooper
Author:
Tadj Oreszczyn
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics
