The University of Southampton
University of Southampton Institutional Repository

Do coefficients of variation of response propensities approximate non-response biases during survey data collection?

Do coefficients of variation of response propensities approximate non-response biases during survey data collection?
Do coefficients of variation of response propensities approximate non-response biases during survey data collection?

We evaluate the utility of coefficients of variation of response propensities (CVs) as measures of risks of survey variable non-response biases when monitoring survey data collection. CVs quantify variation in sample response propensities estimated given a set of auxiliary attribute covariates observed for all subjects. If auxiliary covariates and survey variables are correlated, low levels of propensity variation imply low bias risk. CVs can also be decomposed to measure associations between auxiliary covariates and propensity variation, informing collection method modifications and post-collection adjustments to improve dataset quality. Practitioners are interested in such approaches to managing bias risks, but risk indicator performance has received little attention. We describe relationships between CVs and expected biases and how they inform quality improvements during and post-data collection, expanding on previous work. Next, given auxiliary information from the concurrent 2011 UK census and details of interview attempts, we use CVs to quantify the representativeness of the UK Labour Force Survey dataset during data collection. Following this, we use survey data to evaluate inference based on CVs concerning survey variables with analogues measuring the same quantities among the auxiliary covariate set. Given our findings, we then offer advice on using CVs to monitor survey data collection.

adaptive survey designs, data collection efficiency savings, non-response bias, phase capacity, representativeness indicators
0964-1998
301-323
Moore, Jamie C.
5f015c47-3165-4f64-8561-7c047a9d2186
Durrant, Gabriele B.
14fcc787-2666-46f2-a097-e4b98a210610
Smith, Peter W.F.
961a01a3-bf4c-43ca-9599-5be4fd5d3940
Moore, Jamie C.
5f015c47-3165-4f64-8561-7c047a9d2186
Durrant, Gabriele B.
14fcc787-2666-46f2-a097-e4b98a210610
Smith, Peter W.F.
961a01a3-bf4c-43ca-9599-5be4fd5d3940

Moore, Jamie C., Durrant, Gabriele B. and Smith, Peter W.F. (2021) Do coefficients of variation of response propensities approximate non-response biases during survey data collection? Journal of the Royal Statistical Society. Series A: Statistics in Society, 184 (1), 301-323. (doi:10.1111/rssa.12624).

Record type: Article

Abstract

We evaluate the utility of coefficients of variation of response propensities (CVs) as measures of risks of survey variable non-response biases when monitoring survey data collection. CVs quantify variation in sample response propensities estimated given a set of auxiliary attribute covariates observed for all subjects. If auxiliary covariates and survey variables are correlated, low levels of propensity variation imply low bias risk. CVs can also be decomposed to measure associations between auxiliary covariates and propensity variation, informing collection method modifications and post-collection adjustments to improve dataset quality. Practitioners are interested in such approaches to managing bias risks, but risk indicator performance has received little attention. We describe relationships between CVs and expected biases and how they inform quality improvements during and post-data collection, expanding on previous work. Next, given auxiliary information from the concurrent 2011 UK census and details of interview attempts, we use CVs to quantify the representativeness of the UK Labour Force Survey dataset during data collection. Following this, we use survey data to evaluate inference based on CVs concerning survey variables with analogues measuring the same quantities among the auxiliary covariate set. Given our findings, we then offer advice on using CVs to monitor survey data collection.

Text
RSSsubmission_JRSS-SA-Sep-17-0142R2main_paper_revision - Accepted Manuscript
Download (659kB)

More information

Accepted/In Press date: 21 June 2020
e-pub ahead of print date: 3 November 2020
Published date: January 2021
Additional Information: Funding Information: This research was funded by the ESRC National Centre for Research Methods, Workpackage 1 (grant reference number ES/L008351/1) and the ESRC Administrative Research Centre for England (ADRCE) (grant reference number ES/L007517/1). This work contains statistical data from ONS which is Crown Copyright. The use of the ONS statistical data in this work does not imply the endorsement of the ONS in relation to the interpretation or analysis of the statistical data. This work uses research datasets which may not exactly reproduce National Statistics aggregates. Publisher Copyright: © 2020 The Authors. Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
Keywords: adaptive survey designs, data collection efficiency savings, non-response bias, phase capacity, representativeness indicators

Identifiers

Local EPrints ID: 447409
URI: http://eprints.soton.ac.uk/id/eprint/447409
ISSN: 0964-1998
PURE UUID: b680cd30-021f-4da8-9d7a-9f36bec453c8
ORCID for Gabriele B. Durrant: ORCID iD orcid.org/0009-0001-3436-1512
ORCID for Peter W.F. Smith: ORCID iD orcid.org/0000-0003-4423-5410

Catalogue record

Date deposited: 10 Mar 2021 17:44
Last modified: 18 May 2024 04:01

Export record

Altmetrics

Contributors

Author: Jamie C. Moore

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×