Survey response rates: trends and a validity assessment framework
Survey response rates: trends and a validity assessment framework
Survey methodology has been and continues to be a pervasively used data-collection method in social science research. To better understand the state of the science, we first analyze response-rate information reported in 1014 surveys described in 703 articles from 17 journals from 2010 to 2020. Results showed a steady increase in average response rate from 48% in 2005 to 53% in 2010 to 56% in 2015 and 68% in 2020; a marked increase in the number of surveys per published article from 1.27 in 2015 to 1.79 in 2020; and that variables that predict response-rate fluctuations over time are related to research design (e.g. data-collection medium), participant motivation (e.g. incentives), and researcher motivation (i.e. number of surveys per article). Second, we propose complementary information on contemporary response-rate norms and benchmarks with a response-rate validity assessment framework to gather evidence on accuracy of inferences based on a particular response-rate level. Implementing this validation process involves gathering information on the researcher–participant relationship, participant qualifications and motivation, survey length and complexity, and cultural and national context. Future survey research should implement the validity assessment framework in addition to reporting the response-rate value to better indicate a sample’s quality, appropriateness, and representativeness.
generalizability, methodology, response rate, survey, validity
1560-1584
Holtom, Brooks
fa3c1005-0555-433f-9414-66cf2c4c361d
Baruch, Yehuda
25b89777-def4-4958-afdc-0ceab43efe8a
Aguinis, Herman
ea2d90de-45d8-4db0-b932-5b93d0f3fd5c
August 2022
Holtom, Brooks
fa3c1005-0555-433f-9414-66cf2c4c361d
Baruch, Yehuda
25b89777-def4-4958-afdc-0ceab43efe8a
Aguinis, Herman
ea2d90de-45d8-4db0-b932-5b93d0f3fd5c
Holtom, Brooks, Baruch, Yehuda and Aguinis, Herman
(2022)
Survey response rates: trends and a validity assessment framework.
Human Relations, 75 (8), .
(doi:10.1177/00187267211070769).
Abstract
Survey methodology has been and continues to be a pervasively used data-collection method in social science research. To better understand the state of the science, we first analyze response-rate information reported in 1014 surveys described in 703 articles from 17 journals from 2010 to 2020. Results showed a steady increase in average response rate from 48% in 2005 to 53% in 2010 to 56% in 2015 and 68% in 2020; a marked increase in the number of surveys per published article from 1.27 in 2015 to 1.79 in 2020; and that variables that predict response-rate fluctuations over time are related to research design (e.g. data-collection medium), participant motivation (e.g. incentives), and researcher motivation (i.e. number of surveys per article). Second, we propose complementary information on contemporary response-rate norms and benchmarks with a response-rate validity assessment framework to gather evidence on accuracy of inferences based on a particular response-rate level. Implementing this validation process involves gathering information on the researcher–participant relationship, participant qualifications and motivation, survey length and complexity, and cultural and national context. Future survey research should implement the validity assessment framework in addition to reporting the response-rate value to better indicate a sample’s quality, appropriateness, and representativeness.
Text
Hum Rel Response rate as accepted_
- Accepted Manuscript
Text
00187267211070769
- Version of Record
More information
Accepted/In Press date: 25 November 2021
e-pub ahead of print date: 1 February 2022
Published date: August 2022
Additional Information:
Funding Information:
The authors sincerely appreciate the excellent research assistance rendered by Scott Kelly, Hannah Evans, and Jayme Cloninger, who are students at Georgetown University. We also thank the Editor and the anonymous reviewers for their constructive support. The authors received no financial support for the research, authorship, and/or publication of this article.
Publisher Copyright:
© The Author(s) 2022.
Keywords:
generalizability, methodology, response rate, survey, validity
Identifiers
Local EPrints ID: 454322
URI: http://eprints.soton.ac.uk/id/eprint/454322
ISSN: 0018-7267
PURE UUID: f862b588-d826-4c3a-8ec5-99c09fa54fb4
Catalogue record
Date deposited: 07 Feb 2022 17:42
Last modified: 06 Jun 2024 01:52
Export record
Altmetrics
Contributors
Author:
Brooks Holtom
Author:
Herman Aguinis
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics
