Non-response biases in surveys of school children: the case of the English PISA samples
Non-response biases in surveys of school children: the case of the English PISA samples
We analyse response patterns to an important survey of school children, exploiting rich auxiliary information on respondents’ and non-respondents’ cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control data quality. We analyse the case of England for 2000 when response rates were deemed high enough by the PISA organisers to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of non-response using two methods, propensity scores and the GREG estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to data quality
non-response, bias, school survey, data linkage, PISA
1-41
Micklewright, John
744a4bca-41f2-4cbb-9a4e-3e0effdaa739
Schnepf, Sylke V.
c987c810-d33c-4675-9764-b5e15c581dbc
Skinner, Chris J.
dec5ef40-49ef-492a-8a1d-eb8c6315b8ce
Micklewright, John
744a4bca-41f2-4cbb-9a4e-3e0effdaa739
Schnepf, Sylke V.
c987c810-d33c-4675-9764-b5e15c581dbc
Skinner, Chris J.
dec5ef40-49ef-492a-8a1d-eb8c6315b8ce
Micklewright, John, Schnepf, Sylke V. and Skinner, Chris J.
(2012)
Non-response biases in surveys of school children: the case of the English PISA samples.
Journal of the Royal Statistical Society: Series A (Statistics in Society), .
(In Press)
Abstract
We analyse response patterns to an important survey of school children, exploiting rich auxiliary information on respondents’ and non-respondents’ cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control data quality. We analyse the case of England for 2000 when response rates were deemed high enough by the PISA organisers to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of non-response using two methods, propensity scores and the GREG estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to data quality
Text
PISA paper_6 Sept 2011.doc
- Accepted Manuscript
Text
s3ri-Workingpaper-A10-01.pdf
- Other
Restricted to Registered users only
Request a copy
More information
Accepted/In Press date: January 2012
Keywords:
non-response, bias, school survey, data linkage, PISA
Organisations:
Social Statistics & Demography
Identifiers
Local EPrints ID: 79403
URI: http://eprints.soton.ac.uk/id/eprint/79403
ISSN: 0964-1998
PURE UUID: 11229318-bae2-4196-a798-075075e5f365
Catalogue record
Date deposited: 15 Mar 2010
Last modified: 14 Mar 2024 00:29
Export record
Contributors
Author:
John Micklewright
Author:
Sylke V. Schnepf
Author:
Chris J. Skinner
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics