The University of Southampton
University of Southampton Institutional Repository

Non-response biases in surveys of school children: the case of the English PISA samples

Non-response biases in surveys of school children: the case of the English PISA samples
Non-response biases in surveys of school children: the case of the English PISA samples
We analyse response patterns to an important survey of school children, exploiting rich auxiliary information on respondents’ and non-respondents’ cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control data quality. We analyse the case of England for 2000 when response rates were deemed high enough by the PISA organisers to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of non-response using two methods, propensity scores and the GREG estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to data quality

non-response, bias, school survey, data linkage, PISA
0964-1998
1-41
Micklewright, John
744a4bca-41f2-4cbb-9a4e-3e0effdaa739
Schnepf, Sylke V.
c987c810-d33c-4675-9764-b5e15c581dbc
Skinner, Chris J.
dec5ef40-49ef-492a-8a1d-eb8c6315b8ce
Micklewright, John
744a4bca-41f2-4cbb-9a4e-3e0effdaa739
Schnepf, Sylke V.
c987c810-d33c-4675-9764-b5e15c581dbc
Skinner, Chris J.
dec5ef40-49ef-492a-8a1d-eb8c6315b8ce

Micklewright, John, Schnepf, Sylke V. and Skinner, Chris J. (2012) Non-response biases in surveys of school children: the case of the English PISA samples. Journal of the Royal Statistical Society: Series A (Statistics in Society), 1-41. (In Press)

Record type: Article

Abstract

We analyse response patterns to an important survey of school children, exploiting rich auxiliary information on respondents’ and non-respondents’ cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control data quality. We analyse the case of England for 2000 when response rates were deemed high enough by the PISA organisers to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of non-response using two methods, propensity scores and the GREG estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to data quality

Text
PISA paper_6 Sept 2011.doc - Accepted Manuscript
Download (490kB)
Text
s3ri-Workingpaper-A10-01.pdf - Other
Restricted to Registered users only
Download (450kB)
Request a copy

More information

Accepted/In Press date: January 2012
Keywords: non-response, bias, school survey, data linkage, PISA
Organisations: Social Statistics & Demography

Identifiers

Local EPrints ID: 79403
URI: http://eprints.soton.ac.uk/id/eprint/79403
ISSN: 0964-1998
PURE UUID: 11229318-bae2-4196-a798-075075e5f365

Catalogue record

Date deposited: 15 Mar 2010
Last modified: 14 Mar 2024 00:29

Export record

Contributors

Author: John Micklewright
Author: Sylke V. Schnepf
Author: Chris J. Skinner

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×