The University of Southampton
University of Southampton Institutional Repository

A little bias goes a long way: the effects of feedback on the strategic regulation of accuracy on formula-scored tests

A little bias goes a long way: the effects of feedback on the strategic regulation of accuracy on formula-scored tests
A little bias goes a long way: the effects of feedback on the strategic regulation of accuracy on formula-scored tests
Under formula-scoring rules for multiple-choice exams, a penalty is applied to incorrect responses to reduce noise in the observed score. To avoid the penalty individuals are allowed to “pass,” and therefore they must be able to strategically regulate the accuracy of their reporting by deciding which and how many questions to answer. To investigate the effect of bias within this framework, Higham (2007) introduced bias profiles, which show the score obtained under formula scoring (corrected score) as a function of the omission rate. Bias profiles estimate the optimal number of questions that should be answered to maximize the corrected score (i.e., optimal bias). Our initial research showed that individuals tend to be too conservative when setting reporting criteria, “omitting” too many answers. The present three experiments introduced a feedback manipulation whereby participants were informed of the optimal omission rate after completing a test and asked to alter their reporting decisions accordingly. This feedback and concomitant alteration of reporting decisions led to improved corrected scores on true/false (Experiment 1), two-alternative tests (Experiments 2), and four-alternative tests (Experiment 3). Importantly, corrected scores at optimal bias also were higher than at forced-report for both true/false and two-alternative tests. Furthermore, in Experiment 3, feedback based on one test improved scores on a second test, and participants were more likely to perform optimally on a third test without feedback. These effects suggest that optimal-bias feedback may have long-term effects and generalize to new tests.
1076-898X
383-402
Arnold, M.M.
36761703-6e77-4a91-9207-4f70b719bed2
Higham, P.A.
4093b28f-7d58-4d18-89d4-021792e418e7
Martin-Luengo, B.
ffb1583e-d9fd-4c84-a565-75eca34cbc4f
Arnold, M.M.
36761703-6e77-4a91-9207-4f70b719bed2
Higham, P.A.
4093b28f-7d58-4d18-89d4-021792e418e7
Martin-Luengo, B.
ffb1583e-d9fd-4c84-a565-75eca34cbc4f

Arnold, M.M., Higham, P.A. and Martin-Luengo, B. (2013) A little bias goes a long way: the effects of feedback on the strategic regulation of accuracy on formula-scored tests. Journal of Experimental Psychology: Applied, 19, 383-402.

Record type: Article

Abstract

Under formula-scoring rules for multiple-choice exams, a penalty is applied to incorrect responses to reduce noise in the observed score. To avoid the penalty individuals are allowed to “pass,” and therefore they must be able to strategically regulate the accuracy of their reporting by deciding which and how many questions to answer. To investigate the effect of bias within this framework, Higham (2007) introduced bias profiles, which show the score obtained under formula scoring (corrected score) as a function of the omission rate. Bias profiles estimate the optimal number of questions that should be answered to maximize the corrected score (i.e., optimal bias). Our initial research showed that individuals tend to be too conservative when setting reporting criteria, “omitting” too many answers. The present three experiments introduced a feedback manipulation whereby participants were informed of the optimal omission rate after completing a test and asked to alter their reporting decisions accordingly. This feedback and concomitant alteration of reporting decisions led to improved corrected scores on true/false (Experiment 1), two-alternative tests (Experiments 2), and four-alternative tests (Experiment 3). Importantly, corrected scores at optimal bias also were higher than at forced-report for both true/false and two-alternative tests. Furthermore, in Experiment 3, feedback based on one test improved scores on a second test, and participants were more likely to perform optimally on a third test without feedback. These effects suggest that optimal-bias feedback may have long-term effects and generalize to new tests.

This record has no associated files available for download.

More information

Published date: 2013

Identifiers

Local EPrints ID: 356647
URI: http://eprints.soton.ac.uk/id/eprint/356647
ISSN: 1076-898X
PURE UUID: 3c424111-d241-4f49-9c07-b15ed47c6c58
ORCID for P.A. Higham: ORCID iD orcid.org/0000-0001-6087-7224

Catalogue record

Date deposited: 27 Sep 2013 13:40
Last modified: 28 Apr 2022 01:47

Export record

Contributors

Author: M.M. Arnold
Author: P.A. Higham ORCID iD
Author: B. Martin-Luengo

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×