Higham, P.A. and Arnold, M.M.
Beyond reliability and validity: The role of metacognition in psychological testing
DeGregorio, R.A. (eds.)
New Developments in Psychological Testing.
Full text not available from this repository.
Much research on psychological testing in educational contexts has focused on issues to do with reliability and validity. An excellent example is the debate over whether or not to penalize errors on multiple-choice tests, so called formula scoring. The case in support of formula scoring has typically focused on the idea that it may improve the reliability and validity of the test by removing error variance from the observed score. However, because students writing formula-scored tests are given the opportunity to "pass" on questions for which the answer is not known, opponents of formula scoring have argued that it contaminates the test score by introducing strategic factors. For example, conservative or risk-averse students may penalize themselves by answering too few questions on the test. A key factor that has been missing from this debate is the role of metacognitive monitoring, that is, the extent to which students can assess the accuracy of their own answers. Students with good metacognitive monitoring are at an advantage relative to students with poor monitoring because they know better which answers to offer (correct ones) and which to omit (incorrect ones). This parameter contaminates the corrected test score and varies between individuals just as aptitude or knowledge does, yet commonly used methods of scoring have no way of estimating its influence. In this chapter, we outline a signal-detection model that allows this metacognitive parameter to be estimated separately from other test parameters, review research on its influence, and make recommendations to test designers as to how they can obtain purer measures of the different aspects of test performance
Actions (login required)