The University of Southampton
University of Southampton Institutional Repository

Targeting data collection in games based assessment

Targeting data collection in games based assessment
Targeting data collection in games based assessment
Educational game performance data has the potential to allow new types of complex, procedural skills to be assessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessment paradigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments in games. Choice and the iterative nature of games were found to allow curriculum specialisation. We found evidence that early attempts at new games are less
reliable and perhaps best discarded, and we propose a solution to weight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaborator ability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigated the use of response time as a proxy for ability. The physical measure of time proved difficult and potentially
unfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.
Walsh, Clare
3972b47c-5ce7-45fc-b843-7dcbde9504de
Bokhove, Christian
7fc17e5b-9a94-48f3-a387-2ccf60d2d5d8
Walsh, Clare
3972b47c-5ce7-45fc-b843-7dcbde9504de
Bokhove, Christian
7fc17e5b-9a94-48f3-a387-2ccf60d2d5d8

Walsh, Clare and Bokhove, Christian (2021) Targeting data collection in games based assessment. Computers & Education Open, 2, [100054]. (doi:10.1016/j.caeo.2021.100054).

Record type: Article

Abstract

Educational game performance data has the potential to allow new types of complex, procedural skills to be assessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessment paradigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments in games. Choice and the iterative nature of games were found to allow curriculum specialisation. We found evidence that early attempts at new games are less
reliable and perhaps best discarded, and we propose a solution to weight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaborator ability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigated the use of response time as a proxy for ability. The physical measure of time proved difficult and potentially
unfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.

Text
article_pure_accepted_141021 - Accepted Manuscript
Download (3MB)

More information

Accepted/In Press date: 14 October 2021
e-pub ahead of print date: 17 October 2021
Published date: 1 December 2021

Identifiers

Local EPrints ID: 451903
URI: http://eprints.soton.ac.uk/id/eprint/451903
PURE UUID: 29a6a2ac-4bec-4105-8cfb-2a12a8e4bade
ORCID for Clare Walsh: ORCID iD orcid.org/0000-0002-7757-2301
ORCID for Christian Bokhove: ORCID iD orcid.org/0000-0002-4860-8723

Catalogue record

Date deposited: 03 Nov 2021 17:30
Last modified: 17 Oct 2022 04:01

Export record

Altmetrics

Contributors

Author: Clare Walsh ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×