Cross-cultural differential item functioning in the PISA 2022 creative thinking assessment

Abstract

Conceptions of creativity differ systematically across cultures, shaping how they are expressed and valued. The introduction of a Creative Thinking (CT) domain in PISA 2022 marks the first large-scale, internationally comparable assessment of this competence among 15-year-olds. However, the availability of this rich cross-national data does not resolve the fundamental validity question of whether the creative thinking tasks function equivalently across diverse cultural contexts at the item level. This study investigates the extent, patterns, and mechanisms of cultural differences in the PISA 2022 CT assessment. We employ a sequential explanatory mixed-methods approach, pairing an IRT-based Differential Functioning of Items and Tests (DFIT) framework with qualitative item diagnosis. DFIT analyses of 30 PISA 2022 CT items across 10 Global Leadership and Organizational Behavior Effectiveness (GLOBE) cultural clusters reveal that while cultural non-invariance is pervasive (flagging 83.3% of items), the severity of bias is uneven. At the cluster level, bias does not suggest an “East–West” divide but distinct regional assessment profiles. To explain these quantitative patterns, a subsequent qualitative diagnosis of the most severely biased items was conducted. This analysis located a four-dimensional bias architecture that may contribute to the observed disadvantage: (1) context universality and familiarity, (2) the cultural match of scenario-activated cognitive prototypes, (3) implicit value assumptions, and (4) response-mode alignment. These findings call for a shift in the field from reactive, post-hoc bias screening to a proactive approach of embedding fairness considerations into the design process, aimed at creating culturally inclusive assessments.

More information

Identifiers

Catalogue record

Export record

ASCII CitationAtomBibTeXData Cite XMLDublin CoreDublin CoreEP3 XMLEndNoteHTML CitationHTML CitationHTML ListJSONMETSMODSMPEG-21 DIDLOpenURL ContextObjectOpenURL ContextObject in SpanRDF+N-TriplesRDF+N3RDF+XMLRIOXX2 XMLReferReference ManagerSimple Metadata

Altmetrics