Grasping AI Reliance in Program Comprehension through the AIRELI Persona Taxonomy

Abstract

Artificial Intelligence (AI) assistance has become an integral part of software development, helping developers plan, explain, and generate code. As the boundary between human agency and AI reliance blurs, traditional measures of program comprehension, such as task success or completion time, increasingly capture AI effectiveness rather than the depth of human understanding. Without a better understanding of how developers rely on AI and how it replaces
human expertise, it is difficult to assess its short- and long-term effects on comprehension and capability. We conducted a controlled study with 21 participants working on two realistic change tasks, with or without AI assistance. Using quantitative and qualitative data from screen recordings, performance metrics, and questionnaires, we performed a thematic analysis and derived nine key characteristics that informed three AI reliance personas: self-sufficient, understanding-gated, and AI-steered developers. Analyzing participants through this persona lens revealed substantial differences in comprehension and capability that aggregate comparisons between AI and No AI conditions masked. Self-sufficient developers demonstrated deep understand-
ing, understanding-gated developers retained conceptual understanding but relied on AI for execution, and AI-steered developers completed tasks quickly yet without meaningful comprehension. These findings highlight the importance of accounting for AI reliance, as short-term AI-assisted productivity gains can mask a growing comprehension debt, where cognitive work is outsourced
to AI at the expense of human expertise and sustainable skill development.

More information

Identifiers

ORCID for Victoria Jackson: orcid.org/0000-0002-6326-931X

Catalogue record

Export record