Assessing ChatGPT as a tool for research on US state and territory politics

Abstract

Debates about the moral and ethical implications of using Artificial Intelligence (AI) abound. Yet, AI is swiftly being embedded into many aspects of society, becoming a key tool for sharing and gaining new knowledge. Benefiting from a systematic methodology that has led to the generation of profiles of all US states and territories in ChatGPT, as well as interviews with 17 experts on state-level politics, this paper provides insights for those wishing to understand the value of tools such as ChatGPT for political research generally, and for state and territory level US politics specifically. It demonstrates that generative AI cannot yet produce robust politically-oriented content. Scores given by experts for different aspects of the ChatGPT-generated profiles suggest it is somewhat better suited to accurately capturing the history of states as opposed to contemporary politics or insights that can be garnered from academic literature. The findings further highlight the often vague nature of sources provided by ChatGPT, and detail other inaccuracies in sourcing and content. This paper demonstrates that at present, ChatGPT cannot serve as a meaningful resource for students and scholars of US state and territorial politics and that these findings are likely robust for other types of political research.

More information

Identifiers

ORCID for Caroline Victoria Leicht: orcid.org/0000-0001-9822-857X

Catalogue record

Export record