Identifying social group constructions in political online discussions: ambiguity, subjectivity, and label variation

Abstract

Since political discourse often revolves around determining which social groups are deemed deserving of resources, political social media discussions often reference them. Thereby, “social group” is a broad concept referring to collections of individuals who are categorized based on diverse criteria or attributions. Thus, social group mentions in text are prime examples of complex and ambiguous social concepts that have only recently garnered attention in natural language processing research, particularly within the emerging field of perspectivism. Following this line of research, we propose that disagreements between annotators provide important insights into what constitutes a social group. By analyzing diverging annotation judgments, we develop a taxonomy of label variation that distinguishes between ambiguities in identifying social group mentions and genuine annotation errors. We further investigate whether specific types of social group mentions are related to distinct types of label variation. Our findings reveal that linguistic and interpretative ambiguities represent a majority of variation when it comes to subjective and ambiguous concepts, underscoring the complex and multifaceted nature of social group mentions. This observation aligns with the perspectivist view, which posits that annotators’ diverse perspectives can enrich labeled data by capturing multiple valid interpretations rather than forcing a single “correct” ground truth. To explore this, we present Reddit-Social Group Mentions (Reddit-SGM), a novel, non-aggregated dataset for studying social group mentions in texts. By embracing the perspectivist approach, our dataset lays the groundwork for the automatic identification of social group mentions, enabling a nuanced representation of this inherently ambiguous concept.

More information

Identifiers

ORCID for Steffen Staab: orcid.org/0000-0002-0780-4154

Catalogue record

Export record

Altmetrics