Computer vision face detectors predict the strength of human face pareidolia

Abstract

The strength of face pareidolia—our tendency to perceive faces in non-face images— varies across individuals as well as across pareidolic stimuli. To explain why some people experience pareidolic faces more readily than others, we must first understand what makes some images more face-like than others. This requires quantifying the signal which makes a stimulus 'face-like' from its visual features. We thus turned to computational face-detection models (Hamilton et al., ECCV 377-395). We fine-tuned RetinaFace models with MobileNet and ResNet-50 backbones on the Faces in Things dataset (3217/1320 training/test pareidolic images), which includes human annotations indicating how Easy/Medium/Hard it is to spot the pareidolic faces. Pre-trained models showed low recall on the held-out test set (28.4% and 6.4%), but fine-tuning improved performance (76.2% and 70.2%), with the largest gains for Hard stimuli across both architectures. In addition to detecting faces, these models output “confidence” scores that indicate how certain a model is that it has detected a face. Crucially, model confidence decreased monotonically with human-rated difficulty (p<.001 both models), with Easy pareidolic stimuli detected with higher confidence than Hard ones. These findings suggest that pareidolia-tuned detectors may capture the same image cues that drive human judgments, potentially enabling us to isolate the specific visual features that elicit pareidolia. Further, we plan to leverage model confidence to predict which images are most likely to elicit pareidolia, to chart a graded ‘faceness’ continuum, and to derive objective, image-computable indices that explain individual differences in the strength and tendency to experience face pareidolia.

More information

Identifiers

ORCID for Saivydas Villani: orcid.org/0009-0001-5137-0744

ORCID for Guido Maiello: orcid.org/0000-0001-6625-2583

Catalogue record

Export record