Decoding mental images: combining pareidolia, genetic algorithms, and deep learning for objectively quantifying mental imagery

Abstract

Mental imagery—our ability to create or recreate perceptual experiences in our mind’s eye—is traditionally assessed using subjective self-report measures. To objectively measure mental images, we explored a novel approach using pareidolia, our tendency to perceive familiar patterns in noise. Specifically, we conducted a mind-reading experiment. Eighteen participants were first asked to choose a digit from 1-9, and then performed a pareidolia task. Participants were shown images of pure pixel noise and selected those most resembling their mental image of their chosen digit. Using a genetic search algorithm, we guided this selection process, progressively growing the digit image out of the noise. This created a ‘classification image’, a visual representation of the participant’s mental image of their chosen digit. We input the generated classification images to deep neural network (DNN) image classifiers trained to recognise digits. The classifiers correctly identified participants’ imagined digits 46% of the time, significantly above the 11% chance level (p<.001). Additionally, a new set of 20 participants correctly identified the digits from the classification images 66% of the time (p<.001). These results demonstrate that both humans and DNNs could decode the digits participants were thinking about. Our work thus provides a proof-of-concept demonstration of how we can visualize, decode, and objectively quantify mental images, and opens new avenues for exploring visual imagination. Future applications include fine tuning DNNs and using them as simulated observers to design stimulus sets to induce and further test pareidolia, enabling experiments measuring individual differences in mental imagery activation and frequency.

More information

Identifiers

ORCID for Saivydas Villani: orcid.org/0009-0001-5137-0744

ORCID for Guido Maiello: orcid.org/0000-0001-6625-2583

Catalogue record

Export record