Predictability modulates neurocognitive semantic processing of non-verbal narratives

Abstract

Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities. Here we ask whether similar cognitive processes are at play in predicting upcoming events in a non-verbal visual narrative. Typically developing adults viewed comics sequences in which a target panel was highly predictable (“high cloze”), less predictable (“low cloze”), or incongruent with the preceding narrative context (“anomalous”) during EEG recording. High and low predictable sequences were determined by a pretest where participants assessed “what happened next?”, resulting in cloze probability scores for sequence outcomes comparable to those used to measure predictability in sentence processing. Through both factorial and correlational analyses, we show a significant modulation of neural responses by cloze such that N400 effects are diminished as a target panel in a comic sequence becomes more predictable. Predictability thus appears to play a similar role in non-verbal comprehension of sequential images as in language comprehension, providing further evidence for the domain generality of semantic processing in the brain.