PPubMed1 Apr 2024Cited 12×

Cognitive Computational Neuroscience of Language: Using Computational Models to Investigate Language Processing in the Brain

Alessandro Lopopolo, Evelina Fedorenko, Roger Levy, Milena Rabovsky

Abstract

The articles in this special issue introduce or leverage insights from novel or well-established language-processing models. One distinctive dimension corresponds to the motivation behind the employed models. Some studies utilize cognitively motivated architectures explicitly designed to mimic aspects of human language processing based on pre-existing theories. In contrast, other studies leverage large language models (LLMs) developed by engineers for NLP applications. These LLMs are employed to obtain either rich high-dimensional representations or measures of performance on some language tas

§ The Valyu brief

The paper, in five minutes

Read from the full text via Valyu Contents

Computational models, from simple embeddings to large language models, are used to link linguistic theory with brain activity, revealing semantic content drives brain similarity and modest data suffices.

What’s new

i.Identified lexical semantic content as the main driver of model‑to‑brain similarity in fMRI (Kauf et al.)
ii.Showed that language models trained on ~100 million tokens achieve near‑maximal prediction of neural and behavioral responses (Hosseini et al.)
iii.Demonstrated surprisal predicts N400 amplitudes better than semantic similarity, supporting predictive coding theories (Michaelov et al.)
iv.Introduced a neural‑encoding simulation paradigm for generating controlled‑experiment‑like brain data from naturalistic fMRI (Jain et al.)
v.Provided evidence that syntactic‑aware RNN grammars better explain activity in certain frontal and temporoparietal regions than sequential models (Sugimoto et al.)

How they did it

The issue combines neural encoding of fMRI and ERP data with computational descriptors derived from word embeddings, recurrent networks, LSTMs, transformers and syntactic grammars. Linear regression models map model activations to brain signals, and performance is evaluated by held‑out prediction accuracy. Some studies also simulate brain responses by feeding controlled stimuli into trained encoding models.

What they found

Removing content words sharply reduces model‑to‑brain similarity, while function‑word removal or word‑order perturbation does not (Kauf et al.)
Models trained on 100 M tokens reach near‑maximal brain‑behavior prediction, comparable to models trained on far larger corpora (Hosseini et al.)
Word surprisal explains more variance in N400 amplitudes than cosine similarity of static embeddings (Michaelov et al.)
Semantic update and surprisal each independently predict N400 amplitude, indicating two subprocesses (Lopopolo & Rabovsky)
Combining surprisal with an Agent Preference principle yields the best N400 predictions across German, Basque and Hindi (Huber et al.)
RNN grammars outperform sequential models in inferior frontal and temporoparietal areas during Japanese reading (Sugimoto et al.)

Caveats

✕The majority of findings rely on correlational encoding models, limiting causal inference.
✕Results are based on specific languages (English, Japanese, German, Basque, Hindi) and may not generalize universally.
✕Encoding performance depends on the choice of model layer and stimulus set, introducing methodological variability.
✕The predictive power of large language models is assessed on existing datasets, not on real‑time interactive language use.

§ The Valyu brief

Reading the full paper and taking notes. This takes a few seconds…

§ Ask this paper

Ask a question about this paper

Valyu reads the full text and answers from what the paper actually says.

§ Related work in other archives

Cognitive Computational Neuroscience of Language: Using Computational Models to Investigate Language Processing in the Brain

The paper, in five minutes

Ask a question about this paper

Similar papers in all four archives

The paper, in five minutes

Similar papers in all four archives