A recurrent vision transformer shows signatures of primate visual attention

Abstract

attention Authors: ['Jonathan Morgan' 'Badr F. Albanna' 'James P. Herman'] Attention has emerged as a core component of both biological and artificial intelligences (AIs). Despite decades of parallel research, studies of animal and AI attention remain largely separate. The self-attention mechanism ubiquitous in contemporary AI applications is not grounded in biology, and the powerful capabilities of AIs equipped with self-attention have yet to offer fresh insight into the biological mechanisms of attention. Here, we offer a unifying perspective, drawing together insights from primate neurophysiology and contemporary machine learning in a Recurrent Vision Transformer (Recurrent ViT). Our model extends self-attention by allowing both input and memory to guide attention allocation. Our model learns purely through sparse reward feedback, emulating how animals must learn in a laboratory environment. We benchmark our model by challenging it to perform a spatially cued orientation-change detection task widely used to study attention in the laboratory and comparing its performance to non-human primates (NHPs). The model exhibits hallmark behavioral signatures of primate visual attention — improved accuracy and faster responses for cued stimuli, both scaling with cue validity. Analysis of self-attention maps reveals rich attention dynamics, with the model maintaining spatial priorities through delays and stimulus onsets, reactivating them prior to anticipated change events. Perturbing these attention maps produces performance changes that mirror effects of causal manipulations in primate attention nodes such as the frontal eye fields (FEF) or superior colliculus (SC). These findings not only validate the effectiveness of integrating recurrent memory with self-attention for emulating primate-like attention, but also establish a promising framework for probing the neural underpinnings of attentional control. Ultimately, our work attempts to bridge the gap between biological and artificial attention, paving the way for more interpretable and neurologically informed AI systems.