15 papers · ranked by Valyu relevance
Rajesh P. N. Rao, Vishwas Sathish, Linxing Preston Jiang, Matthew Bryan + 1 more
The phenomenal advances in large language models (LLMs) and other foundation models over the past few years have been based on optimizing large-scale transformer models on the surprisingly simple objective of minimizing next-token prediction loss, a form of predictive coding that is also the backbone of an increasingly…
Antony W. N'dri, Thomas Barbier, Céline Teulière, Jochen Triesch
The ability to predict the future is of great value for biological and artificial cognitive systems alike. However, successfully predicting the future typically requires maintaining a memory of the recent past. It is currently unclear how biological or artificial spiking neural networks can learn to maintain past…
Gaspard Oliviers, Elene Lominadze, Rafal Bogacz
Predictive Coding (PC) is an influential account of cortical learning. Much of recent work has focused on comparing PC to Backpropagation (BP) to find whether PC offers any advantages. Small scale experiments show that PC enables learning that is more sample efficient and effective in many contexts, though a thorough…
Sourabh Bhattacharya
Predictive coding offers a powerful theory of cortical computation, but corresponding scalable algorithmic implementations for artificial intelligence have remained elusive. This paper introduces the Bayesian reflex, a computational framework that directly instantiates predictive coding through three pillars: belief…
Aleksandrs Baskakovs, Sylvain Estebe, Kenneth Enevoldsen, Kristoffer Nielbo + 2 more
Predictive coding (PC) offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks, yet to date, it remains slower, and performance degrades sharply as network depth increases. We trace both problems to a single simplification: current PC networks fix the…
Shogo Ohmae, Keiko Ohmae
Recent advances in general-purpose AIs with attention-based transformers offer a potential window into how the neocortex and cerebellum, despite their relatively uniform circuit architectures, give rise to diverse functions and, ultimately, to human intelligence. This Perspective provides a cross-domain comparison…
Amirhossein Mohammadi, Alexander G. Ororbia
Predictive coding networks (PCNs) offer a biologically-plausible, local-learning alternative to back-propagation of errors (backprop). Nevertheless, they have remained largely confined to shallow architectures and evaluated on simple machine intelligence benchmarks. A central obstacle to scaling PCNs is that the…
Yamada, Yohei, Zenas C. Chao
The brain predicts the external world through an internal model refined by prediction errors. A complete prediction specifies what will happen, when it will happen, and with what probability, a construct we call the "prediction object." Existing models usually capture only what and when, omit probabilities, and rely on…
Andrew L. Smith, Linxing Preston Jiang, Jason K. Eshraghian, Matthew S. Bull + 1 more
Hierarchical predictive coding proposes a compelling hypothesis of brain computation, suggesting that the cortex builds layered predictions to minimize surprise. Yet most models rely on error-coding neurons or generative modeling of unclear biological plausibility. Here, we examine a biologically plausible framework in…
Wu Yonggang
| 1. Introduction4 | | |----------------------------------------------------------------------|--| | 2. Module 1: Visual processing system (VPS) module4 | | | 2.1 Complementary plasticity hypothesis (CPH)5 | | | 2.2 Training and tuning6 | | | 2.3 Specificity and invariance7 | | | 2.4 Invariance through excitatory…
John R. Minnick, Jesus Gonzalez-Ferrer, Kamran Hussain, Jinghui Geng + 5 more
Closed-loop brain-computer interfaces often require both a forecast of upcoming neural population activity and a readout of the animal's behavioral state. A single Mamba forecaster, trained only on next-step spike counts at Neuropixels scale, can deliver both in one forward pass. A lightweight per-session linear head…
Sean Niklas Semmler
Understanding intelligence and consciousness requires moving beyond cataloging cognitive abilities to identifying the fundamental operations that produce them. This paper argues that intelligence emerges from a single purpose:forming, refining, and integrating causal connections between signals, actions, internal…
Gal Fybish, Teo Susnjak
Machine learning models are increasingly used in high-stakes domains where their predictions can actively shape the environments in which they operate, a phenomenon known as performative prediction. This dynamic, in which the deployment of the model influences the very outcome it seeks to predict, can lead to…
Lukas Schelenz, Shobha Rajanna, Denis Gosalci, Lucas Heublein + 5 more
Forecasting within signal processing pipelines is crucial for mitigating delays, particularly in predicting the dynamic movements of objects such as NBA players. This task poses significant challenges due to the inherently interactive and unpredictable nature of sports, where abrupt changes in velocity and direction…
Daniel Brownell
Continuous attractor networks (CANs) are a well-established class of models for representing lowdimensional continuous variables such as head direction, spatial position, and phase [1, 2, 3, [4]]. In canonical spatial domains, transitions along the attractor manifold are driven by continuous displacement signals—such…