13 papers · ranked by Valyu relevance
Kai Yue, Richeng Jin, Chau-Wai Wong, Huaiyu Dai
—Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical bottleneck due to limited power and bandwidth. Prior work has utilized various data…
André Ofner, Sebastian Stober
This paper deals with differentiable dynamical models congruent with neural process theories that cast brain function as the hierarchical refinement of an internal generative model explaining observations. Our work extends existing implementations of gradient-based predictive coding with automatic differentiation and…
Lucie Fontaine
—Complementary Learning Systems theory holds that intelligent agents need two learning systems. Semantic memory is encoded in the neocortex with dense, overlapping representations and acquires structured knowledge. Episodic memory is encoded in the hippocampus with sparse, pattern-separated representations and quickly…
David Calhas, Arlindo L. Oliveira
While biological vision systems rely heavily on feedback connections to iteratively refine perception, most artificial neural networks remain purely feedforward, processing input in a single static pass. In this work, we propose a predictive coding inspired feedback mechanism that introduces a recurrent loop from…
Anthony Lin
We introduce a novel predictive coding framework for studying attachment theory. Building off an established model of attachment, the dynamic-maturational model (DMM), as well as the neuroanatomical Embodied Predictive Interoception Coding (EPIC) model of interoception and emotion, we not only elucidate how neural…
Linxing Preston Jiang, Rajesh P. N. Rao
Predictive coding is a unifying framework for understanding perception, action and neocortical organization. In predictive coding, dierent areas of the neocortex implement a hierarchical generative model of the world that is learned from sensory inputs. Cortical circuits are hypothesized to perform Bayesian inference…
Rajesh P. N. Rao, Vishwas Sathish, Linxing Preston Jiang, Matthew Bryan + 1 more
The phenomenal advances in large language models (LLMs) and other foundation models over the past few years have been based on optimizing large-scale transformer models on the surprisingly simple objective of minimizing next-token prediction loss, a form of predictive coding that is also the backbone of an increasingly…
Antony W. N’Dri, William Gebhardt, Céline Teulière, Fleur Zeldenrust + 3 more
'Rajesh P. N. Rao' 'Jochen Triesch' 'Alexander G. Ororbia'] In this article, we review a class of neuro-mimetic computational models that we place under the label of spiking predictive coding. Specifically, we review the general framework of predictive processing in the context of neurons that emit discrete action…
Ehsan Ganjidoost, Mallory Snow, Jeff Orchard
Neuroscience and Artificial Intelligence (AI) have progressed in tandem, each contributing to our understanding of the brain, and inspiring recent developments in biologically-plausible neural networks (NNs) and learning rules. Predictive coding (PC), and its learning rule, have been shown to approximate error…
Nikolas McNeal, Jennifer T. Huang, Aniekan Umoren, Shuqi Dai + 3 more
'Roger B. Dannenberg' 'Richard Randall' 'Tai Sing Lee'] We explore the use of a neural network inspired by predictive coding for modeling human music perception. This network was developed based on the computational neuroscience theory of recurrent interactions in the hierarchical visual cortex. When trained with video…
Mufeng Tang, Helen C. Barron, Rafał Bogacz
Forming accurate memory of sequential stimuli is a fundamental function of biological agents. However, the computational mechanism underlying sequential memory in the brain remains unclear. Inspired by neuroscience theories and recent successes in applying predictive coding (PC) to static memory tasks, in this work we…
Xingsi Dong, Peng Yuan, Si Wu
Predictive models are highly advanced in understanding the mechanisms of brain function. Recent advances in machine learning further underscore the power of prediction for optimal representation in learning. However, there remains a gap in creating a biologically plausible model that explains how the neural system…
Zhichao Yang, Weisong Liu, Dan R. Berlowitz, Hong Yu
Longitudinal electronic health records (EHRs) can be used for enhancing the prediction of individual patient disease or outcome onsets. Early prediction approaches include regression, traditional machine-learning-based, and ensemble models (e.g., support vectors machines, random forest, and gradient boosting). 1…