23 papers · ranked by Valyu relevance
Dylan R. Muir
Recurrent neural network architectures can have useful computational properties, with complex temporal dynamics and input-sensitive attractor states. However, evaluation of recurrent dynamic architectures requires solution of systems of differential equations, and the number of evaluations required to determine their…
Katharina Flügel, Daniel Coquelin, Marie Weiel, Charlotte Debus + 2 more
'Achim Streit' 'Markus Götz'] Backpropagation has long been criticized for being biologically implausible, relying on concepts that are not viable in natural learning processes. This paper proposes an alternative approach to solve two core issues, i.e., weight transport and update locking, for biological plausibility…
Adrian Moldovan, Angel Caţaron, Răzvan Andonie
Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an…
Ellen Boven, Joseph Pemberton, Paul Chadderton, Richard Apps + 1 more
'Rui Ponte Costa'] Behavioural feedback is critical for learning in the cerebral cortex. However, such feedback is often not readily available. How the cerebral cortex learns efficiently despite the sparse nature of feedback remains unclear. Inspired by recent deep learning algorithms, we introduce a systems-level…
Yulia Revina, Lucy S. Petro, Cristina B. Denk-Florea, Isa S. Rao + 1 more
The majority of synaptic inputs to the primary visual cortex (V1) are non-feedforward, instead originating from local and anatomical feedback connections. Animal electrophysiology experiments show that feedback signals originating from higher visual areas with larger receptive fields modulate the surround receptive…
Adrian Moldovan, Angel Caţaron, Răzvan Andonie
Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an…
Aditya Gilra, Wulfram Gerstner, Peter Latham
The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous…
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, Colin J. Akerman
'Colin J. Akerman'] The brain processes information through multiple layers of neurons. This deep architecture is representationally powerful, but complicates learning because it is difficult to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame by…
Alessandro Sanzeni, Mark H Histed
Simulations predict a paradoxical effect that should be revealed by patterned stimulation of the cortex.
Edward Zagha
Cortical feedback pathways are proposed to guide cognition and behavior according to context and goal-direction. At the cellular level, cortical feedback pathways target multiple excitatory and inhibitory populations. However, we currently lack frameworks that link how the cellular mechanisms of cortical feedback…
Mashbayar Tugsbayar, Mingze Li, Eilif B. Muller, Blake Richards
Artificial neural networks (ANNs) are an important tool for studying neural computation, but many features of the brain are not captured by standard ANN architectures. One notable missing feature in most ANN models is top-down feedback, i.e. projections from higher-order layers to lower-order layers in the network.…
Romesa Khan, Hongsheng Zhong, Shuvam Das, Jack Cai + 1 more
Seminal frameworks of predictive coding propose a hierarchy of generative modules, each attempting to infer the neural representation of the module one level below; the predictions are carried by top-down feedback projections, while the predictive error is propagated by reciprocal forward pathways. Such symmetric…
Laura Bella Naumann, Joram Keijser, Henning Sprekeler
Sensory systems reliably process incoming stimuli in spite of changes in context. Most recent models accredit this context invariance to an extraction of increasingly complex sensory features in hierarchical feedforward networks. Here, we study how context-invariant representations can be established by feedback rather…
Georg Layher, Fabian Schrodt, Martin V. Butz, Heiko Neumann
The categorization of real world objects is often reflected in the similarity of their visual appearances. Such categories of objects do not necessarily form disjunct sets of objects, neither semantically nor visually. The relationship between categories can often be described in terms of a hierarchical structure. For…
Sebastian Herzog, Christian Tetzlaff, Florentin Wörgötter
The structure of the majority of modern deep neural networks is characterized by unidirectional feed-forward connectivity across a very large number of layers. By contrast, the architecture of the cortex of vertebrates contains fewer hierarchical levels but many recurrent and feedback connections. Here we show that a…
Marco Kemmerling
Feedback connections play a prominent role in the human brain but have not received much attention in artificial neural network research. Here, a biologically inspired feedback mechanism which gates rectified linear units is proposed. On the MNIST dataset, autoencoders with feedback show faster convergence, better…
Authors not listed
We improved the potential energy surfaces for fourteen coupled 3A′ states of O3 by using parametrically managed diabatization by deep neural network (PM-DDNN) with three improvements: (1) We use a new functional form for the parametrically managed activation function, which ensures the continuity of the coordinates…
Esther Heid, Charles J. McGill, Florence H. Vermeire, William H. Green
Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further…
Aleksandr Fedorov, Anna Perechodjuk, David Linke
Artificial neural networks (ANNs) are powerful tools for solving a wide range of tasks in fundamental and applied science. However, training and building reliable ANN models requires a lot of data which so far hinders their wider application in kinetic modelling where typically only small (experimental) datasets are…
Authors not listed
This research presents a novel approach to obstacle detection during navigation using a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. The primary objective is to generate accurate image captions that describe the content of images, which is crucial for applications such…
Kamil Rocki
Recurrent neural nets are widely used for predicting temporal data. Their inherent deep feedforward structure allows learning complex sequential patterns. It is believed that top-down feedback might be an important missing ingredient which in theory could help disambiguate similar patterns depending on broader context.…
Farideh Badichi Akher, Yinan Shu, Zoltan Varga, Suman Bhaumik + 1 more
Machine-learned representations of potential energy surfaces generated in the output layer of a feedforward neural network are becoming increasingly popular. One difficulty with neural-network output is that it is often unreliable in regions where training data is missing or sparse. Humandesigned potentials often build…
Authors not listed
Inverse problems, where we seek the values of inputs to a model that lead to a desired set of outputs, are a challenges subset of problems in science and engineering. In this work we demonstrate the use of two generative AI methods to solve inverse problems. We compare this approach to two more conventional approaches…