24 papers · ranked by Valyu relevance
Claus Metzner, Achim Schilling, Andreas Maier, Patrick Krauss
Previous work has shown that the dynamical regime of Recurrent Neural Networks (RNNs)—ranging from oscillatory to chaotic and fixpoint behavior—can be controlled by the global distribution of weights in connection matrices with statistically independent elements. However, it remains unclear how network dynamics respond…
Louis Kirsch, Julius Kunze, David Barber
Scaling model capacity has been vital in the success of deep learning. For a typical network, necessary compute resources and training time grow dramatically with model size. Conditional computation is a promising way to increase the number of parameters with a relatively small increase in resources. We propose a…
Ziyu Wang, Wenhao Jiang, Zixuan Zhang, Wei Tang + 1 more
—Sequential processes in real-world often carry a combination of simple subsystems that interact with each other in certain forms. Learning such a modular structure can often improve the robustness against environmental changes. In this paper, we propose recurrent independent Grid LSTM (RigLSTM), composed of a group of…
Kohei Ichikawa, Kunihiko Kaneko, Thomas Serre
Various animals, including humans, have been suggested to perform Bayesian inferences to handle noisy, time-varying external information. In performing Bayesian inference by the brain, the prior distribution must be acquired and represented by sampling noisy external inputs. However, the mechanism by which neural…
Mani Hamidi, Sina Khajehabdollahi, Emmanouil Giannakakis, Tim Jakob Schäfer + 2 more
Curriculum Learning Authors: ['Mani Hamidi' 'Sina Khajehabdollahi' 'Emmanouil Giannakakis' 'Tim Jakob Schäfer' 'Anna Levina' 'Charley M. Wu'] Structural modularity is a pervasive feature of biological neural networks, which have been linked to several functional and computational advantages. Yet, the use of modular…
Barna Zajzon, Renato Duarte, Abigail Morrison
To acquire statistical regularities from the world, the brain must reliably process, and learn from, spatio-temporally structured information. Although an increasing number of computational models have attempted to explain how such sequence learning may be implemented in the neural hardware, many remain limited in…
I. Cone, H. Z. Shouval
The ability to express and learn temporal sequences is an essential part of learning and memory. Learned temporal sequences are expressed in multiple brain regions and as such there may be common design in the circuits that mediate it. This work proposes a substrate for such representations, via a biophysically…
Yuma Osako, Timothy J. Buschman, Mriganka Sur
Complex behaviors are thought to be built by combining simpler cognitive components. Computational modeling has shown that artificial neural networks can perform a variety of tasks by flexibly combining small functional modules of neurons, each specialized for a specific computation, to construct a complex task.…
Antonio Carta, Alessandro Sperduti, Davide Bacciu
The effectiveness of recurrent neural networks can be largely influenced by their ability to store into their dynamical memory information extracted from input sequences at different frequencies and timescales. Such a feature can be introduced into a neural architecture by an appropriate modularization of the dynamic…
Ian Cone, Harel Z Shouval, Tatyana O Sharpee, Ronald L Calabrese
Multiple brain regions are able to learn and express temporal sequences, and this functionality is an essential component of learning and memory. We propose a substrate for such representations via a network model that learns and recalls discrete sequences of variable order and duration. The model consists of a network…
Zachariah Carmichael, Humza Syed, Stuart Burtner, Dhireesha Kudithipudi
'Dhireesha Kudithipudi'] Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they…
Filip Milisav, Andrea I. Luppi, Laura E. Suárez, Guillaume Lajoie + 1 more
Modularity is a fundamental principle of brain organization, reflected in the presence of segregated sub-networks that enable specialized information processing. These small, densely connected modules are often nested within larger, higher-order modules, giving rise to a hierarchical modular architecture. This…
Fatemeh Hadaeghi, Kayson Fakhar, Moein Khajehnejad, Claus C. Hilgetag
Cerebral cortical networks in the mammalian brain exhibit a non-random organization that systematically avoids strong reciprocal projections, particularly in sensory hierarchies. This “no-strong-loops” principle is thought to prevent runaway excitation and maintain stability, yet its computational impact remains…
Julia C. Costacurta, Shaunak Bhandarkar, David M. Zoltowski, Scott W. Linderman
The goal of theoretical neuroscience is to develop models that help us better understand biological intelligence. Such models range broadly in complexity and biological detail. For example, task-optimized recurrent neural networks (RNNs) have generated hypotheses about how the brain may perform various computations…
Julia C. Costacurta, Shaunak Bhandarkar, David M. Zoltowski, Scott W. Linderman
The goal of theoretical neuroscience is to develop models that help us better understand biological intelligence. Such models range broadly in complexity and biological detail. For example, task-optimized recurrent neural networks (RNNs) have generated hypotheses about how the brain may perform various computations…
Liuyi Yang, Zhaoze Wang, Guoyu Wang, Lixin Liang + 2 more
Previous studies have successfully applied a lightweight recurrent neural network (RNN) called Echo State Network (ESN) for EEG-based emotion recognition. These studies use intrinsic plasticity (IP) and synaptic plasticity (SP) to tune the hidden reservoir layer of ESN, yet they require extra training procedures and…
Sanjar Adilov
Generative neural networks have shown promising results in de novo drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this…
Courtney J Spoerer, Tim C Kietzmann, Johannes Mehrer, Ian Charest + 1 more
Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a…
Chenxi Sui, Ziyang Jiang, Genesis Higueros, David Carlson + 1 more
High-performance batteries are poised for electrification of vehicles and therefore mitigate greenhouse gas emissions, which, in turn, promote a sustainable future. However, the design of optimized batteries is challenging due to the nonlinear governing physics and electrochemistry. Recent advancements have…
Sou Nobukawa, Aya Shirama, Yusuke Sakemi, Eiji Watanabe + 3 more
Biological brain networks flexibly reconfigure functional connectivity between integration and segregation through neuromodulatory systems—noradrenaline (NA) and acetylcholine (ACh)—without altering structural connectivity. Inspired by this mechanism, we propose a modular echo state network (ESN) with context-dependent…
Authors not listed
This research presents a novel approach to obstacle detection during navigation using a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. The primary objective is to generate accurate image captions that describe the content of images, which is crucial for applications such…
Authors not listed
The automatic generation of image captions in natural language is a critical and challenging task, particularly in the context of environmental monitoring and control. This paper presents a novel deep learning-driven image captioning system designed for real-time monitoring and predictive control of pollutant gas…
Morgan Thomas, Noel M. O'Boyle, Andreas Bender, Chris de Graaf
A plethora of AI-based techniques now exists to conduct de novo molecule generation that can devise molecules conditioned towards a particular endpoint in the context of drug design. One popular approach is using reinforcement learning to update a recurrent neural network or language-based de novo molecule generator.…
Zachary Humphreys, Xenophon Evangelopoulos, Stavros Gerolymatos, Edward O. Pyzer-Knapp + 1 more
Graph neural networks have recently met huge success in various inference tasks including materials property prediction amongst many others. Nevertheless, having an inherently locally-based representation capacity as they do, global representation of materials' structures can only only be achieved by expanding the…