29 papers · ranked by Valyu relevance
Hojjat Salehinejad, Julianne Baarbé, Sharan Sankar, Joseph Barfett + 2 more
'Errol Colak' 'Shahrokh Valaee'] Abstract—Recurrent neural networks (RNNs) are capable of learning features and long term dependencies from sequential and time-series data. The RNNs have a stack of non-linear units where at least one connection between units forms a directed cycle. A well-trained RNN can model any…
Frank Emmert-Streib, Zhen Yang, Han Feng, Shailesh Tripathi + 1 more
'Matthias Dehmer'] Deep learning models stand for a new learning paradigm in artificial intelligence (AI) and machine learning. Recent breakthrough results in image analysis and speech recognition have generated a massive interest in this field because also applications in many other domains providing big data seem…
H. Francis Song, Guangyu R. Yang, Xiao-Jing Wang, Olaf Sporns
The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze…
Razvan Pascanu, Çağlar Gülçehre, Kyunghyun Cho, Yoshua Bengio
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a deep RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points of an RNN…
Jian-wei Liu, Bingrong Xu, Zhiyan Song
Recursive and recurrent neural networks are the main realization forms of sequence models based on neural networks, which have been developed rapidly in recent years. Recurrent neural networks are basically standard processing methods for machine translation, question and answer, and sequence video analysis. It is also…
Filippo Maria Bianchi, Enrico Maiorino, Michael Kampffmeyer, Antonello Rizzi + 1 more
'Antonello Rizzi' 'Robert Jenssen'] The key component in forecasting demand and consumption of resources in a supply network is an accurate prediction of real-valued time series. Indeed, both service interruptions and resource waste can be reduced with the implementation of an effective forecasting system. Significant…
Jonathan C Kao
Recurrent neural networks (RNNs) are increasingly being used to model complex cognitive and motor tasks performed by behaving animals. Here, RNNs are trained to reproduce animal behavior while also recapitulating key statistics of empirically recorded neural activity. In this manner, the RNN can be viewed as an in…
R. Devon Hjelm, Eswar Damaraju, Kyunghyun Cho, Helmut Laufs + 2 more
'Sergey M. Plis' 'Vince D. Calhoun'] We introduce a novel recurrent neural network (RNN) approach to account for temporal dynamics and dependencies in brain networks observed via functional magnetic resonance imaging (fMRI). Our approach directly parameterizes temporal dynamics through recurrent connections, which can…
Felix Grezes
Introduction: Even before Artificial Intelligence was its own field of computational science, humanity has tried to mimic the activity of the human brain. In the early 1940s the first artificial neuron models were created as purely mathematical concepts. Over the years, ideas from neuroscience and computer science were…
Courtney J. Spoerer, Tim C. Kietzmann, Johannes Mehrer, Ian Charest + 2 more
'Nikolaus Kriegeskorte' 'Leyla Isik'] Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and…
Sanjar Adilov
Generative neural networks have shown promising results in de novo drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this…
Sarah E. Marzen, Paul M. Riechers, James P. Crutchfield
Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a…
Kazunori D Yamada
Recurrent neural networks (RNNs) are among the most promising of the many artificial intelligence techniques now under development, showing great potential for memory, interaction, and linguistic understanding. Among the more sophisticated RNNs are long short-term memory (LSTM) and gated recurrent units (GRUs), which…
Chenxi Sui, Ziyang Jiang, Genesis Higueros, David Carlson + 1 more
High-performance batteries are poised for electrification of vehicles and therefore mitigate greenhouse gas emissions, which, in turn, promote a sustainable future. However, the design of optimized batteries is challenging due to the nonlinear governing physics and electrochemistry. Recent advancements have…
Jianshu Chen, Li Deng
We present an architecture of a recurrent neural network (RNN) with a fullyconnected deep neural network (DNN) as its feature extractor. The RNN is equipped with both causal temporal prediction and non-causal look-ahead, via auto-regression (AR) and moving-average (MA), respectively. The focus of this paper is a…
Yuval Jacoby, Clark Barrett, Guy Katz
Deep neural networks are revolutionizing the way complex systems are developed. However, these automatically-generated networks are opaque to humans, making it difficult to reason about them and guarantee their correctness. Here, we propose a novel approach for verifying properties of a widespread variant of neural…
Ying Ma, José C. Prı́ncipe
—In this paper, a taxonomy for memory networks is proposed based on their memory organization. The taxonomy includes all the popular memory networks: vanilla recurrent neural network (RNN), long short term memory (LSTM ), neural stack and neural Turing machine and their variants. The taxonomy puts all these networks…
Nicolas Vecoven, Damien Ernst, Guillaume Drion, Maciej Huk
Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to…
Klavdia Zemlianova, Amitabha Bose, John Rinzel
Despite music’s omnipresence, the specific neural mechanisms responsible to perceive and anticipate temporal patterns in music are unknown. To study potential mechanisms for keeping time in rhythmic contexts, we train a biologically constrained RNN on seven different stimulus tempos (2 – 8Hz) on a synchronization and…
Huzi Cheng, Joshua W. Brown
How episodic memories are formed in the brain is an outstanding puzzle for the neuroscience community. The brain areas that are critical for episodic learning (e.g., the hippocampus) are characterized by recurrent connectivity and generate frequent offline replay events. The function of the replay events is a subject…
Pavel Tolmachev, Tatiana A. Engel
Trained recurrent neural networks (RNNs) have become the leading framework for modeling neural dynamics in the brain, owing to their capacity to mimic how population-level computations arise from interactions among many units with heterogeneous responses. RNN units are commonly modeled using various nonlinear…
James M Murray, Peter Latham, Michael J Frank
Recurrent neural networks (RNNs) enable the production and processing of time-dependent signals such as those involved in movement or working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but are inconsistent with biological features of the brain, such as causality and…
Waddah Waheeb, Rozaida Ghazali, Tutut Herawan, Zhaohong Deng
Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over…
Sweta Kumari, C Vigneswaran, V. Srinivasa Chakravarthy
Sequential decision making tasks that require information integration over extended durations of time are challenging for several reasons including the problem of vanishing gradients, long training times and significant memory requirements. To this end we propose a neuron model fashioned after the JK flip-flops in…
Courtney J Spoerer, Tim C Kietzmann, Johannes Mehrer, Ian Charest + 1 more
Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a…
Authors not listed
This research presents a novel approach to obstacle detection during navigation using a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. The primary objective is to generate accurate image captions that describe the content of images, which is crucial for applications such…
Morgan Thomas, Noel M. O'Boyle, Andreas Bender, Chris de Graaf
A plethora of AI-based techniques now exists to conduct de novo molecule generation that can devise molecules conditioned towards a particular endpoint in the context of drug design. One popular approach is using reinforcement learning to update a recurrent neural network or language-based de novo molecule generator.…
Authors not listed
The automatic generation of image captions in natural language is a critical and challenging task, particularly in the context of environmental monitoring and control. This paper presents a novel deep learning-driven image captioning system designed for real-time monitoring and predictive control of pollutant gas…
Fuzhan Rahmanian, Robert M. Lee, Dominik Linzner, Kathrin Michel + 4 more
Predicting and monitoring battery life early and across chemistries is a significant challenge due to the plethora of degradation paths, form factors, and electrochemical testing protocols. Existing models typically translate poorly across different electrode, electrolyte, and additive materials, mostly require a fixed…