Search · four archives
Search · four archives
27 papers · ranked by Valyu relevance
Alex Graves
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is…
Fei Zhu, Fei Ye, Yuchen Fu, Quan Liu + 1 more
Heart disease is a malignant threat to human health. Electrocardiogram (ECG) tests are used to help diagnose heart disease by recording the heart’s activity. However, automated medical-aided diagnosis with computers usually requires a large volume of labeled clinical data without patients' privacy to train the model…
Kai Standvoss, Silvan C. Quax, Marcel A.J. van Gerven
Allocating visual attention through saccadic eye movements is a key ability of intelligent agents. Attention is both influenced through bottom-up stimulus properties as well as top-down task demands. The interaction of these two attention mechanisms is not yet fully understood. A parsimonious reconciliation posits that…
Sanjar Adilov
Generative neural networks have shown promising results in de novo drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this…
Mohammad Pezeshki
In this paper, we have used Recurrent Neural Networks to capture and model human motion data and generate motions by prediction of the next immediate data point at each time-step. Our RNN is armed with recently proposed Gated Recurrent Units which has shown promissing results in some sequence modeling problems such as…
Sebastian Bitzer, Stefan J. Kiebel
Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely…
Chenxi Sui, Ziyang Jiang, Genesis Higueros, David Carlson + 1 more
High-performance batteries are poised for electrification of vehicles and therefore mitigate greenhouse gas emissions, which, in turn, promote a sustainable future. However, the design of optimized batteries is challenging due to the nonlinear governing physics and electrochemistry. Recent advancements have…
R. Devon Hjelm, Eswar Damaraju, Kyunghyun Cho, Helmut Laufs + 2 more
'Sergey M. Plis' 'Vince D. Calhoun'] We introduce a novel recurrent neural network (RNN) approach to account for temporal dynamics and dependencies in brain networks observed via functional magnetic resonance imaging (fMRI). Our approach directly parameterizes temporal dynamics through recurrent connections, which can…
Zexuan Yin, Paolo Barucca
Time series forecasting based on deep architectures has been gaining popularity in recent years due to their ability to model complex non-linear temporal dynamics. The recurrent neural network is one such model capable of handling variable-length input and output. In this paper, we leverage recent advances in deep…
Ares Fisher, Rajesh P N Rao, Derek Abbott
Human vision, thought, and planning involve parsing and representing objects and scenes using structured representations based on part-whole hierarchies. Computer vision and machine learning researchers have recently sought to emulate this capability using neural networks, but a generative model formulation has been…
Tom M George, Caswell Barry, Kimberly Stachenfeld, Claudia Clopath + 1 more
Advances in generative models have recently revolutionised machine learning. Meanwhile, in neuroscience, generative models have long been thought fundamental to animal intelligence. Understanding the biological mechanisms that support these processes promises to shed light on the relationship between biological and…
Francesco Visin, Kyle Kastner, Kyunghyun Cho, Matteo Matteucci + 2 more
'Aaron Courville' 'Yoshua Bengio'] In this paper, we propose a deep neural network architecture for object recognition based on recurrent neural networks. The proposed network, called ReNet, replaces the ubiquitous convolution+pooling layer of the deep convolutional neural network with four recurrent neural networks…
Morgan Thomas, Noel M. O'Boyle, Andreas Bender, Chris de Graaf
A plethora of AI-based techniques now exists to conduct de novo molecule generation that can devise molecules conditioned towards a particular endpoint in the context of drug design. One popular approach is using reinforcement learning to update a recurrent neural network or language-based de novo molecule generator.…
Dileep George, Alexander Lavin, J. Swaroop Guntupalli, David Mely + 2 more
Understanding the information processing roles of cortical circuits is an outstanding problem in neuroscience and artificial intelligence. The theoretical setting of Bayesian inference has been suggested as a framework for understanding cortical computation. Based on a recently published generative model for visual…
Alexander Ororbia, Daniel Kifer
Neural generative models can be used to learn complex probability distributions from data, to sample from them, and to produce probability density estimates. We propose a computational framework for developing neural generative models inspired by the theory of predictive processing in the brain. According to predictive…
Haider Al-Tahan, Yalda Mohsenzadeh
While vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model…
Gauthier Boeshertz, Claudia Clopath
The brain builds predictive models to plan future actions. These models generalize remarkably well to new environments, but it is unclear how neural circuits acquire this flexibility. Here, we show that compositional representations emerge in recurrent neural networks (RNNs) trained solely to predict future sensory…
Amrit Kashyap, Shella Keilholz
Large scale patterns of spontaneous whole brain activity seen in resting state functional Magnetic Resonance Imaging (rsfMRI), are in part believed to arise from neural populations interacting through the structural fiber network [18]. Generative models that simulate this network activity, called Brain Network Models…
Josep Arús-Pous, Thomas Blaschke, Silas Ulander, Jean-Louis Reymond + 2 more
Recent applications of Recurrent Neural Networks enable training models that sample the chemical space. In this study we train RNN with molecular string representations (SMILES) with a subset of the enumerated database GDB-13 (975 million molecules). We show that a model trained with 1 million structures (0.1 % of the…
Gido M. van de Ven, Hava T. Siegelmann, Andreas S. Tolias
Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those…
Josep Arús-Pous, Thomas Blaschke, Jean-Louis Reymond, Hongming Chen + 1 more
Recent applications of Recurrent Neural Networks enable training models that sample the chemical space. In this study we train RNN with molecular string representations (SMILES) with a subset of the enumerated database GDB-13 (975 million molecules). We show that a model trained with 1 million structures (0.1 % of the…
Authors not listed
In recent years, generative deep learning has emerged as a transformative approach in drug design, promising to explore the vast chemical space and generate novel molecules with desired biological properties. This perspective examines the challenges and opportunities of applying generative models to drug discovery…
César Mattos, Zhenwen Dai, Andreas Damianou, Jeremy Forth + 2 more
'Guilherme A. Barreto' 'Neil D. Lawrence'] We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations…
Chenpeng Zhang, Shuai Li, Mao Ye, Ce Zhu + 1 more
Recurrent neural networks (RNNs) are widely used as a memory model for sequence-related problems. Many variants of RNN have been proposed to solve the gradient problems of training RNNs and process long sequences. Although some classical models have been proposed, capturing long-term dependence while responding to…
Marco Zorzi, Alberto Testolin, Ivilin P. Stoianov
Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this…
Courtney J Spoerer, Tim C Kietzmann, Johannes Mehrer, Ian Charest + 1 more
Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a…
Felix Jimenez, Amanda Koepke, Mary Gregg, Michael Frey
Settings Authors: ['Felix Jimenez' 'Amanda Koepke' 'Mary Gregg' 'Michael Frey'] A generative adversarial network (GAN) is an artifcial neural network with a distinctive training architecture, designed to create examples that faithfully reproduce a target distribution. GANs have recently had particular success in…