22 papers · ranked by Valyu relevance
Zhiyuan Liu, Yankai Lin, Maosong Sun
© The Editor(s) (if applicable) and The Author(s) 2020. This book is an open access publication. Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and…
Lorenzo Ferrone, Fabio Massimo Zanzotto
Natural language is inherently a discrete symbolic representation of human knowledge. Recent advances in machine learning (ML) and in natural language processing (NLP) seem to contradict the above intuition: discrete symbols are fading away, erased by vectors or tensors called distributed and distributional…
Lorenzo Ferrone, Fabio Massimo Zanzotto
Natural language and symbols are intimately correlated. Recent advances in machine learning (ML) and in natural language processing (NLP) seem to contradict the above intuition: symbols are fading away, erased by vectors or tensors called distributed and distributional representations. However, there is a strict link…
Asim Roy
In this article, I present the theory that localist representation is used widely in the brain starting from its earliest levels of processing. Page () argued for localist representation and Bowers () claimed that the brain uses grandmother cells to code for objects and concepts. However, neither Page () nor Bowers ()…
Simon Dobnik, Robin Cooper, Adam Ek, Bill Noble + 4 more
'Nikolai Ilinykh' 'Vladislav Maraev' 'Vidya Somashekarappa'] In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the…
Zhenglong Zhou, Dhairyya Singh, Marlie C. Tandoc, Anna C. Schapiro
Inferring relationships that go beyond our direct experience is essential for understanding our environment. This capacity requires either building representations that directly reflect structure across experiences as we encounter them, or computing the indirect relationships across experiences as the need arises.…
Jingcheng Du, Peilin Jia, Yulin Dai, Cui Tao + 2 more
Existing functional description of genes are categorical, discrete, and mostly through manual process. In this work, we explore the idea of gene embedding, distributed representation of genes, in the spirit of word embedding. From a pure data-driven fashion, we trained a 300 dimension vector representation of all human…
Yan Wang, Zhiyuan Liu, Maosong Sun
Combined with neural language models, distributed word representations achieve significant advantages in computational linguistics and text mining. Most existing models estimate distributed word vectors from large-scale data in an unsupervised fashion, which, however, do not take rich linguistic knowledge into…
Chao Wei, Zhiwei Ye, Junying Zhang, Aimin Li
Long non-coding RNAs (lncRNAs) play a crucial role in numbers of biological processes and have received wide attention during the past years. Meanwhile, the rapid development of high-throughput transcriptome sequencing technologies (RNA-seq) lead to a large amount of RNA data, it is urgent to develop a fast and…
Hitoshi Iuchi, Taro Matsutani, Keisuke Yamada, Natsuki Iwano + 5 more
Remarkable advances in high-throughput sequencing have resulted in rapid data accumulation, and analyzing biological (DNA/RNA/protein) sequences to discover new insights in biology has become more critical and challenging. To tackle this issue, the application of natural language processing (NLP) to biological sequence…
Samuel A. Nastase, Andrew C. Connolly, Nikolaas N. Oosterhof, Yaroslav O. Halchenko + 5 more
Humans prioritize different semantic qualities of a complex stimulus depending on their behavioral goals. These semantic features are encoded in distributed neural populations, yet it is unclear how attention might operate across these distributed representations. To address this, we presented participants with…
Yizhen Zhang, Kuan Han, Robert Worth, Zhongming Liu
In the brain, the semantic system is thought to store concepts. However, little is known about how it connects different concepts and infers semantic relations. To address this question, we collected hours of functional magnetic resonance imaging data from human subjects listening to natural stories. We developed a…
Yuzhu Wu, Zhen Zhang, Gang Kou, Hengjie Zhang + 4 more
'Cong‐Cong Li' 'Yucheng Dong' 'Francisco Herrera'] Abstract: Distributed linguistic representations are powerful tools for modelling the uncertainty and complexity of preference information in linguistic decision making. To provide a comprehensive perspective on the development of distributed linguistic representations…
Ying Xiong, Shuai Chen, Haoming Qin, He Cao + 5 more
'Xiaolong Wang' 'Qingcai Chen' 'Jun Yan' 'Buzhou Tang'] Background Semantic textual similarity (STS) is a fundamental natural language processing (NLP) task which can be widely used in many NLP applications such as Question Answer (QA), Information Retrieval (IR), etc. It is a typical regression problem, and almost all…
Zeb Kurth-Nelson, A. David Redish, Olaf Sporns
Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the…
Yarin Gal
Over the past 50 years many have debated what representation should be used to capture the meaning of natural language utterances. This modelling problem forms the first step towards the extraction of information conveyed in a sentence. But even the question of what a representation of meaning should satisfy is subject…
Darren J. Edwards, Ciara McEnteggart, Yvonne Barnes-Holmes
Psychology has benefited from an enormous wealth of knowledge about processes of cognition in relation to how the brain organizes information. Within the categorization literature, this behavior is often explained through theories of memory construction called exemplar theory and prototype theory which are typically…
Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber
Contemporary neural networks still fall short of human-level generalization, which extends far beyond our direct experiences. In this paper, we argue that the underlying cause for this shortcoming is their inability to dynamically and flexibly bind information that is distributed throughout the network. This binding…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…
Anthony Onwuli, Keith T. Butler, Aron Walsh
High-dimensional representations of the elements have become common within the field of materials informatics to build useful, structure-agnostic models for the chemistry of materials. However, the characteristics of elements change when they adopt a given oxidation state, with distinct structural preferences and…
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi + 3 more
Massive scale, both in terms of data availability and computation, enables significant breakthroughs in key application areas of deep learning such as natural language processing (NLP) and computer vision. There is emerging evidence that scale may be a key ingredient in scientific deep learning, but the importance of…
Authors not listed
Deriving versatile and robust mechanistic models from experimental data is a key challenge in engineering and natural sciences. This is especially true in chemical reaction engineering, where reactor manufacturers and operators increasingly pursue the development and maintenance of digital twins that rely on frequent…