26 papers · ranked by Valyu relevance
Armando J. Pinho, Paulo J. S. G. Ferreira, António J. R. Neves, Carlos A. C. Bastos + 1 more
'Carlos A. C. Bastos' 'Christos A. Ouzounis'] A finite-context (Markov) model of order yields the probability distribution of the next symbol in a sequence of symbols, given the recent past up to depth . Markov modeling has long been applied to DNA sequences, for example to find gene-coding regions. With the first…
Amirmehdi Jafari Fesharaki, Mohammadamin Rami, Aslan Tchamkerten
Transformers predict over a representation of a sequence. The same data can be written as bytes, characters, or subword tokens, and these representations may be lossless. Yet, under a fixed context window, they need not expose the same information to the model. This raises a basic question: how does the choice of…
Kristijan Armeni, Roel M. Willems, Stefan Frank
Cognitive neuroscientists of language comprehension study how neural computations relate to cognitive computations during comprehension. On the cognitive part of the equation, it is important that the computations and processing complexity are explicitly defined. Probabilistic language models can be used to give a…
Mai H. Vu, Ashkan Zehfroosh, Kristina Strother-Garcia, Michael Sebok + 2 more
'Jeffrey Heinz' 'Herbert G. Tanner'] This paper shows how methods from statistical relational learning can be used to address problems in grammatical inference using model-theoretic representations of strings. These model-theoretic representations are the basis of representing formal languages logically. Conventional…
Ehsan Shareghi, Gholamreza Haffari, Trevor Cohn, Ann E. Nicholson
Linguistic structures exhibit a rich array of global phenomena, however commonly used Markov models are unable to adequately describe these phenomena due to their strong locality assumptions. We propose a novel hierarchical model for structured prediction over sequences and trees which exploits global context by…
Refael Tikochinski, Ariel Goldstein, Yoav Meiri, Uri Hasson + 1 more
Accumulated evidence suggests that Large Language Models (LLMs) are beneficial in predicting neural signals related to narrative processing. The way LLMs integrate context over large timescales, however, is fundamentally different from the way the brain does it. In this study, we show that unlike LLMs that apply…
Shailee Jain, Alexander G Huth
Language encoding models help explain language processing in the human brain by learning functions that predict brain responses from the language stimuli that elicited them. Current word embedding-based approaches treat each stimulus word independently and thus ignore the influence of context on language understanding.…
Yaqing Su, Lucy J. MacGregor, Itsaso Olasagasti, Anne-Lise Giraud
Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed on-line remains an open question. Here, we present a model that extracts multiple levels of…
Youssef Oualil, Mittul Singh, Clayton Greenberg, Dietrich Klakow
The goal of language modeling techniques is to capture the statistical and structural properties of natural languages from training corpora. This task typically involves the learning of short range dependencies, which generally model the syntactic properties of a language and/or long range dependencies, which are…
Yangfeng Ji, Trevor Cohn, Lingpeng Kong, Chris Dyer + 1 more
'Jacob Eisenstein'] Text documents are structured on multiple levels of detail: individual words are related by syntax, and larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward…
Harsh V. P. Singh, Qusay H. Mahmoud
A novel approach presented herein transforms the Human Machine Interface (HMI) states, as a pattern of visual feedback states that encompass both operator actions and process states, from a multi-variate time-series to a natural language processing (NLP) modeling domain. The goal of this approach is to predict operator…
Pieter Floris Jacobs, Robert Pollice
Scientists across domains are often challenged to master domain-specific languages (DSLs) for their research, which are merely a means to an end but are pervasive in fields like computational chemistry. Automated code generation promises to overcome this barrier, allowing researchers to focus on their core expertise.…
Ankur Mali, Alexander G. Ororbia, Daniel Kifer, C. Lee Giles
Recurrent neural networks (RNNs) are a widely used deep architecture for sequence modeling, generation, and prediction. Despite success in applications such as machine translation and voice recognition, these stateful models have several critical shortcomings. Specifically, RNNs generalize poorly over very long…
Andrea Gasparetto, Alessandro Zangari, Matteo Marcuzzo, Andrea Albarelli + 1 more
Text Classification methods have been improving at an unparalleled speed in the last decade thanks to the success brought about by deep learning. Historically, state-of-the-art approaches have been developed for and benchmarked against English datasets, while other languages have had to catch up and deal with…
Maksim Gladyshev, Natasha Alechina, Mehdi Dastani, Dragan Doder + 1 more
Models Authors: ['Maksim Gladyshev' 'Natasha Alechina' 'Mehdi Dastani' 'Dragan Doder' 'Brian Logan'] Structural Equation Models (SEM) are the standard approach to representing causal dependencies between variables in causal models. In this paper we propose a new interpretation of SEMs when reasoning about Actual…
Steven T. Piantadosi, Yuan Yang
Our recent work (1) shows a program-learning model can acquire some key structures in natural language, including recursive hierarchies and patterns that require more than context-free capacities (2). Kodner et al.’s (KCY) commentary (3) is based on several fundamental misunderstandings. Most notably, they claim that…
Mathias Peirlinck, Kevin Linka, Juan A. Hurtado, Ellen Kuhl
Constitutive modeling is the cornerstone of computational and structural mechanics. In a finite element analysis, the constitutive model is encoded in the material subroutine, a function that maps local strains onto stresses. This function is called within every finite element, at each integration point, within every…
Authors not listed
The interdisciplinary nature of redox flow batteries (RFBs), spanning chemistry, materials, and engineering, has led to a vast and fragmented body of research, hindering the efficient synthesis of knowledge. An intelligent question-answering system is there-fore essential to organize this dispersed knowledge, enhance…
Sanjar Adilov
Generative neural networks have shown promising results in de novo drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this…
Felix Bowers, Richard Lane, Wahab Kawafi, Danielle Paul + 4 more
Automated segmentation of three-dimensional micro-computed tomography (CT) scan data is a critical bottleneck in computational morphometrics and biomechanical modelling across musculoskeletal biology. Although advances in imaging have generated increasingly large and complex datasets, manual segmentation remains…
Yair Lakretz, Stanislas Dehaene, Jean-Rémi King
Sentence comprehension requires inferring, from a sequence of words, the structure of syntactic relationships that bind these words into a semantic representation. Our limited ability to build some specific syntactic structures, such as nested center-embedded clauses (e.g., “The dog that the cat that the mouse bit…
Authors not listed
Accurately modeling the dynamics of open quantum systems is critical for advancing quantum technologies, yet traditional methods often struggle with balancing accuracy and efficiency. Machine learning (ML) offers a promising alternative, particularly through recursive models that predict system evolution based on the…
Darren J. Edwards, Ciara McEnteggart, Yvonne Barnes-Holmes
Psychology has benefited from an enormous wealth of knowledge about processes of cognition in relation to how the brain organizes information. Within the categorization literature, this behavior is often explained through theories of memory construction called exemplar theory and prototype theory which are typically…
Samuel Genheden, Agnes Mårdh, Gustav Lahti, Ola Engkvist + 2 more
We present machine learning models for predicting the chemical context for Buchwald-Hartwig coupling reactions. Using reaction data from in-house electronic lab notebooks, we train two models: one based on single-label data and one based on multi-label data. Both models show excellent top-3 accuracy around 90%, which…
Rachana Niranjan Murthy, Sai Teja Potu, Akhil Thomas, Lokesh Mishra + 2 more
Retrieving structured materials information from unstructured textual data is essential for data mining and automatically developing comprehensive ontologies. Information extraction is a complex task composed of multiple subtasks and thus often relies on systems of task-specialized language models. A foundation…
Authors not listed
Step-by-step thinking is essential in all domains of chemical sciences and engineering. While machine learning tools are broadly used, algorithms that automate reasoning are far less common. We elaborate on seven categories of human reasoning activities and connect each to applications in chemical science and…