Search · four archives
Search · four archives
28 papers · ranked by Valyu relevance
William Dee
Antimicrobial peptides (AMPs) are increasingly being used in the development of new therapeutic drugs, in areas such as cancer therapy and hypertension. Additionally, they are seen as an alternative to antibiotics due to the increasing occurrence of bacterial resistance. Wet-laboratory experimental identification…
Chengwei Wei, Yun-Cheng Wang, Bin Wang, C.‐C. Jay Kuo
Language modeling studies the probability distributions over strings of texts. It is one of the most fundamental tasks in natural language processing (NLP). It has been widely used in text generation, speech recognition, machine translation, etc. Conventional language models (CLMs) aim to predict the probability of…
Felipe Maia Polo, Gabriel Caiaffa Floriano Mendonça, Kauê Capellato J. Parreira, Lucka Gianvechio + 5 more
'Kauê Capellato J. Parreira' 'Lucka Gianvechio' 'Peterson Cordeiro' 'Jonathan Batista Ferreira' 'Leticia Maria Paz de Lima' 'Antônio Carlos do Amaral Maia' 'Renato Vicente'] Abstract. We present and make available pre-trained language models (Phraser, Word2Vec, Doc2Vec, FastText, and BERT) for the Brazilian legal…
Zak Hussain, Marcel Binz, Rui Mata, Dirk U. Wulff
Large language models (LLMs) have the potential to revolutionize behavioral science by accelerating and improving the research cycle, from conceptualization to data analysis. Unlike closed-source solutions, open-source frameworks for LLMs can enable transparency, reproducibility, and adherence to data protection…
Christian Blüthgen
Hintergrund Große Sprachmodelle (Large Language Models, LLMs) wie ChatGPT haben die Art und Weise, wie Computer menschliche Sprache analysieren können und wie wir mit Computern interagieren können, in nur kurzer Zeit revolutioniert. Fragestellung Überblick über die Entstehung und die Grundprinzipien von…
Mohamad Ballout, Ulf Krumnack, Gunther Heidemann, Kai‐Uwe Kühnberger
Pre-trained language models have recently emerged as a powerful tool for fine-tuning a variety of language tasks. Ideally, when models are pre-trained on large amount of data, they are expected to gain implicit knowledge. In this paper, we investigate the ability of pre-trained language models to generalize to…
Tyler A. Chang, Benjamin Bergen
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax…
Yanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier + 3 more
In recent years, pre-trained language models (PLMs) achieve the best performance on a wide range of natural language processing (NLP) tasks. While the first models were trained on general domain data, specialized ones have emerged to more effectively treat specific domains. In this paper, we propose an original study…
Dongqiu Zhang, Wenkui Li
Natural Language Understanding (NLU) and Natural Language Generation (NLG) are the general methods that support machine understanding of text content. They play a very important role in the text information processing system including recommendation and question and answer systems. There are many researches in the…
Adrielli Lopes Rego, Joshua Snell, Martijn Meeter
Although word predictability is commonly considered an important factor in reading, sophisticated accounts of predictability in theories of reading are yet lacking. Computational models of reading traditionally use cloze norming as a proxy of word predictability, but what cloze norms precisely capture remains unclear.…
Arkaitz Zubiaga
Since their inception in the 1980s, language models (LMs) have been around for more than four decades as a means for statistically modeling the properties observed from natural language (Rosenfeld, ). Given a collection of texts as input, a language model computes statistical properties of language from those texts…
Ziqi Tang, Nirali Somia, Yiyang Yu, Peter K Koo
The emergence of genomic language models (gLMs) offers an unsupervised approach to learning a wide diversity of cis-regulatory patterns in the non-coding genome without requiring labels of functional activity generated by wet-lab experiments. Previous evaluations have shown that pre-trained gLMs can be leveraged to…
Wenduo Cheng, Junhong Shen, Mikhail Khodak, Jian Ma + 1 more
Pre-trained language models have transformed the field of natural language processing (NLP), and their success has inspired efforts in genomics to develop domain-specific foundation models (FMs). However, creating high-quality genomic FMs from scratch is resource-intensive, requiring significant computational power and…
Raphaël Millière
This chapter critically examines the potential contributions of modern language models to theoretical linguistics. Despite their focus on engineering goals, these models' ability to acquire sophisticated linguistic knowledge from mere exposure to data warrants a careful reassessment of their relevance to linguistic…
José Alberto Benítez-Andrades, Álvaro González-Jiménez, Álvaro López-Brea, Jose Aveleira-Mata + 3 more
'Álvaro López-Brea' 'Jose Aveleira-Mata' 'José-Manuel Alija-Pérez' 'María Teresa García-Ordás' 'Carlos Fernandez-Lozano'] With the growth that social networks have experienced in recent years, it is entirely impossible to moderate content manually. Thanks to the different existing techniques in natural language…
Michael R. Douglas
Artificial intelligence is making spectacular progress, and one of the best examples is the development of large language models (LLMs) such as OpenAI's GPT series. In these lectures, written for readers with a background in mathematics or physics, we give a brief history and survey of the state of the art, and…
Ningyuan You, Chang Liu, Hai Lin, Sai Wu + 2 more
RNA plays a pivotal role in diverse cellular functions across organisms. Developing computational algorithms for RNA sequence related questions is highly valuable. Recently, genomic language models (gLMs) with pre-training have emerged, offering flexibility for various downstream prediction tasks. However…
Ken Chen, Yue Zhou, Maolin Ding, Yu Wang + 2 more
RNA splicing is an important post-transcriptional process of gene expression in eukaryotic cells. Predicting RNA splicing from primary sequences can facilitate the interpretation of genomic variants. In this study, we developed a novel self-supervised pre-trained language model, SpliceBERT, to improve sequence-based…
Richard Antonello, Alexander Huth
Many recent studies have shown that representations drawn from neural network language models are extremely effective at predicting brain responses to natural language. But why do these models work so well? One proposed explanation is that language models and brains are similar because they have the same objective: to…
Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller + 6 more
In this article, we introduce a sociolinguistic perspective on language modeling. We claim that language models in general are inherently modeling varieties of language, and we consider how this insight can inform the development and deployment of language models. We begin by presenting a technical definition of the…
Authors not listed
Predicting molecular properties is a key challenge in drug discovery. Machine learning models, especially those based on transformer architectures, are increasingly used to make these predictions from chemical structures. Inspired by recent progress in natural language processing, many studies have adopted encoder-only…
Andrew Blanchard, Pei Zhang, Debsindhu Bhowmik, Kshitij Mehta + 4 more
Efficient methods for searching the chemical space of molecular compounds are needed to automate and accelerate the design of new functional molecules such as pharmaceuticals. Given the high cost in both resources and time for experimental efforts, computational approaches play a key role in guiding the selection of…
Sylwia Nowakowska
Two versions of Large Language ChemBERTa-2 models, pre-trained with two different methods, were fine-tuned in this work for HIV replication inhibition prediction. The best model achieved AUROC of 0.793. The changes in distributions of molecular embeddings prior to and following fine-tuning reveal models’ enhanced…
Authors not listed
Drug Discovery is a very lengthy and resource-consuming process. However, a variety of advanced Artificial Intelligence (AI) and Deep Learning (DL) techniques are being utilized to accelerate and advance DD, such as Large Language Models (LLMs). This survey is in aim of discovering and comparing the currently available…
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi + 3 more
Massive scale, both in terms of data availability and computation, enables significant breakthroughs in key application areas of deep learning such as natural language processing (NLP) and computer vision. There is emerging evidence that scale may be a key ingredient in scientific deep learning, but the importance of…
Authors not listed
Accelerating computational materials science relies not only on hardware advances but also on software that increases the ease of working with the relevant abstractions. Creation and manipulation of crystal structures is a part of many routine materials science workflows. In this work, we demonstrate how fine tuning…
Authors not listed
Large language models (LLMs) have garnered increasing attention owing to their potential as collaborative assistants in scientific studies. However, adapting an LLM to specialized domains remains challenging because of the difficulty in incorporating domain-specific knowledge. In the present study, we propose a…
Dimitris Gkoumas, Maria Liakata
The intersection of chemistry and Artificial Intelligence (AI) is an active area of research focused on accelerating scientific discovery. While using large language models (LLMs) with scientific modalities has shown potential, there are significant challenges to address, such as improving training efficiency and…