29 papers · ranked by Valyu relevance
Arkaitz Zubiaga
Since their inception in the 1980s, language models (LMs) have been around for more than four decades as a means for statistically modeling the properties observed from natural language (Rosenfeld, ). Given a collection of texts as input, a language model computes statistical properties of language from those texts…
Yi Dong, Ronghui Mu, Yanghao Zhang, Siqi Sun + 8 more
In the burgeoning field of Large Language Models (LLMs), developing a robust safety mechanism, colloquially known as “safeguards” or “guardrails”, has become imperative to ensure the ethical use of LLMs within prescribed boundaries. This article provides a systematic literature review on the current status of this…
Gonzalo Martínez, Javier Conde, Elena Merino-Gómez, Beatriz Bermúdez-Margaretto + 4 more
'Beatriz Bermúdez-Margaretto' 'José Alberto Hernández' 'Pedro Reviriego' 'Marc Brysbaert' 'Jessie S. Barrot'] Vocabulary tests, once a cornerstone of language modeling evaluation, have been largely overlooked in the current landscape of Large Language Models (LLMs) like Llama 2, Mistral, and GPT. While most LLM…
Daniel S. Roll, Zeyneb Kurt, Yulei Li, Wai Lok Woo + 1 more
This preliminary study covers the construction and application of a Graph-based Retrieval-Augmented Generation (GraphRAG) system integrating a multimodal LLM, Large Language and Vision Assistant (LLaVA) with graph database software (Neo4j) to enhance LLM output quality through structured knowledge retrieval. This is…
Javier Conde, María Grandury, Tairan Fu, Carlos Arriaga + 6 more
Word-level psycholinguistic norms are necessary to test theories of language processing. However, obtaining such human-based measures is not always feasible or straightforward. One promising approach is to augment human norming datasets by using large language models (LLMs) to predict these characteristics directly, a…
Zhuoqiao Hong, Haocheng Wang, Zaid Zada, Harshvardhan Gazula + 12 more
Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have rapidly grown in complexity, leading to improved language processing capabilities. However, neuroscience researchers haven’t kept up with the quick progress in LLM…
İbrahim ÖZKAL, Fatih BAŞÇİFTÇİ
The metaverse refers to a digital environment that enables real-time user interaction through immersive technologies. Recent advancements in deep learning have particularly strengthened the capabilities of Natural Language Processing (NLP) and Large Language Models (LLMs). These developments have made human-computer…
Yuanhao Gong
—Large language models have made significant progress in the past few years. However, they are either generic or field specific, splitting the community into different groups. In this paper, we unify these large language models into a larger map, where the generic and specific models are linked together and can improve…
Zhibo Chu, Shiwen Ni, Zichong Wang, Xi Feng + 5 more
Introductory Survey Authors: ['Zhibo Chu' 'Shiwen Ni' 'Zichong Wang' 'Xi Feng' 'Chengming Li' 'Xiping Hu' 'Ruifeng Xu' 'Min Yang' 'Wenbin Zhang'] Language models serve as a cornerstone in natural language processing (NLP), utilizing mathematical methods to generalize language laws and knowledge for prediction and…
Giorgio Valentini, Dario Malchiodi, Jessica Gliozzo, Marco Mesiti + 5 more
'Mauricio Soto-Gomez' 'Alberto Cabri' 'Justin Reese' 'Elena Casiraghi' 'Peter N. Robinson'] The recent breakthroughs of Large Language Models (LLMs) in the context of natural language processing have opened the way to significant advances in protein research. Indeed, the relationships between human natural language and…
Authors not listed
Artificial intelligence (AI) is reshaping scientific research by accelerating discovery and enabling the analysis of complex data that traditional methods struggle to handle. This review examines over 310,000 journal articles and patents from the CAS Content Collection (2015–2025), with a focus on, biomedical research…
Wazir Ali, Sampo Pyysalo
Large Language Models (LLMs) have gained significant attention due to their high performance on a wide range of natural language tasks since the release of ChatGPT. The LLMs learn to understand and generate language by training billions of model parameters on vast volumes of text data. Despite being a relatively new…
Rick Rejeleene, Xiaowei Xu, John R. Talburt
Large language models (LLM) are generating information at a rapid pace, requiring users to increasingly rely and trust the data. Despite remarkable advances of LLM, Information generated by LLM is not completely trustworthy, due to challenges in information quality. Specifically, integrity of Information quality…
Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller + 6 more
In this article, we introduce a sociolinguistic perspective on language modeling. We claim that language models in general are inherently modeling varieties of language, and we consider how this insight can inform the development and deployment of language models. We begin by presenting a technical definition of the…
Changjiang Gao, Zhengwu Ma, Jiajun Chen, Ping Li + 2 more
Transformer-based large language models (LLMs) have significantly advanced our understanding of meaning representation in the human brain. However, increasingly large LLMs have been questioned as valid cognitive models due to their extensive training data and their ability to access context thousands of words long. In…
Chunhe Ni, Jiang Wu, Hongbo Wang, Wenran Lu + 1 more
and Transformer Models Authors: ['Chunhe Ni' 'Jiang Wu' 'Hongbo Wang' 'Wenran Lu' 'Chenwei Zhang'] Large Language Models (LLMs) are a class of generative AI models built using the Transformer network, capable of leveraging vast datasets to identify, summarize, translate, predict, and generate language. LLMs promise to…
Mahrad Almotahari
Cooperative speech is purposive. From the speaker's perspective, one crucial purpose is the transmission of knowledge. Cooperative speakers care about getting things right for their conversational partners. This attitude is a kind of respect. Cooperative speech is an ideal form of communication because participants…
Authors not listed
Step-by-step thinking is essential in all domains of chemical sciences and engineering. While machine learning tools are broadly used, algorithms that automate reasoning are far less common. We elaborate on seven categories of human reasoning activities and connect each to applications in chemical science and…
Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, Berend Smit
Machine learning has revolutionized many fields and has recently found applications in chemistry and materials science. The small datasets commonly found in chemistry lead to various sophisticated machine-learning approaches that incorporate chemical knowledge for each application and therefore require a lot of…
Authors not listed
Large Language Models (LLMs) based on transformer architectures excel at internet-scale tasks. However, real-world scientific scenarios—such as synthetic chemistry laboratories and autonomous experimental setups—typically involve incremental data generation in batches as new chemical reactions are conducted, unlike…
Hlib Babii, Andrea Janes, Romain Robbes
When building machine learning models that operate on source code, several decisions have to be made to model source-code vocabulary. These decisions can have a large impact: some can lead to not being able to train models at all, others significantly affect performance, particularly for Neural Language Models. Yet…
James Fodor, Carsten Murawski, Shinsuke Suzuki
Large language models based on the transformer architecture are now capable of producing human-like language. But do they encode and process linguistic meaning in a human-like way? Here, we address this question by analysing 7T fMRI data from 30 participants reading 108 sentences each. These sentences are carefully…
Adrielli Lopes Rego, Joshua Snell, Martijn Meeter
Although word predictability is commonly considered an important factor in reading, sophisticated accounts of predictability in theories of reading are yet lacking. Computational models of reading traditionally use cloze norming as a proxy of word predictability, but what cloze norms precisely capture remains unclear.…
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi + 3 more
Massive scale, both in terms of data availability and computation, enables significant breakthroughs in key application areas of deep learning such as natural language processing (NLP) and computer vision. There is emerging evidence that scale may be a key ingredient in scientific deep learning, but the importance of…
Authors not listed
Accelerating computational materials science relies not only on hardware advances but also on software that increases the ease of working with the relevant abstractions. Creation and manipulation of crystal structures is a part of many routine materials science workflows. In this work, we demonstrate how fine tuning…
Anantharaman Palacode Narayana Iyer
— Statistical language models are central to many applications that use semantics. Recurrent Neural Networks (RNN) are known to produce state of the art results for language modelling, outperforming their traditional n-gram counterparts in many cases. To generate a probability distribution across a vocabulary, these…
Lucas Busta, Alan R. Oyler
Transformer-based large language models are receiving considerable attention because of their ability to analyze scientific literature. Small language models (SLMs), however, also have potential in this area, have smaller compute footprints, and allow users to keep data in-house. Here, we quantitatively evaluate the…
Kristijan Armeni, Roel M. Willems, Stefan Frank
Cognitive neuroscientists of language comprehension study how neural computations relate to cognitive computations during comprehension. On the cognitive part of the equation, it is important that the computations and processing complexity are explicitly defined. Probabilistic language models can be used to give a…
Sanjar Adilov
Generative neural networks have shown promising results in de novo drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this…