28 papers · ranked by Valyu relevance
Mark A Chia, Fares Antaki, Yukun Zhou, Angus W Turner + 2 more
'Pearse A Keane'] Title: Abstract Foundation models represent a paradigm shift in artificial intelligence (AI), evolving from narrow models designed for specific tasks to versatile, generalisable models adaptable to a myriad of diverse applications. Ophthalmology as a specialty has the potential to act as an exemplar…
Mieko Ochi, Daisuke Komura, Shumpei Ishikawa
Pathology plays a crucial role in diagnosing and evaluating patient tissue samples obtained via surgeries and biopsies. The advent of whole slide scanners and the development of deep learning technologies have considerably advanced this field, promoting extensive research and development in pathology artificial…
Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto + 2 more
'Mark A. Lemley' 'Percy Liang'] Existing foundation models are trained on copyrighted material. Deploying these models can pose both legal and ethical risks when data creators fail to receive appropriate attribution or compensation. In the United States and several other countries, copyrighted content may be used to…
Max S. Y. Lau, C. Jessica E. Metcalf, Zewen Liu, Bryan T. Grenfell + 1 more
Foundation models-large AI systems pretrained on broad, heterogeneous data-are transforming scientific discovery. These models (e.g., GPT, GenCast, AlphaFold) excel at learning generalizable representations and adapting to new tasks with limited data. Yet, epidemic modeling has not experienced a comparable…
Srijan Atti, Shankar Subramaniam
Recent applications of foundation models in biology have focused on pretraining using large-scale single-cell datasets comprising millions of cells, across diverse patho-physiological states. These models are then fine-tuned for downstream tasks such as cell-type classification. In this study, we evaluated the…
Jeremie Kalfon, Laura Cantini, Gabriel Peyre
We have reached a point where many bio foundation models exist across 4 different scales, from molecules to molecular chains, cells, and tissues. However, while related in many ways, these models do not yet bridge these scales. We present a framework and architecture called Xpressor that enables cross-scale learning by…
Daniel Truhn, Jan-Niklas Eckardt, Dyke Ferber, Jakob Nikolas Kather
The technological progress in artificial intelligence (AI) has massively accelerated since 2022, with far-reaching implications for oncology and cancer research. Large language models (LLMs) now perform at human-level competency in text processing. Notably, both text and image processing networks are increasingly based…
Franck Le, Mudhakar Srivatsa, Raghu Ganti, Vyas Sekar
Foundational models have caused a paradigm shift in the way artificial intelligence (AI) systems are built. They have had a major impact in natural language processing (NLP), and several other domains, not only reducing the amount of required labeled data or even eliminating the need for it, but also significantly…
Kasia Z. Kedzierska, Lorin Crawford, Ava P. Amini, Alex X. Lu
The advent and success of foundation models such as GPT has sparked growing interest in their application to single-cell biology. Models like Geneformer and scGPT have emerged with the promise of serving as versatile tools for this specialized field. However, the efficacy of these models, particularly in zero-shot…
Masahiro Oda
In recent years, generative AI has attracted significant public attention, and its use has been rapidly expanding across a wide range of domains. From creative tasks such as text summarization, idea generation, and source code generation, to the streamlining of medical support tasks like diagnostic report generation…
Kasia Z. Kedzierska, Lorin Crawford, Ava P. Amini, Alex X. Lu
Foundation models such as scGPT and Geneformer have not been rigorously evaluated in a setting where they are used without any further training (i.e., zero-shot). Understanding the performance of models in zero-shot settings is critical to applications that exclude the ability to fine-tune, such as discovery settings…
William G. Coon, Mattson Ogg
Accurate sleep assessment is critical to the practice of sleep medicine and sleep research. The recent availability of large quantities of publicly available sleep data, alongside recent breakthroughs in AI like transformer architectures, present novel opportunities for data-driven discovery efforts. Transformers are…
Russell Dinnage
1. Species distribution models (SDMs) are crucial tools for understanding and predicting biodiversity patterns, yet they often struggle with limited data, biased sampling, and complex species-environment relationships. Here I present NicheFlow, a novel foundation model for SDMs that leverages generative AI to address…
Junho Song, Jong-Hwan Jang, Byeong Tak Lee, DongGyun Hong + 2 more
'Joon-myoung Kwon' 'Yong-Yeon Jo'] Foundation models, enhanced by self-supervised learning (SSL) techniques, represent a cutting-edge frontier in biomedical signal analysis, particularly for electrocardiograms (ECGs), crucial for cardiac health monitoring and diagnosis. This study conducts a comprehensive analysis of…
Daniel J. B. Clarke, Giacomo B. Marino, Avi Ma’ayan
Trained with large datasets, foundation models can capture complex patterns within these datasets to create embeddings that can be used for a variety of useful applications. Here we created a gene set foundation model that was trained on a massive collection of unlabeled gene sets from two databases: Rummagene and…
Xingcun Fan, Wenbin Liao, Luchi Xiao, Xuefeng Yan + 1 more
Pre-trained large models have emerged as a pivotal technological approach for foundational cell modeling. However, existing deep learning-based foundational models for cells have predominantly focused on human or murine systems, with a relative scarcity of research on model microorganisms such as Saccharomyces…
Zhao, Yiqin, Tian Guo
Mobile sensing systems have long faced a fundamental tradeoff between sensing quality and efficiency due to constraints in computation, power, and other limitations. Sparse sensing, which aims to acquire and process only a subset of sensor data, has been a key strategy for maintaining performance under such…
Abdel Rahman Alsabbagh, Alberto Maillo Ruiz de Infante, David Gomez-Cabrero, Narsis A. Kiani + 2 more
With the emergence of single-cell foundation models, an important question arises: how do these models perform when trained on datasets having an imbalance in cell type distribution due to rare cell types or biased sampling? We benchmark three foundation models, scGPT, scBERT, and Geneformer, using skewed single-cell…
Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, Berend Smit
Machine learning has revolutionized many fields and has recently found applications in chemistry and materials science. The small datasets commonly found in chemistry sparked the development of sophisticated machine-learning approaches that incorporate chemical knowledge for each application and, therefore, require…
Authors not listed
The rapid advancement of machine learning in computational chemistry has opened new doors for designing molecules, predicting molecular properties, and discovering novel materials. However, building scalable and robust models for molecular property prediction remains a significant challenge due to the vast size and…
Keisuke Ozawa, Teppei Suzuki, Shunsuke Tonogai, Tomoya Itakura
Developing foundation models for materials science has attracted attention. However, there is a lack of studies on inorganic materials due to the difficulty in the comprehensive representation of geometric concepts composing crystals: local atomic environments, their connections, and the global symmetries. We present a…
Keisuke Ozawa, Teppei Suzuki, Shunsuke Tonogai, Tomoya Itakura
Developing foundation models for materials science has attracted attention. However, there is a lack of studies on inorganic materials due to the difficulty in the comprehensive representation of geometric concepts composing crystals: local atomic environments, their connections, and the global symmetries. We present a…
Authors not listed
Recent years have seen a growing interest in machine learning approaches for chemical tasks. The best existing methods focus on building base models that combine molecular graphs (“2D structures”) with atomic coordinates in 3D to predict molecular properties, typically through pre-training followed by fine-tuning on…
Authors not listed
Supervised deep learning has become a standard approach to deliver competitive predictive tools that allow relating the structure of molecules and their physicochemical features to properties such as binding to protein targets, performance as electronic materials, and reactivity. However, efforts to understand how…
Tianfan Jin, Brett M Savoie
Contemporary machine learning algorithms have largely succeeded in automating the development of mathematical models from data. Although this is a striking accomplishment, it leaves unaddressed the multitude of scenarios, especially across the chemical sciences and engineering, where deductive, rather than inductive…
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen + 39 more
'Ganqu Cui' 'Zheni Zeng' 'Xuanhe Zhou' 'Y.-Y. Huang' 'Chaojun Xiao' 'Chi Han' 'Yi Fung' 'Yusheng Su' 'Huadong Wang' 'Cheng Qian' 'Runchu Tian' 'Kunlun Zhu' 'Shihao Liang' 'Xingyu Shen' 'Bokai Xu' 'Zhen Zhang' 'Yining Ye' 'Bowen Li' 'Ziwei Tang' 'Jing Yi' 'Yuzhang Zhu' 'Zhenning Dai' 'Yan Lan' 'Xin Cong' 'Yaxi Lu'…
Freddy Kamdem Simo, Dominique Ernadote, Dominique Lenné
In a Systems Engineering setting, various models are produced using a variety of methods and tools. Focusing on a type of models – called descriptive models – which we shall describe, we argue that, while the clarity and precision of models are essential for their exchange and reuse, the way in which the data of these…
Vahid Moosavi
We discuss that how the majority of traditional modeling approaches are following the idealism point of view in scientific modeling, which follow the set theoretical notions of models based on abstract universals. We show that while successful in many classical modeling domains, there are fundamental limits to the…