21 papers · ranked by Valyu relevance
Simin Fan, Maria Ios Glarou, Martin Jaggi
The performance of large language models (LLMs) across diverse downstream applications is fundamentally governed by the quality and composition of their pretraining corpora. Existing domain reweighting algorithms primarily optimize data mixtures for a single target task, thereby resulting in models that overfit to…
Ashwini B, Arka Sarkar, Pruthivi Raj Behera, Jainendra Shukla
Deep learning techniques have proven to be effective in solving the facial emotion recognition (FER) problem. However, it demands a significant amount of supervision data which is often unavailable due to privacy and ethical concerns. In this paper, we present a novel approach for addressing the FER problem using…
Xuancheng Huang, Jingfang Xu, Maosong Sun, Yang Liu
Multi-source sequence generation (MSG) is an important kind of sequence generation tasks that takes multiple sources, including automatic post-editing, multi-source translation, multi-document summarization, etc. As MSG tasks suffer from the data scarcity problem and recent pretrained models have been proven to be…
Authors not listed
Equivariant graph neural networks have shown remarkable success in molecular property prediction, but their performance on novel molecular geometries remains limited without extensive training data. We present a computationally efficient approach to cross-geometry pretraining for molecular systems that improves…
Piotr Teterwak, Kuniaki Saito, Theodoros Tsiligkaridis, Bryan A. Plummer + 1 more
'Bryan A. Plummer' 'Kate Saenko'] Multi-Source Domain Generalization (DG) is the task of training on multiple source domains and achieving high classification performance on unseen target domains. Recent methods combine robust features from web-scale pretrained backbones with new features learned from source data, and…
Ling Ge, Chunming Hu, Guanghui Ma, Jihong Liu + 1 more
Cross-Lingual Transfer Learning Authors: ['Ling Ge' 'Chunming Hu' 'Guanghui Ma' 'Jihong Liu' 'Hong Zhang'] Multi-Source cross-lingual transfer learning deals with the transfer of task knowledge from multiple labelled source languages to an unlabeled target language under the language shift. Existing methods typically…
Seongmin Lee, Hyunsik Jeon, U. Kang, Chi-Hua Chen
Given multiple source datasets with labels, how can we train a target model with no labeled data? Multi-source domain adaptation (MSDA) aims to train a model using multiple source datasets different from a target dataset in the absence of target data labels. MSDA is a crucial problem applicable to many practical cases…
Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang + 2 more
'Xiang Ren'] Sequence labeling is a fundamental framework for various natural language processing problems. Its performance is largely influenced by the annotation quality and quantity in supervised learning scenarios, and obtaining ground truth labels is often costly. In many cases, ground truth labels do not exist…
Yajie Li, Qichang Zhao, Jianxin Wang
Accurate prediction of molecular properties is essential for accelerating drug discovery. While graph neural networks (GNNs) have achieved impressive progress, most models rely on atomic graphs with limited chemical semantics, leading to suboptimal generalization across diverse biochemical tasks. Pretraining alleviates…
Authors not listed
Alzheimer’s disease (AD) is one of the most common progressive neurodegenerative diseases, and the number of AD patients has increased year after year with the global aging trend. The onset of AD has a long preclinical stage. If doctors can make an initial diagnosis in the mild cognitive impairment (MCI) stage, it is…
Authors not listed
The rapid advancement of machine learning in computational chemistry has opened new doors for designing molecules, predicting molecular properties, and discovering novel materials. However, building scalable and robust models for molecular property prediction remains a significant challenge due to the vast size and…
Mittul Singh, Peter Smit, Sámi Virpioja, Mikko Kurimo
Character-based Neural Network Language Models (NNLM) have the advantage of smaller vocabulary and thus faster training times in comparison to NNLMs based on multi-character units. However, in low-resource scenarios, both the character and multi-character NNLMs suffer from data sparsity. In such scenarios…
Aidan Dempster, Brokoslaw Laschowski
A grand challenge in brain decoding is to develop algorithms that generalize across multiple subjects and tasks. Here, we developed a new computational framework to minimize negative transfer for domain-adaptive brain decoding by reframing source selection as a mixture model parameter estimation problem, allowing each…
Jiang Guo, Darsh Shah, Regina Barzilay
We propose a mixture-of-experts approach for unsupervised domain adaptation from multiple sources. The key idea is to explicitly capture the relationship between a target example and different source domains. This relationship, expressed by a point-to-set metric, determines how to combine predictors trained on various…
Tianyu Liu, Tinyi Chu, Xiao Luo, Hongyu Zhao
Drug synergy prediction is a challenging and important task in the treatment of complex diseases including cancer. In this manuscript, we present a novel Foundation Model, known as BAITSAO, for tasks related to drug synergy prediction with a unified pipeline to handle different datasets. We construct the training…
Hosein Fooladi, Steffen Hirte, Johannes Kirchmair
Today, machine learning methods are widely employed in drug discovery. However, the chronic lack of data continues to hamper their further development, validation, and application. Several modern strategies aim to mitigate the challenges associated with data scarcity by learning from data on related tasks. These…
Authors not listed
The accurate prediction of fuel mixture properties is essential for the development of alternative fuels, yet remains challenging under data-scarce conditions due to the combinatorial complexity of multi-component systems. In this study, we present a systematic evaluation of three machine learning (ML)…
Fan Hu, Yishen Hu, Weihong Zhang, Huazhen Huang + 2 more
Proteins are the building blocks of life, carrying out fundamental functions in biology. In computational biology, an effective protein representation facilitates many important biological quantifications. Most existing protein representation methods are derived from self-supervised language models designed for text…
Han Li, Xinyi Zhao, Shuya Li, Fangping Wan + 2 more
Understanding the molecular properties (e.g., physical, chemical or physiological characteristics and biological activities) of small molecules plays essential roles in biomedical researches. The accumulating amount of datasets has enabled the development of data-driven computational methods, especially the machine…
Lilan Liu, Xiang Wan, Jiaying Li, Wenxi Wang + 5 more
'Long Wen' 'Haidong Shao' 'Xinyu Li' 'Zhuyun Chen'] Due to the rapid development of industrial internet technology, the traditional manufacturing industry is in urgent need of digital transformation, and one of the key technologies to achieve this is multi-source data fusion. For this problem, this paper proposes an…
John M Giorgi, Gary D Bader
The explosive increase of biomedical literature has made information extraction an increasingly important tool for biomedical research. A fundamental task is the recognition of biomedical named entities (NER) such as genes/proteins, diseases, and species. Recently, a domain-independent method based on deep learning and…