25 papers · ranked by Valyu relevance
Michael Heinzinger, Maria Littmann, Ian Sillitoe, Nicola Bordin + 2 more
Experimental structures are leveraged through multiple sequence alignments, or more generally through homology-based inference (HBI), facilitating the transfer of information from a protein with known annotation to a query without any annotation. A recent alternative expands the concept of HBI from sequence-distance…
Saeed Shurrab, Rehab Duwairi, Jiachen Yang
The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need…
Zhenyuan Lu
To improve performance in visual feature representation from photos or videos for practical applications, we generally require large-scale human-annotated labeled data while training deep neural networks. However, the cost of gathering and annotating human-annotated labeled data is expensive. Given that there is a lot…
Bihi Sabiri, Amal Khtira, Bouchra El Asri, Maryem Rhanoui + 2 more
'Raimondo Schettini' 'Donald Bailey'] In recent years, contrastive learning has been a highly favored method for self-supervised representation learning, which significantly improves the unsupervised training of deep image models. Self-supervised learning is a subset of unsupervised learning in which the learning…
Valentino Vito, Lim Yohanes Stefanus, Andrea Prati, Luis Javier García Villalba + 1 more
'Luis Javier García Villalba' 'Vincent A. Cicirello'] Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process is typically conducted using a two-stage…
Xuemei Bai, Yuqing Zhang, Chenjie Zhang, Zhijun Wang + 1 more
Person re-identification (ReID) technology has many applications in intelligent surveillance and public safety. However, the domain difference between the source and target domains makes the generalization ability of the model extremely challenging. To reduce the dependence on labeled data, Unsupervised Domain…
Carlo Alberto Barbano, Benoit Dufumier, Enzo Tartaglione, Marco Grangetto + 1 more
'Marco Grangetto' 'Pietro Gori'] Many datasets are biased, namely they contain easy-to-learn features that are highly correlated with the target class only in the dataset but not in the true underlying distribution of the data. For this reason, learning unbiased models from biased data has become a very relevant…
Martin J. Falk, Adam T. Strupp, Benjamin Scellier, Arvind Murugan
The backpropagation method has enabled transformative uses of neural networks. Alternatively, for energy-based models, local learning methods involving only nearby neurons offer benefits in terms of decentralized training, and allow for the possibility of learning in computationally-constrained substrates. One class of…
Phuong-Nam Nguyen
Contrastive learning has emerged as a pivotal technique in representation learning, particularly for self-supervised and unsupervised tasks. Link prediction, crucial for network analysis, forecasts the formation of connections between nodes. Machine learning enhances link prediction by learning patterns from data…
Kisung Moon, Hyeon-Jin Im, Sunyoung Kwon
Self-supervised learning (SSL) is a method that learns the data representation by utilizing supervision inherent in the data. This learning method is in the spotlight in the drug field, lacking annotated data due to time-consuming and expensive experiments. SSL using enormous unlabeled data has shown excellent…
Filip Thor, Carl Nettelblad
We introduce a framework for using contrastive learning for dimensionality reduction on genetic datasets to create PCA-like population visualizations. Contrastive learning is a self-supervised deep learning method that uses similarities between samples to train the neural network to discriminate between samples. Many…
Menachem Stern, Andrea J. Liu, Vijay Balasubramanian
Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform specific tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active…
Fabian Mager, Hiba Nassar, Lars Kai Hansen
We study the modality gap in CLIP-style dual-encoder contrastive learning, where image and text embeddings remain misaligned despite being trained in a shared space. We argue that the gap is induced by a failure of the InfoNCE formulation with independent encoders. We conduct a uni-modal experiment with two independent…
Wei Huang, Andi Han, Yongqiang Chen, Yuan Cao + 2 more
Learning Authors: ['Wei Huang' 'Andi Han' 'Yongqiang Chen' 'Yuan Cao' 'Zhiqiang Xu' 'Taiji Suzuki'] Multi-modal contrastive learning with language supervision has presented a paradigm shift in modern machine learning. By pre-training on a web-scale dataset, multi-modal contrastive learning can learn high-quality…
Hangyu Lin, Chen Liu, Chengming Xu, Zhengqi Gao + 2 more
'Yuan Yao'] Cross-modality distillation arises as an important topic for data modalities containing limited knowledge such as depth maps and high-quality sketches. Such techniques are of great importance, especially for memory and privacy-restricted scenarios where labeled training data is generally unavailable. To…
Yan Han, Soumava Kumar Roy, Mehrtash Harandi, Lars Petersson
Deep learning with noisy labels is an interesting challenge in weakly supervised learning. Despite their significant learning capacity, CNNs have a tendency to overfit in the presence of samples with noisy labels. Alleviating this issue, the well known Co-Training framework is used as a fundamental basis for our work.…
Authors not listed
Machine learning approaches for conceptualizing and designing in silico compounds have attracted significant attention. However, the applicability of these compounds is often challenged by synthetic viability and cost-effectiveness. Researchers introduced proxy-scores, known as synthethic accessiblity scoring, to…
Rouzbeh Meshkinnejad, Jie Mei, Daniel J. Lizotte, Yalda Mohsenzadeh
Contrastive representation learning has emerged as a promising technique for continual learning as it can learn representations that are robust to catastrophic forgetting and generalize well to unseen future tasks. Previous work in continual learning has addressed forgetting by using previous task data and trained…
Authors not listed
Recent years have seen a growing interest in machine learning approaches for chemical tasks. The best existing methods focus on building base models that combine molecular graphs (“2D structures”) with atomic coordinates in 3D to predict molecular properties, typically through pre-training followed by fine-tuning on…
Authors not listed
Machine learning models are increasingly applied to heterogeneous materials datasets spanning different synthesis routes, measurement protocols, and structural classes. Although multi-task and representation-learning approaches are commonly used to improve predictive performance, the latent representations learned by…
Xinqiang Ding
Despite force field improvements over the past decades, we still encounter situations where simulation results disagree with experiments due to force field inaccuracies. Such situations provide opportunities to improve force fields. In this study, we introduce a novel framework for optimizing force fields using…
Joseph Scott German, Guofeng Cui, Chenliang Xu, Robert A. Jacobs + 1 more
'Ming Bo Cai'] We propose the “runtime learning” hypothesis which states that people quickly learn to perform unfamiliar tasks as the tasks arise by using task-relevant instances of concepts stored in memory during mental training. To make learning rapid, the hypothesis claims that only a few class instances are used…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…
Authors not listed
Large Language Models (LLMs) based on transformer architectures excel at internet-scale tasks. However, real-world scientific scenarios—such as synthetic chemistry laboratories and autonomous experimental setups—typically involve incremental data generation in batches as new chemical reactions are conducted, unlike…
Mohammad Rostami, Aram Galstyan
Humans continually expand their learned knowledge to new domains and learn new concepts without any interference with past learned experiences. In contrast, machine learning models perform poorly in a continual learning setting, where input data distribution changes over time. Inspired by the nervous system learning…