26 papers · ranked by Valyu relevance
Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min, Sungjun Cho + 3 more
'Honglak Lee' 'Seunghoon Hong'] We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice. Given a graph, we simply treat all nodes and edges as independent tokens, augment them with token embeddings, and feed them to a…
Sanggeon Yun, Raheeb Hassan, Ryozo Masukawa, Sungheon Jeong + 1 more
Graph Transformers typically rely on explicit positional or structural encodings and dense global attention to incorporate graph topology. In this work, we show that neither is essential. We introduce HopFormer, a graph Transformer that injects structure exclusively through head-specific n-hop masked sparse attention…
Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Philip H. S. Torr + 1 more
'Mark Coates'] Transformers have attained outstanding performance across various modalities, owing to their simple but powerful scaled-dot-product (SDP) attention mechanisms. Researchers have attempted to migrate Transformers to graph learning, but most advanced Graph Transformers (GTs) have strayed far from plain…
Thang Chu, Tuan Nguyen
Previous models have shown that learning drug features from their graph representation is more efficient than learning from their strings or numeric representations. Furthermore, integrating multi-omics data of cell lines increases the performance of drug response prediction. However, these models showed drawbacks in…
Jingtao Hu, Yi Zhang, Chengzhang Zhu, Changsheng Hou + 1 more
Attributed graphs have recently emerged as a powerful tool for representing diverse data in numerous real-world sensors. Among various applications, unsupervised graph anomaly detection (UGAD) aims to identify abnormal data that significantly deviate from the majority of normal nodes without label annotations. Hence…
Xin Lai, Yang Liu, Rui Qian, Yong Lin + 2 more
'Kinkar Chandra Das'] Graph-structured data, operating as an abstraction of data containing nodes and interactions between nodes, is pervasive in the real world. There are numerous ways dedicated to extract graph structure information explicitly or implicitly, but whether it has been adequately exploited remains an…
M V Sai Prakash, N Siddartha Reddy, Ganesh Parab, V Varun + 2 more
Molecular property prediction is a critical task in computational drug discovery. While recent advances in Graph Neural Networks (GNNs) and Transformers have shown to be effective and promising, they face the following limitations: Transformer self-attention does not explicitly consider the underlying molecule…
Jinyoung Park, Seongjun Yun, Hyeonjin Park, Jaewoo Kang + 4 more
'Kyung-Min Kim' 'Jung-Woo Ha' 'Hyunwoo J. Kim'] Transformer-based models have recently shown success in representation learning on graph-structured data beyond natural language processing and computer vision. However, the success is limited to small-scale graphs due to the drawbacks of full dot-product attention on…
Hyeongjin Kim, Byoung Chul Ko, Jing Tian
In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with GHA by applying a graph to the attention head of the transformer.…
Grégoire Mialon, Dexiong Chen, Margot Selosse, Julien Mairal
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations learned with classical graph neural networks (GNNs). Our model, GraphiT, encodes such information by (i) leveraging relative positional…
David Buterez, Jon Paul Janet, Dino Oglic, Pietro Liò
There has been a recent surge in transformer-based architectures for learning on graphs, mainly motivated by attention as an effective learning mechanism and the desire to supersede the hand-crafted operators characteristic of message passing schemes. However, concerns over their empirical effectiveness, scalability…
T. Warren Liao, Ziwei Zhang, Yufei Sun, Chunyu Hu
Graph Transformers (GTs) have demonstrated great effectiveness across various graph analytical tasks. However, the existing GTs focus on training and testing graph data originated from the same distribution, but fail to generalize under distribution shifts. Graph invariant learning, aiming to capture generalizable…
Ahmad Khajenezhad, Seyed Ali Osia, Mahmood Karimian, Hamid Beigy
—Transformers have become widely used in modern models for various tasks such as natural language processing and machine vision. This paper proposes Gransformer, an algorithm for generating graphs based on the Transformer. We extend a simple autoregressive Transformer encoder to exploit the structural information of…
Cailum Stienstra, Liam Hebert, Patrick Thomas, Alexander Haack + 2 more
Given that Infrared (IR) spectroscopy is a crucial tool in various chemical and forensic domains, improved in silico methods for predicting experimental spectra are needed due to the time and accuracy limitations of ab initio methods. We employ Graphormer, a graph neural network (GNN) transformer, to predict IR spectra…
Akshata Hegde, Jianlin Cheng
Deciphering gene regulatory networks (GRNs) from single-cell transcriptomics data remains a fundamental challenge in computational biology. It is hindered by data sparsity, high dimensionality, and the lack of scalable, generalizable inference models. To address this, we present GRNFormer, a generalizable graph…
Binon Teji, Swarup Roy, Dinabandhu Bhandari, Jugal Kalita
The inference of gene regulatory networks (GRNs) is critical for understanding the regulatory mechanisms underlying cellular development, functional specialization, and disease progression. Predicting regulatory gene interactions-often framed as a link prediction task-is a foundational step toward modeling cellular…
Allan Costa, Manvitha Ponnapati, Joseph M. Jacobson, Pranam Chatterjee
Determining the structure of proteins has been a long-standing goal in biology. Language models have been recently deployed to capture the evolutionary semantics of protein sequences. Enriched with multiple sequence alignments (MSA), these models can encode protein tertiary structure. In this work, we introduce an…
Mingxuan Zhang, Vinay Swamy, Rowan Cassius, Léo Dupire + 5 more
The ever-increasing availability of large-scale single-cell profiles presents an opportunity to develop foundation models to capture cell properties and behavior. However, standard language models such as transformers are best suited for sequentially structured data with well defined absolute or relative positional…
Sai Pooja Mahajan, Jeffrey A. Ruffolo, Jeffrey J. Gray
The optimal residue identity at each position in a protein is determined by its structural, evolutionary, and functional context. We seek to learn the representation space of the optimal amino-acid residue in different structural contexts in proteins. Inspired by masked language modeling (MLM), our training aims to…
Authors not listed
Deep generative models are transforming early-stage drug discovery, yet most current approaches are not well suited for realistic, small-data settings and often rely on simplified molecular representations such as linear strings, overlooking the inherent graph-based structure of molecules. To address this, we first…
Jie Lin, Mingyuan Xu, Hongming Chen
Shape-based virtual screening is a widely utilized method in ligand-based de novo drug design, aiming to identify molecules in chemical libraries that share similar 3D shapes but simultaneously possess novel 2D chemical structures compared to the reference compound. As an emerging technology, generative model is an…
David Buterez, Jon Paul Janet, Steven Kiddle, Pietro Liò
We investigate the potential of graph neural networks for transfer learning and improving molecular property prediction on sparse and expensive to acquire high-fidelity data by leveraging low-fidelity measurements as an inexpensive proxy for a targeted property ofinterest. This problem arises in discovery processes…
Sourya Bhattacharyya, Ferhat Ay
Decoding gene expression from epigenomic landscapes remains a fundamental challenge in genomics. We introduce EpiExpr, a flexible deep learning framework that predicts gene expression from 1D epigenetic tracks (EpiExpr-1D) and integrates 3D chromatin interactions (EpiExpr-3D) to capture distal regulatory effects.…
Authors not listed
Digital twins are virtual companions for the design, scale-up, and control of chemical processes. Equipping digital twins with mechanistic models of their mirrored unit operation expands their range of applicability compared to pure data-driven models. As constructing mechanistic models requires time, effort, and…
Authors not listed
Predicting protein-ligand binding affinity from three-dimensional (3D) structural data is a central task in structure-based drug discovery, yet it remains challenging due to limited data availability, structural complexity, and the sparse nature of 3D molecular representations. In this study, we investigate the…
Authors not listed
SynTemp is a framework designed to extract and hierarchically cluster reaction templates from large-scale reaction data repositories. Reaction templates are partial Imaginary Transition State graphs representing the reaction center as well as surrounding context. These graphs are equivalent to Double Pushout graph…