26 papers · ranked by Valyu relevance
Shilin Pu, Liang Chu, Jincheng Hu, Shibo Li + 3 more
'James Covington'] Accurate traffic prediction is significant in intelligent cities’ safe and stable development. However, due to the complex spatiotemporal correlation of traffic flow data, establishing an accurate traffic prediction model is still challenging. Aiming to meet the challenge, this paper proposes…
Vijay Prakash Dwivedi, Xavier Bresson
We propose a generalization of transformer neural network architecture for arbitrary graphs. The original transformer was designed for Natural Language Processing (NLP), which operates on fully connected graphs representing all connections between the words in a sequence. Such architecture does not leverage the graph…
Akshata Hegde, Jianlin Cheng
We introduce GRNFormer, a generalizable graph transformer framework for accurate gene regulatory network (GRN) inference from transcriptomics data. Designed to work across species, cell types, and platforms without requiring cell-type annotations or prior regulatory information, GRNFormer integrates a transformer-based…
Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Philip H. S. Torr + 1 more
'Mark Coates'] Transformers have attained outstanding performance across various modalities, owing to their simple but powerful scaled-dot-product (SDP) attention mechanisms. Researchers have attempted to migrate Transformers to graph learning, but most advanced Graph Transformers (GTs) have strayed far from plain…
Xin Lai, Yang Liu, Rui Qian, Yong Lin + 2 more
'Kinkar Chandra Das'] Graph-structured data, operating as an abstraction of data containing nodes and interactions between nodes, is pervasive in the real world. There are numerous ways dedicated to extract graph structure information explicitly or implicitly, but whether it has been adequately exploited remains an…
Kelong Mao, Peilin Zhao, Tingyang Xu, Yu Rong + 2 more
With massive possible synthetic routes in chemistry, retrosynthesis prediction is still a challenge for researchers. Recently, retrosynthesis prediction is formulated as a Machine Translation (MT) task. Namely, since each molecule can be represented as a Simplified Molecular-Input Line-Entry System (SMILES) string, the…
Akshata Hegde, Jianlin Cheng
Deciphering gene regulatory networks (GRNs) from single-cell transcriptomics data remains a fundamental challenge in computational biology. It is hindered by data sparsity, high dimensionality, and the lack of scalable, generalizable inference models. To address this, we present GRNFormer, a generalizable graph…
Thang Chu, Tuan Nguyen
Previous models have shown that learning drug features from their graph representation is more efficient than learning from their strings or numeric representations. Furthermore, integrating multi-omics data of cell lines increases the performance of drug response prediction. However, these models showed drawbacks in…
Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min, Sungjun Cho + 3 more
'Honglak Lee' 'Seunghoon Hong'] We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice. Given a graph, we simply treat all nodes and edges as independent tokens, augment them with token embeddings, and feed them to a…
Meng Zhang, Jie Sun, Qinghao Hu, Peng Sun + 3 more
'Tianwei Zhang'] Abstract—Graph Transformer is a new architecture that surpasses GNNs in graph learning. While there emerge inspiring algorithm advancements, their practical adoption is still limited, particularly on real-world graphs involving up to millions of nodes. We observe existing graph transformers fail on…
Yingke Yang, Peiluan Li
Background In the field of computational personalized medicine, drug response prediction (DRP) is a critical issue. However, existing studies often characterize drugs as strings, a representation that does not align with the natural description of molecules. Additionally, they ignore gene pathway-specific combinatorial…
Jinyoung Park, Seongjun Yun, Hyeonjin Park, Jaewoo Kang + 4 more
'Kyung-Min Kim' 'Jung-Woo Ha' 'Hyunwoo J. Kim'] Transformer-based models have recently shown success in representation learning on graph-structured data beyond natural language processing and computer vision. However, the success is limited to small-scale graphs due to the drawbacks of full dot-product attention on…
Sanghyun Yoo, Young Seok Kim, Kang Hyun Lee, Kuhwan Jeong + 3 more
'Junhwi Choi' 'Hoshik Lee' 'Young Sang Choi'] Graphs are the natural data structure to represent relational and structural information in many domains. To cover the broad range of graph-data applications including graph classification as well as graph generation, it is desirable to have a general and flexible model…
Grégoire Mialon, Dexiong Chen, Margot Selosse, Julien Mairal
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations learned with classical graph neural networks (GNNs). Our model, GraphiT, encodes such information by (i) leveraging relative positional…
Hyeongjin Kim, Byoung Chul Ko, Jing Tian
In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with GHA by applying a graph to the attention head of the transformer.…
Ahmad Khajenezhad, Seyed Ali Osia, Mahmood Karimian, Hamid Beigy
—Transformers have become widely used in modern models for various tasks such as natural language processing and machine vision. This paper proposes Gransformer, an algorithm for generating graphs based on the Transformer. We extend a simple autoregressive Transformer encoder to exploit the structural information of…
Romeo Šajina, Goran Oreški, Marina Ivašić-Kos, Marco Leo + 2 more
'Paolo Russo' 'Fabiana Di Ciaccio'] Highlights This paper presents the GCN-Transformer, a novel deep learning model that integrates Graph Convolutional Networks (GCNs) and Transformers to enhance multi-person pose forecasting. The model effectively captures both spatial and temporal dependencies, improving the…
Authors not listed
Deep generative models are transforming early-stage drug discovery, yet most current approaches are not well suited for realistic, small-data settings and often rely on simplified molecular representations such as linear strings, overlooking the inherent graph-based structure of molecules. To address this, we first…
David Buterez, Jon Paul Janet, Steven Kiddle, Pietro Liò
We investigate the potential of graph neural networks for transfer learning and improving molecular property prediction on sparse and expensive to acquire high-fidelity data by leveraging low-fidelity measurements as an inexpensive proxy for a targeted property ofinterest. This problem arises in discovery processes…
Peng Li, Lu Huang, Jin Li, Haiyan Yan + 1 more
Vision Transformers (ViTs) have achieved impressive results in large-scale image classification. However, when training from scratch on small datasets, there is still a significant performance gap between ViTs and Convolutional Neural Networks (CNNs), which is attributed to the lack of inductive bias. To address this…
Jie Lin, Mingyuan Xu, Hongming Chen
Shape-based virtual screening is a widely utilized method in ligand-based de novo drug design, aiming to identify molecules in chemical libraries that share similar 3D shapes but simultaneously possess novel 2D chemical structures compared to the reference compound. As an emerging technology, generative model is an…
Dong Wang, Meiyan Lin, Xiaoxu Zhang, Yonghui Huang + 2 more
'Marcin Woźniak'] In recent years, neural network algorithms have demonstrated tremendous potential for modulation classification. Deep learning methods typically take raw signals or convert signals into time-frequency images as inputs to convolutional neural networks (CNNs) or recurrent neural networks (RNNs).…
Xueying Tian, Yash Patel, Yue Wang
Gene regulatory networks (GRNs) play a crucial role in the control of cellular functions. Numerous methods have been developed to infer GRNs from gene expression data, including mechanism-based approaches, information-based approaches, and more recent deep learning techniques, the last of which often overlooks the…
Authors not listed
Recent advances in generative artificial intelligence have enabled in silico molecular design to become a powerful approach for exploring chemical space toward specific design goals across various domains. However, in actual design workflows, determining the appropriate generation conditions, including generative…
Authors not listed
Predicting protein-ligand binding affinity from three-dimensional (3D) structural data is a central task in structure-based drug discovery, yet it remains challenging due to limited data availability, structural complexity, and the sparse nature of 3D molecular representations. In this study, we investigate the…
Chonghuan Zhang, Qianyue Zhang, Alexei Lapkin
Biochemical transformations may allow significant improvements in synthetic efficiency of complex functional molecules through reduction in the number of synthetic steps or avoidance of harsh conditions and/or toxic solvents/reactants. Yet, there is a limited access to biochemical reaction data, which reduces the…