23 papers · ranked by Valyu relevance
Lujing Chen, Rui Liu, Xin Yang, Dongsheng Zhou + 2 more
'Xiaopeng Wei'] In recent years, human motion prediction has become an active research topic in computer vision. However, owing to the complexity and stochastic nature of human motion, it remains a challenging problem. In previous works, human motion prediction has always been treated as a typical inter-sequence…
Billy Dickson, James Mochizuki-Freeman, Md Rysul Kabir, Zoran Tiganj
Human language processing, characterized by its ability to capture long-range dependencies in sequential inputs, operates under the constraints of limited working memory. In contrast, state-of-the-art transformer models in artificial intelligence rely on access to the fixed context window, which deviates from the…
Hongjiang Chen, Pengfei Jiao, Ming Du, Xuan Guo + 3 more
The growing interest in Temporal Graph Neural Networks (TGNNs) stems from their ability to model complex dynamics and deliver superior performance. However, TGNNs encounter fundamental challenges in capturing long-term dependencies and identifying periodic patterns. To address these limitations, we propose TGFormer, a…
Radhika Chandrasekaran, Senthil Kumar Paramasivan
Short-term load forecasting plays a vital role in today's modern life to ensure the balance between energy demand and supply. Dynamic variations in weather and electricity consumption patterns can significantly influence load patterns, resulting in complex modeling and challenging forecasting accuracy due to…
Krishnakanta Barik, Goutam Paul
The Temporal Fusion Transformer (TFT), proposed by Lim et al., published in International Journal of Forecasting (2021), is a state-of-the-art attention-based deep neural network architecture specifically designed for multi-horizon time series forecasting. It has demonstrated significant performance improvements over…
Ziyang Song, Qincheng Lu, Hao Xu, Ziqi Yang + 3 more
Purpose Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success in Natural Language Processing and Computer Vision domains. However, the development of PTMs on healthcare time-series data is lagging behind. This underscores the limitations of the existing transformer-based…
Jiaheng Yin, Zhengxin Shi, Jianshen Zhang, Xiaomin Lin + 3 more
Temporal Information for Time-Series Forecasting Authors: ['Jiaheng Yin' 'Zhengxin Shi' 'Jianshen Zhang' 'Xiaomin Lin' 'Yulin Huang' 'Yongzhi Qi' 'Qi Wei'] In recent years, numerous Transformer-based models have been applied to long-term time-series forecasting (LTSF) tasks. However, recent studies with linear models…
Jake Grigsby, Zhe Wang, Yanjun Qi
Multivariate time series forecasting focuses on predicting future values based on historical context. State-of-the-art sequence-tosequence models rely on neural attention between timesteps, which allows for temporal learning but fails to consider distinct spatial relationships between variables. In contrast, methods…
Stanislav Vakaruk, Amit Karamchandani, Jesús Enrique Sierra-García, Alberto Mozo + 4 more
'Alberto Mozo' 'Sandra Gómez-Canaval' 'Antonio Pastor' 'Dieter Schramm' 'Philipp Sieberg'] Recently, a novel approach in the field of Industry 4.0 factory operations was proposed for a new generation of automated guided vehicles (AGVs) that are connected to a virtualized programmable logic controller (PLC) via a 5G…
Yijun Ma, Zehong Wang, Weixiang Sun, Yanfang Ye
Temporal graph learning is pivotal for deciphering dynamic systems, where the core challenge lies in explicitly modeling the underlying evolving patterns that govern network transformation. However, prevailing methods are predominantly task-centric and rely on restrictive assumptions such as short-term dependency…
Zekun Li, Shiyang Li, Xifeng Yan
Irregularly sampled time series are increasingly prevalent, particularly in medical domains. While various specialized methods have been developed to handle these irregularities, effectively modeling their complex dynamics and pronounced sparsity remains a challenge. This paper introduces a novel perspective by…
Jiahe Yan, Honghui Li, Yanhui Bai, Jie Liu + 3 more
'Konstantinos Kalpakis'] Accurate time-series forecasting plays a vital role in sensor-driven applications such as energy monitoring, traffic flow prediction, and environmental sensing. While most existing approaches focus on extracting local patterns from historical observations, they often overlook the global…
Junfeng Zuo, Xiao Liu, Ying Nian Wu, Si Wu + 1 more
Time perception is fundamental in our daily life. An important feature of time perception is temporal scaling (TS): the ability to generate temporal sequences (e.g., movements) with different speeds. However, it is largely unknown about the mathematical principle underlying TS in the brain. The present theoretical…
Joel Ye, Chethan Pandarinath
Neural population activity is theorized to reflect an underlying dynamical structure. This structure can be accurately captured using state space models with explicit dynamics, such as those based on recurrent neural networks (RNNs). However, using recurrence to explicitly model dynamics necessitates sequential…
Palash Bera, Jagannath Mondal
Capturing the time evolution and predicting future conformational states of physicochemical systems present significant challenges due to the precision and computational effort required. In this study, we demonstrate that the transformer, a machine learning model renowned for machine translation and natural language…
Bo Peng, Yuanming Ding, Wei Kang, Zahir M. Hussain + 1 more
Since introducing the Transformer model, it has dramatically influenced various fields of machine learning. The field of time series prediction has also been significantly impacted, where Transformer family models have flourished, and many variants have been differentiated. These Transformer models mainly use attention…
Ariel Goldstein, Eric Ham, Samuel A. Nastase, Zaid Zada + 15 more
Deep language models (DLMs) provide a novel computational paradigm for how the brain processes natural language. Unlike symbolic, rule-based models described in psycholinguistics, DLMs encode words and their context as continuous numerical vectors. These “embeddings” are constructed by a sequence of layered…
Authors not listed
Accurate prediction of chemical reaction yields remains essential for accelerating synthesis optimization, yet current machine learning models face critical limitations in capturing temporal dynamics, providing calibrated uncertainty estimates, and explicitly modeling reactant-to-product transformations. Here we…
Julian Ng-Kee-Kwong, Mufeng Tang, Thomas Akam, Rafal Bogacz
The ability to extract and exploit temporal structure across diverse tasks is central to human cognition. Neuroscientists have typically relied on recurrent neural networks (RNNs) trained with backpropagation through time (BPTT) when modelling neural and behavioural processes such as decision-making and motor control.…
Alessandro Tibo, Jiazhen He, Jon Paul Janet, Eva Nittinger + 1 more
How many near-neighbors does a molecule have? This is a simple, fundamental, but unsolved question in chemistry. It is key for solving many important molecular optimization problems, for example in lead optimization in drug discovery under the similarity principle assumption. Generative models can sample virtual…
Emma Tysinger, Brajesh Rai, Anton Sinitskiy
Meaningful exploration of the chemical space of druglike molecules in drug design is a highly challenging task due to a combinatorial explosion of possible modifications of molecules. In this work, we address this problem with transformer models, a type of machine learning (ML) model, with recent demonstrated success…
Authors not listed
Efficiency of machine learning (ML) models is crucial to minimize inference times and reduce carbon footprints of models deployed in production environments. Current models employed in retrosynthesis to generate a synthesis route from a target molecule to purchasable compounds are prohibitively slow. The model operates…
Samuel Renaud, Rachael Mansbach
Current antibacterial treatments cannot overcome the rapidly growing resistance of bacteria to antibiotic drugs, and novel treatment methods are required. One option is the development of new antimicrobial peptides (AMPs), to which bacterial resistance build-up is comparatively slow. Deep generative models have…