28 papers · ranked by Valyu relevance
Gianni Brauwers, Flavius Frăsincar
—Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework…
Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello + 2 more
'Anselm Levskaya' 'Jonathon Shlens'] Convolutions are a fundamental building block of modern computer vision systems. Recent approaches have argued for going beyond convolutions in order to capture long-range dependencies. These efforts focus on augmenting convolutional models with content-based interactions, such as…
Mohammed Hassanin, Saeed Anwar, Ibrahim Radwan, Fahad Khan + 1 more
'Ajmal Mian'] Abstract—Inspired by the human cognitive system, attention is a mechanism that imitates the human cognitive awareness about specific information, amplifying critical details to focus more on the essential aspects of data. Deep learning has employed attention to boost performance for many applications.…
Isaac Lin, Tianye Wang, Shang Gao, Shiming Tang + 1 more
Convolutional neural networks (CNNs) have been shown to be state-of-the-art models for visual cortical neurons. Cortical neurons in the primary visual cortex are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs integrate global contextual information to model…
Michał Bola, Marta Paź, Łucja Doradzińska, Anna Nowicka
It is well established that stimuli representing or associated with ourselves, like our own name or an image of our own face, benefit from preferential processing. However, two key questions concerning the self-prioritization mechanism remain to be addressed. First, does it operate in an automatic manner during the…
Zeyu Zhang, Bin Li, Chenyang Yan, Kengo Furuichi + 2 more
'Chang Seok Bang'] Artificial intelligence, with its remarkable adaptability, has gradually integrated into daily life. The emergence of the self-attention mechanism has propelled the Transformer architecture into diverse fields, including a role as an efficient and precise diagnostic and predictive tool in medicine.…
Sanghoon Lee, Kazi A. Islam, Sai C. Koganti, Varshini Yaganti + 3 more
Digital pathology has played a key role in replacing glass slides with digital images, enhancing various pathology workflows. Whole slide images are digitized pathological images improving the capabilities of digital pathology and contributing to the overall turnaround time for diagnoses. The digitized images have been…
Alana de Santana Correia, Esther Luna Colombini
In humans, Attention is a core property of all perceptual and cognitive operations. Given our limited ability to process competing sources, attention mechanisms select, modulate, and focus on the information most relevant to behavior. For decades, concepts and functions of attention have been studied in philosophy…
Andong Tan, Duc Tam Nguyen, Maximilian Dax, Matthias Nießner + 1 more
'Thomas Brox'] Self-attention networks have shown remarkable progress in computer vision tasks such as image classification. The main benefit of the self-attention mechanism is the ability to capture long-range feature interactions in attention-maps. However, the computation of attention-maps requires a learnable key…
Saga Svensson, Marius Golubickis, Sam Johnson, Johanna K Falbén + 1 more
'C Neil Macrae'] Recent theoretical accounts maintain that core components of attentional functioning are preferentially tuned to self-relevant information. Evidence in support of this viewpoint is equivocal, however, with research overly reliant on personally significant (i.e., familiar) stimulus inputs (e.g., faces…
Meike Scheller, Jan Tünnermann, Katja Fredriksson, Huilin Fang + 1 more
Efficiently processing self-related information is critical for cognition, yet the earliest mechanisms enabling this self-prioritization remain unclear. By combining a temporal order judgement task with computational modelling based on the Theory of Visual Attention (TVA), we show how mere, arbitrary associations with…
Haotong Sun, Yinghui Jiang, Minhao Wang, Xianglu Xiao + 5 more
Rapid and accurate prediction of molecular properties is a fundamental task in drug discovery. In recent years, deep learning-based molecular property prediction methods have received much attention and recent successes have shown that learning the representations of molecular structures by applying graph neural…
Lev V. Utkin, Andrei V. Konstantinov, Stanislav R. Kirpichenko
New models of random forests jointly using the attention and self-attention mechanisms are proposed for solving the regression problem. The models can be regarded as extensions of the attention-based random forest whose idea stems from applying a combination of the Nadaraya-Watson kernel regression and the Huber's…
Ankit Yadav, Arpan Banerjee, Dipanjan Roy
Internally directed attention (IDA), whether spontaneous or intentional, has been associated with impaired performance on externally directed cognitive tasks. Yet, the neurophysiological mechanisms underpinning this disruption remain poorly understood. In the present study, we characterized the neural correlates of IDA…
Authors not listed
Improving the performance of thermoelectric (TE) materials is essential for their wider adoption in sustainable energy and cooling applications. Impurity doping is a common strategy for enhancing transport properties, yet synthesizing every possible TE composition is infeasible, and high-fidelity ab initio simulations…
Jonathan Morgan, Badr Albanna, James P. Herman
We present a neural network model of visual attention (NNMVA) that integrates biased competition and reinforcement learning to capture key aspects of attentional behavior. The model combines selfattention mechanisms from Vision Transformers (ViTs), Long Short-Term Memory (LSTM) networks for working memory, and an…
Xiaoliang Luo, Brett D. Roads, Bradley C. Love
People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation…
Grace W. Lindsay
Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine…
Jonathan Morgan, Badr F. Albanna, James P. Herman
attention Authors: ['Jonathan Morgan' 'Badr F. Albanna' 'James P. Herman'] Attention has emerged as a core component of both biological and artificial intelligences (AIs). Despite decades of parallel research, studies of animal and AI attention remain largely separate. The self-attention mechanism ubiquitous in…
Fuzhan Rahmanian, Robert M. Lee, Dominik Linzner, Kathrin Michel + 4 more
Predicting and monitoring battery life early and across chemistries is a significant challenge due to the plethora of degradation paths, form factors, and electrochemical testing protocols. Existing models typically translate poorly across different electrode, electrolyte, and additive materials, mostly require a fixed…
Saeed Salehi, Jordan Lei, Ari S. Benjamin, Klaus-Robert Müller + 1 more
Attention is a key component of the visual system, essential for perception, learning, and memory. Attention can also be seen as a solution to the binding problem: concurrent attention to all parts of an entity allows separating it from the rest. However, the rich models of attention in computational neuroscience are…
Authors not listed
This research presents a novel approach to obstacle detection during navigation using a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. The primary objective is to generate accurate image captions that describe the content of images, which is crucial for applications such…
David Buterez, Jon Paul Janet, Steven J. Kiddle, Dino Oglic + 1 more
Atom-centred neural networks represent the state-of-the-art for approximating the quantum chemical properties of molecules, such as internal energies. While the design of machine learning architectures that respect chemical principles has continued to advance, the final atom pooling operation that is necessary to…
Sabrina Trapp, Henning Schroll, Fred H. Hamker
consciousness Authors: ['Sabrina Trapp' 'Henning Schroll' 'Fred H. Hamker'] Within recent years, researchers have proposed the independence of attention and consciousness on both empirical and conceptual grounds. However, the elusive nature of these constructs complicates progress in the investigation of their…
Domenico Guarino
“Attention” is a central faculty of the mind (Baars, ; Minsky, ). It has been early proposed that it is a process selecting some of the available information to focus on enhanced processing (Deutsch and Deutsch, ; Treisman and Gelade, ). Contrasting this approach, Richard J. Krauzlis recently (; ) proposed that…
Authors not listed
Predicting protein-ligand binding affinity from three-dimensional (3D) structural data is a central task in structure-based drug discovery, yet it remains challenging due to limited data availability, structural complexity, and the sparse nature of 3D molecular representations. In this study, we investigate the…
Thomas Sanocki, Jong Han Lee, Manoranjan Paul
This article provides an introduction to experimental research on top-down human attention in complex scenes, written for cognitive scientists in general. We emphasize the major effects of goals and intention on mental function, measured with behavioral experiments. We describe top-down attention as an open category of…
Hang Hu, Jyothsna Padmakumar Bindu, Julia Laskin
Mass spectrometry imaging (MSI) is widely used for the label-free molecular mapping of biological samples. The identification of co-localized molecules in MSI data is crucial to the understanding of biochemical pathways. However, complex MSI data are too large for manual annotation but too small for training deep…