12 papers · ranked by Valyu relevance
Guillaume Perez, Janarbek Matai, Takahiro Harada
Implicit neural representations (INRs) are increasingly being used as tools to map coordinates to signals, encompassing applications from neural fields to texture compression, shape representations, and beyond. Most INR methods are based on using high-dimensional projections of the initial coordinates through encoders…
Yang Li, Si Si, Gang Li, Cho‐Jui Hsieh + 1 more
Attentional mechanisms are order-invariant. Positional encoding is a crucial component to allow attention-based deep model architectures such as Transformer to address sequences or images where the position of information matters. In this paper, we propose a novel positional encoding method based on learnable Fourier…
Ezequiel López‐Rubio, Macoris Decena-Gimenez, Rafael Marcos Luque-Baena
A key module in neural transformer-based deep architectures is positional encoding. This module enables a suitable way to encode positional information as input for transformer neural layers. This success has been rooted in the use of sinusoidal functions of various frequencies, in order to capture recurrent patterns…
Amballa, Avinash
Recent studies have demonstrated the effectiveness of position encoding in transformer architectures. By incorporating positional information, this approach provides essential guidance for modeling dependencies between elements across different sequence positions. We introduce CoPE (a lightweight Complex Positional…
Shin Fujieda, Atsushi Yoshimura, Takahiro Harada
In this work, we propose local positional encoding for an MLP which is a hybrid of positional encoding and grid encoding. Local positional encoding can resolve high-frequency signals without using as many frequencies as positional encoding requires, and without preparing a high-resolution grid as grid encoding…
Alejo Lopez-Avila, Jinhua Du, Abbas Shimary, Ze Li
encoding for Sequential recommendation Authors: ['Alejo Lopez-Avila' 'Jinhua Du' 'Abbas Shimary' 'Ze Li'] The expansion of streaming media and ecommerce has led to a boom in recommendation systems, including Sequential recommendation systems, which consider the user's previous interactions with items. In recent years…
Li, Jiaye, Chen, Baoyou + 8 more
Transformers rely on explicit positional encoding to model structure in data. While Rotary Position Embedding (RoPE) excels in 1D domains, its application to image generation reveals significant limitations such as fine-grained spatial relation modeling, color cues, and object counting. This paper identifies key…
Boyang Li, Yulin Wu, Nuoxian Huang
Cell-Inspired Framework Authors: ['Boyang Li' 'Yulin Wu' 'Nuoxian Huang'] Understanding spatial location and relationships is a fundamental capability for modern artificial intelligence systems. Insights from human spatial cognition provide valuable guidance in this domain. Recent neuroscientific discoveries have…
Christoffer Koo Øhrstrøm, Rafael I. Cabral Muchacho, Yifei Dong, Filippos Moumtzidellis + 3 more
In this work, we propose a position encoding that is designed specifically for vision modalities. Transformers (Vaswani et al., 2017) are widely used for computer vision and robotics tasks. They have proven to be highly flexible, finding applications in several visionbased modalities such as images (Dosovitskiy et al.…
Sravan Kumar Ankireddy, S. Ashwin Hebbar, Heping Wan, Joonyoung Cho + 1 more
'Charlie Zhang'] Abstract—Tailoring polar code construction for decoding algorithms beyond successive cancellation has remained a topic of significant interest in the field. However, despite the inherent nested structure of polar codes, the use of sequence models in polar code construction is understudied. In this…
Javad Haghighat, Tolga M. Duman
—Noisy shuffling channels capture the main characteristics of DNA storage systems where distinct segments of data are received out of order, after being corrupted by substitution errors. For realistic schemes with short-length segments, practical indexing and channel coding strategies are required to restore the order…
Chance J. Hamilton, Alfredo Weitzenfeld
This paper presents the Visual Place Cell Encoding (VPCE) model, a biologically inspired computational framework for simulating place cell–like activation using visual input. Drawing on evidence that visual landmarks play a central role in spatial encoding, the proposed VPCE model activates visual place cells by…