24 papers · ranked by Valyu relevance
Mathieu Lefort, Yann Boniface, Bernard Girau
To move in a complex and dynamic environment, according to the active perception theory, an agent needs to learn the multi modal correlations between its actions and the changes they induce in the environment . Human beings interact with their environment through several distant organs, whose sensory flow processing…
William Paul Boyce, Anthony Lindsay, Arkady Zgonnikov, Iñaki Rañó + 1 more
'KongFatt Wong-Lin'] Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However…
Yu Hu, Yalda Mohsenzadeh
Our brain seamlessly integrates distinct sensory information to form a coherent percept. However, when real-world audiovisual events are perceived, the specific brain regions and timings for processing different levels of information remain less investigated. To address that, we curated naturalistic videos and recorded…
Tayfun Lloyd-Esenkaya, Vanessa Lloyd-Esenkaya, Eamonn O’Neill, Michael J. Proulx
'Michael J. Proulx'] Sensory substitution techniques are perceptual and cognitive phenomena used to represent one sensory form with an alternative. Current applications of sensory substitution techniques are typically focused on the development of assistive technologies whereby visually impaired users can acquire…
Yidi Li, Hong Liu, Hao Tang
Multi-modal fusion is proven to be an effective method to improve the accuracy and robustness of speaker tracking, especially in complex scenarios. However, how to combine the heterogeneous information and exploit the complementarity of multi-modal signals remains a challenging issue. In this paper, we propose a novel…
Subha Nawer Pushpita, Leila Wehbe
Understanding how the human brain processes naturalistic audiovisual information remains a central challenge in cognitive neuroscience. Progress has been limited, however, by the difficulty of modeling complex audiovisual feature spaces - most prior work has therefore relied on short, controlled stimuli, or on stimuli…
W. Paul Boyce, Tony Lindsay, Arkady Zgonnikov, Iñaki Rañó + 1 more
'KongFatt Wong‐Lin'] Abstract–Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimises the average error in perceptual representation of stimuli.…
Zhuanghua Shi, Hermann J. Müller
Surrounded by multiple objects and events, receiving multisensory stimulation, our brain must sort through relevant and irrelevant multimodal signals to correctly decode and represent the information from the same and different objects and, respectively, events in the physical world. Over the last two decades…
Maria Casado-Palacios, Alessia Tonelli, Claudio Campus, Monica Gori
When engaging with the environment, multisensory cues interact and are integrated to create a coherent representation of the world around us, process that has been suggested to be affected by the lack of visual feedback in blind individuals. In addition, the presence of voluntary movement seems to be responsible for…
Daniel Martin, Sandra Malpica, Diego Gutiérrez, Belén Masiá + 1 more
'Ana Serrano'] Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR…
Anastasia Maria Kesoglou, Kyriaki Mikellidou
Our brain is skilled with the ability to perceive and process multimodal stimuli. This process known as crossmodal perceptual integration, has been in the research spotlight for a long time, providing evidence for the integration of information coming from different modalities. Prior research mostly utilized pictures…
Woochul Choi, Se-Bum Paik
When the sensory system receives an ambiguous signal, human perception often switches spontaneously between two different interpretations. This phenomenon is called bistable perception, and has been considered important to understanding sensory system. In this study, we investigated the intervals of spontaneous…
Yabo Zheng, Lihan Chen
Perceptual processing integrates information from multiple sensory modalities to form a coherent representation of the environment. A classic example of such is the Sound-Induced Flash Illusion (SIFI), where the perceived number of visual flashes is altered by conflicting auditory stimuli. While the SIFI is a…
Jan Grenzebach, Thomas G. G. Wegner, Wolfgang Einhäuser, Alexandra Bendixen
'Alexandra Bendixen'] Multistable perception occurs in all sensory modalities, and there is ongoing theoretical debate about whether there are overarching mechanisms driving multistability across modalities. Here we study whether multistable percepts are coupled across vision and audition on a moment-by-moment basis.…
Feng Feng, Puhong Li, Tony Stockman
Leveraging the perceptual phenomenon of crossmodal correspondence has been shown to facilitate peoples information processing and improves sensorymotor performance. However, for goal-oriented interactive tasks, the question of how to enhance the perception of specific crossmodal information, and how crossmodal…
Nicolás Navarro-Guerrero, Sibel Toprak, Josip Josifovski, Lorenzo Jamone
'Lorenzo Jamone'] The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of…
Jun Zhang, Dayong Tao
Introduction In the realm of basketball, refining shooting skills and decision-making levels using intelligent agents has garnered significant interest. This study addresses the challenge by introducing an innovative framework that combines multi-modal perception and deep reinforcement learning. The goal is to create…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…
Michele Fornaciai, Irene Togoli, Samuel Binisti, Olivier Collignon
Our perception often shows systematic biases revealing how the brain’s internal representation diverges from the physical world. In particular, what happened in the recent past can systematically attract current percepts, making them appear more similar to previous stimuli. Does this phenomenon reflect a low-level…
Authors not listed
Identifying molecular structure based on spectroscopic readings is a key task in a va- riety of chemical and biological applications. Common spectroscopy techniques, such as Infrared (IR) Spectroscopy and Mass Spectrometry (MS), provide detailed information on the structure of molecular compounds but nonetheless…
Lampros Androutsos, Lorenzo Pallante, Agorakis Bompotas, Filip Stojceski + 8 more
Taste perception plays a pivotal role in guiding nutrient intake and aiding in the avoidance of potentially harmful substances through five basic tastes - sweet, bitter, umami, salty, and sour. Taste perception originates from molecular interactions in the oral cavity between taste receptors and chemical tastants.…
Authors not listed
We report a reusable cross-reactive plasmonic sensing system that generates unique optical fingerprints for any liquid mixture. The platform leverages a multiplexed plasmonic chip and methodology for creating 24 orthogonally modified sensors, coupled with hyperspectral imaging. We demonstrate the versatility and…
Authors not listed
This research delves into olfaction, a sensory modality that remains complex and inadequately understood. We aim to fill in two gaps in recent studies that attempted to use machine learning and deep learning approaches to predict human smell perception. The first one is that molecules are usually represented with…
Vinicius Santana, Erbet Costa, Carine Rebello, Ana Mafalda Ribeiro + 2 more
This study presents an innovative framework for classifying and predicting odor intensity in perfumery, combining scientific machine learning with mechanistic modeling to enhance fragrance design precision. A probabilistic weight assignment is introduced, utilizing scent classifier outputs to determine the contribution…