Unity assumption in audiovisual emotion perception

We experience various sensory stimuli every day. How does this integration occur? What are the inherent mechanisms in this integration? The "unity assumption" proposes a perceiver's belief of unity in individual unisensory information to modulate the degree of multisensory integration...

Full description

Saved in:
Bibliographic Details
Main Authors: Sou, Ka Lon, Say, Ashley, Xu, Hong
Other Authors: School of Social Sciences
Format: Article
Language:English
Published: 2022
Subjects:
Online Access:https://hdl.handle.net/10356/157067
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-157067
record_format dspace
spelling sg-ntu-dr.10356-1570672023-03-05T15:34:48Z Unity assumption in audiovisual emotion perception Sou, Ka Lon Say, Ashley Xu, Hong School of Social Sciences Social sciences::Psychology Multisensory Integration Semantic Emotion Perception We experience various sensory stimuli every day. How does this integration occur? What are the inherent mechanisms in this integration? The "unity assumption" proposes a perceiver's belief of unity in individual unisensory information to modulate the degree of multisensory integration. However, this has yet to be verified or quantified in the context of semantic emotion integration. In the present study, we investigate the ability of subjects to judge the intensities and degrees of similarity in faces and voices of two emotions (angry and happy). We found more similar stimulus intensities to be associated with stronger likelihoods of the face and voice being integrated. More interestingly, multisensory integration in emotion perception was observed to follow a Gaussian distribution as a function of the emotion intensity difference between the face and voice-the optimal cut-off at about 2.50 points difference on a 7-point Likert scale. This provides a quantitative estimation of the multisensory integration function in audio-visual semantic emotion perception with regards to stimulus intensity. Moreover, to investigate the variation of multisensory integration across the population, we examined the effects of personality and autistic traits of participants. Here, we found no correlation of autistic traits with unisensory processing in a nonclinical population. Our findings shed light on the current understanding of multisensory integration mechanisms. AI Singapore Nanyang Technological University National Research Foundation (NRF) Published version This research was supported by the Nanyang Technological University Research Scholarship (KS), SGUnited Traineeship Programme (AS), School of Social Sciences, College of Humanities, Arts and Social Sciences, Nanyang Technological University (HX), and the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-RP-2020-019) to HX. 2022-05-02T06:00:36Z 2022-05-02T06:00:36Z 2022 Journal Article Sou, K. L., Say, A. & Xu, H. (2022). Unity assumption in audiovisual emotion perception. Frontiers in Neuroscience, 16, 782318-. https://dx.doi.org/10.3389/fnins.2022.782318 1662-4548 https://hdl.handle.net/10356/157067 10.3389/fnins.2022.782318 35310087 2-s2.0-85127134427 16 782318 en AISG2-RP-2020-019 Frontiers in Neuroscience © 2022 Sou, Say and Xu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Social sciences::Psychology
Multisensory Integration
Semantic Emotion Perception
spellingShingle Social sciences::Psychology
Multisensory Integration
Semantic Emotion Perception
Sou, Ka Lon
Say, Ashley
Xu, Hong
Unity assumption in audiovisual emotion perception
description We experience various sensory stimuli every day. How does this integration occur? What are the inherent mechanisms in this integration? The "unity assumption" proposes a perceiver's belief of unity in individual unisensory information to modulate the degree of multisensory integration. However, this has yet to be verified or quantified in the context of semantic emotion integration. In the present study, we investigate the ability of subjects to judge the intensities and degrees of similarity in faces and voices of two emotions (angry and happy). We found more similar stimulus intensities to be associated with stronger likelihoods of the face and voice being integrated. More interestingly, multisensory integration in emotion perception was observed to follow a Gaussian distribution as a function of the emotion intensity difference between the face and voice-the optimal cut-off at about 2.50 points difference on a 7-point Likert scale. This provides a quantitative estimation of the multisensory integration function in audio-visual semantic emotion perception with regards to stimulus intensity. Moreover, to investigate the variation of multisensory integration across the population, we examined the effects of personality and autistic traits of participants. Here, we found no correlation of autistic traits with unisensory processing in a nonclinical population. Our findings shed light on the current understanding of multisensory integration mechanisms.
author2 School of Social Sciences
author_facet School of Social Sciences
Sou, Ka Lon
Say, Ashley
Xu, Hong
format Article
author Sou, Ka Lon
Say, Ashley
Xu, Hong
author_sort Sou, Ka Lon
title Unity assumption in audiovisual emotion perception
title_short Unity assumption in audiovisual emotion perception
title_full Unity assumption in audiovisual emotion perception
title_fullStr Unity assumption in audiovisual emotion perception
title_full_unstemmed Unity assumption in audiovisual emotion perception
title_sort unity assumption in audiovisual emotion perception
publishDate 2022
url https://hdl.handle.net/10356/157067
_version_ 1759857698302590976