CausVSR: Causality inspired visual sentiment recognition

Visual Sentiment Recognition (VSR) is an evolving field that aims to detect emotional tendencieswithin visual content. Despite its growing significance, detecting emotions depicted in visual content,such as images, faces challenges, notably the emergence of misleading or spurious correlationsof the...

Full description

Saved in:
Bibliographic Details
Main Authors: ZHANG, Xinyue, WANG, Zhaoxia, WANG, Hailing, XIANG, Jing, WU, Chunwei, CAO, Guitao
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2024
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/9158
https://ink.library.smu.edu.sg/context/sis_research/article/10161/viewcontent/CausVSR_Causality_Inspired_Visual_Sentiment_Recognition_IJCAI.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:Visual Sentiment Recognition (VSR) is an evolving field that aims to detect emotional tendencieswithin visual content. Despite its growing significance, detecting emotions depicted in visual content,such as images, faces challenges, notably the emergence of misleading or spurious correlationsof the contextual information. In response to these challenges, we propose a causality inspired VSRapproach, called CausVSR. CausVSR is rooted in the fundamental principles of Emotional Causalitytheory, mimicking the human process from receiving emotional stimuli to deriving emotional states.CausVSR takes a deliberate stride toward conquering the VSR challenges. It harnesses the power of astructural causal model, intricately designed to encapsulate the dynamic causal interplay between visualcontent and their corresponding pseudo sentiment regions. This strategic approach allows for adeep exploration of contextual information, elevating the accuracy of emotional inference. Additionally,CausVSR utilizes a global category elicitation module, strategically employed to execute frontdooradjustment techniques, effectively detecting and handling spurious correlations. Experiments,conducted on four widely-used datasets, demonstrate CausVSR’s superiority in enhancing emotionperception within VSR, surpassing existing methods.