CausVSR: Causality inspired visual sentiment recognition

• Main Authors: ZHANG, Xinyue, WANG, Zhaoxia, WANG, Hailing, XIANG, Jing, WU, Chunwei, CAO, Guitao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/9158; https://ink.library.smu.edu.sg/context/sis_research/article/10161/viewcontent/CausVSR_Causality_Inspired_Visual_Sentiment_Recognition_IJCAI.pdf
• Institution: Singapore Management University

Visual Sentiment Recognition (VSR) is an evolving field that aims to detect emotional tendencieswithin visual content. Despite its growing significance, detecting emotions depicted in visual content,such as images, faces challenges, notably the emergence of misleading or spurious correlationsof the contextual information. In response to these challenges, we propose a causality inspired VSRapproach, called CausVSR. CausVSR is rooted in the fundamental principles of Emotional Causalitytheory, mimicking the human process from receiving emotional stimuli to deriving emotional states.CausVSR takes a deliberate stride toward conquering the VSR challenges. It harnesses the power of astructural causal model, intricately designed to encapsulate the dynamic causal interplay between visualcontent and their corresponding pseudo sentiment regions. This strategic approach allows for adeep exploration of contextual information, elevating the accuracy of emotional inference. Additionally,CausVSR utilizes a global category elicitation module, strategically employed to execute frontdooradjustment techniques, effectively detecting and handling spurious correlations. Experiments,conducted on four widely-used datasets, demonstrate CausVSR’s superiority in enhancing emotionperception within VSR, surpassing existing methods.