Concept-level image sentiment analysis with semi-supervised learning

• Main Author: Kamath, Shantanu Arun
• Other Authors: Erik Cambria
• Format: Final Year Project
• Language: English
• Published: 2018
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: http://hdl.handle.net/10356/74676
• Institution: Nanyang Technological University

Advancing technological wave and rapid growth in social media platforms have enabled people to represent their experience, stances, and feelings using visual media such as images. Mining sentiments from images over various platforms can be used to provide crucial information on the opinions of the people and the general outlook on the topic. The critical need for this will grow exponentially, in pace with the global growth of content. Expressiveness varies from one person to another. Most images posted on Twitter lack good labels and the accompanying tweets have a lot of noise. Hence, in this paper, we identify the contents and sentiments in images through the extraction of image features. We leverage on the fact that Restricted Boltzmann Machines allows greedy, layer-wise and unsupervised pre-training that provides great performance in image classification. In particular, we present a novel semi-supervised method to extract features from Twitter images by unsupervised pre-training of Convolutional Restricted Boltzmann Machines (CRBM) with Contrastive Divergence (CD) followed by supervised training of Convolutional Neural Networks (CNN) and a Recurrent Neural Network (RNN) with backpropagation. The model is evaluated on a Twitter dataset of images and corresponding labels and show that accuracy is higher than using just Convolutional Neural Networks (CNN).