Combining multiple kernel methods on Riemannian manifold for emotion recognition in the wild

• Main Authors: LIU, M., WANG, R., LI, S., SHAN, S., HUANG, Zhiwu, CHEN, X.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2014
• Subjects: Emotion recognition; EmotiW 2014 challenge; Multiple kernels; Riemannian manifold; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/6387; https://ink.library.smu.edu.sg/context/sis_research/article/7390/viewcontent/Combining_multiple_kernel_methods_on_riemannian_manifold_for_emotion_recognition_in_the_wild.pdf
• Institution: Singapore Management University

In this paper, we present the method for our submission to the Emotion Recognition in the Wild Challenge (EmotiW 2014). The challenge is to automatically classify the emotions acted by human subjects in video clips under realworld environment. In our method, each video clip can be represented by three types of image set models (i.e. linear subspace, covariance matrix, and Gaussian distribution) respectively, which can all be viewed as points residing on some Riemannian manifolds. Then different Riemannian kernels are employed on these set models correspondingly for similarity/distance measurement. For classification, three types of classifiers, i.e. kernel SVM, logistic regression, and partial least squares, are investigated for comparisons. Finally, an optimal fusion of classifiers learned from different kernels and different modalities (video and audio) is conducted at the decision level for further boosting the performance. We perform an extensive evaluation on the challenge data (including validation set and blind test set), and evaluate the effects of different strategies in our pipeline. The final recognition accuracy achieved 50.4% on test set, with a significant gain of 16.7% above the challenge baseline 33.7%.