Covariance pooling for facial expression recognition
Classifying facial expressions into different categories requires capturing regional distortions of facial landmarks. We believe that second-order statistics such as covariance is better able to capture such distortions in regional facial features. In this work, we explore the benefits of using a ma...
Saved in:
Main Authors: | , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2018
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/6388 https://ink.library.smu.edu.sg/context/sis_research/article/7391/viewcontent/Covariance_Pooling_for_Facial_Expression_Recognition.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-7391 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-73912021-11-23T02:37:22Z Covariance pooling for facial expression recognition ACHARYA, D. HUANG, Zhiwu PAUDEL, D. VAN, Gool L. Classifying facial expressions into different categories requires capturing regional distortions of facial landmarks. We believe that second-order statistics such as covariance is better able to capture such distortions in regional facial features. In this work, we explore the benefits of using a manifold network structure for covariance pooling to improve facial expression recognition. In particular, we first employ such kind of manifold networks in conjunction with traditional convolutional networks for spatial pooling within individual image feature maps in an end-to-end deep learning manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on the validation set of Static Facial Expressions in the Wild (SFEW2.0) and 87.0% on the validation set of Real-World Affective Faces (RAF) Database1. Both of these results are the best results we are aware of. Besides, we leverage covariance pooling to capture the temporal evolution of per-frame features for video-based facial expression recognition. Our reported results demonstrate the advantage of pooling image-set features temporally by stacking the designed manifold network of covariance pooling on top of convolutional network layers. 2018-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6388 info:doi/10.1109/CVPRW.2018.00077 https://ink.library.smu.edu.sg/context/sis_research/article/7391/viewcontent/Covariance_Pooling_for_Facial_Expression_Recognition.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Face recognition Covariance matrices Manifolds Image recognition Databases and Information Systems Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Face recognition Covariance matrices Manifolds Image recognition Databases and Information Systems Graphics and Human Computer Interfaces |
spellingShingle |
Face recognition Covariance matrices Manifolds Image recognition Databases and Information Systems Graphics and Human Computer Interfaces ACHARYA, D. HUANG, Zhiwu PAUDEL, D. VAN, Gool L. Covariance pooling for facial expression recognition |
description |
Classifying facial expressions into different categories requires capturing regional distortions of facial landmarks. We believe that second-order statistics such as covariance is better able to capture such distortions in regional facial features. In this work, we explore the benefits of using a manifold network structure for covariance pooling to improve facial expression recognition. In particular, we first employ such kind of manifold networks in conjunction with traditional convolutional networks for spatial pooling within individual image feature maps in an end-to-end deep learning manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on the validation set of Static Facial Expressions in the Wild (SFEW2.0) and 87.0% on the validation set of Real-World Affective Faces (RAF) Database1. Both of these results are the best results we are aware of. Besides, we leverage covariance pooling to capture the temporal evolution of per-frame features for video-based facial expression recognition. Our reported results demonstrate the advantage of pooling image-set features temporally by stacking the designed manifold network of covariance pooling on top of convolutional network layers. |
format |
text |
author |
ACHARYA, D. HUANG, Zhiwu PAUDEL, D. VAN, Gool L. |
author_facet |
ACHARYA, D. HUANG, Zhiwu PAUDEL, D. VAN, Gool L. |
author_sort |
ACHARYA, D. |
title |
Covariance pooling for facial expression recognition |
title_short |
Covariance pooling for facial expression recognition |
title_full |
Covariance pooling for facial expression recognition |
title_fullStr |
Covariance pooling for facial expression recognition |
title_full_unstemmed |
Covariance pooling for facial expression recognition |
title_sort |
covariance pooling for facial expression recognition |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2018 |
url |
https://ink.library.smu.edu.sg/sis_research/6388 https://ink.library.smu.edu.sg/context/sis_research/article/7391/viewcontent/Covariance_Pooling_for_Facial_Expression_Recognition.pdf |
_version_ |
1770575951114010624 |