Feature fusion with covariance matrix regularization in face recognition
The fusion of multiple features is important for achieving state-of-the-art face recognition results. This has been proven in both traditional and deep learning approaches. Existing feature fusion methods either reduce the dimensionality of each feature first and then concatenate all low-dimensional...
Saved in:
Main Authors: | , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2018
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/87999 http://hdl.handle.net/10220/44512 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-87999 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-879992020-03-07T14:02:35Z Feature fusion with covariance matrix regularization in face recognition Lu, Ze Jiang, Xudong Kot, Alex Chichung School of Electrical and Electronic Engineering Rapid-Rich Object Search Lab Feature Fusion CNN The fusion of multiple features is important for achieving state-of-the-art face recognition results. This has been proven in both traditional and deep learning approaches. Existing feature fusion methods either reduce the dimensionality of each feature first and then concatenate all low-dimensional feature vectors, named as DR-Cat, or the vice versa, named as Cat-DR. However, DR-Cat ignores the correlation information between different features which is useful for classification. In Cat-DR, on the other hand, the correlation information estimated from the training data may not be reliable especially when the number of training samples is limited. We propose a covariance matrix regularization (CMR) technique to solve problems of DR-Cat and Cat-DR. It works by assigning weights to cross-feature covariances in the covariance matrix of training data. Thus the feature correlation estimated from training data is regularized before being used to train the feature fusion model. The proposed CMR is applied to 4 feature fusion schemes: fusion of pixel values from 3 color channels, fusion of LBP features from 3 color channels, fusion of pixel values and LBP features from a single color channel, and fusion of CNN features extracted by 2 deep models. Extensive experiments of face recognition and verification are conducted on databases including MultiPIE, Georgia Tech, AR and LFW. Results show that the proposed CMR technique significantly and consistently outperforms the best single feature, DR-Cat and Cat-DR. NRF (Natl Research Foundation, S’pore) MOE (Min. of Education, S’pore) Accepted version 2018-03-06T06:23:18Z 2019-12-06T16:53:48Z 2018-03-06T06:23:18Z 2019-12-06T16:53:48Z 2017 Journal Article Lu, Z., Jiang, X., & Kot, A. (2018). Feature fusion with covariance matrix regularization in face recognition. Signal Processing, 144, 296-305. 0165-1684 https://hdl.handle.net/10356/87999 http://hdl.handle.net/10220/44512 10.1016/j.sigpro.2017.10.024 en Signal Processing © 2017 Elsevier B.V. This is the author created version of a work that has been peer reviewed and accepted for publication by Signal Processing, Elsevier B.V. It incorporates referee’s comments but changes resulting from the publishing process, such as copyediting, structural formatting, may not be reflected in this document. The published version is available at:[http://dx.doi.org/10.1016/j.sigpro.2017.10.024]. 30 p. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
country |
Singapore |
collection |
DR-NTU |
language |
English |
topic |
Feature Fusion CNN |
spellingShingle |
Feature Fusion CNN Lu, Ze Jiang, Xudong Kot, Alex Chichung Feature fusion with covariance matrix regularization in face recognition |
description |
The fusion of multiple features is important for achieving state-of-the-art face recognition results. This has been proven in both traditional and deep learning approaches. Existing feature fusion methods either reduce the dimensionality of each feature first and then concatenate all low-dimensional feature vectors, named as DR-Cat, or the vice versa, named as Cat-DR. However, DR-Cat ignores the correlation information between different features which is useful for classification. In Cat-DR, on the other hand, the correlation information estimated from the training data may not be reliable especially when the number of training samples is limited. We propose a covariance matrix regularization (CMR) technique to solve problems of DR-Cat and Cat-DR. It works by assigning weights to cross-feature covariances in the covariance matrix of training data. Thus the feature correlation estimated from training data is regularized before being used to train the feature fusion model. The proposed CMR is applied to 4 feature fusion schemes: fusion of pixel values from 3 color channels, fusion of LBP features from 3 color channels, fusion of pixel values and LBP features from a single color channel, and fusion of CNN features extracted by 2 deep models. Extensive experiments of face recognition and verification are conducted on databases including MultiPIE, Georgia Tech, AR and LFW. Results show that the proposed CMR technique significantly and consistently outperforms the best single feature, DR-Cat and Cat-DR. |
author2 |
School of Electrical and Electronic Engineering |
author_facet |
School of Electrical and Electronic Engineering Lu, Ze Jiang, Xudong Kot, Alex Chichung |
format |
Article |
author |
Lu, Ze Jiang, Xudong Kot, Alex Chichung |
author_sort |
Lu, Ze |
title |
Feature fusion with covariance matrix regularization in face recognition |
title_short |
Feature fusion with covariance matrix regularization in face recognition |
title_full |
Feature fusion with covariance matrix regularization in face recognition |
title_fullStr |
Feature fusion with covariance matrix regularization in face recognition |
title_full_unstemmed |
Feature fusion with covariance matrix regularization in face recognition |
title_sort |
feature fusion with covariance matrix regularization in face recognition |
publishDate |
2018 |
url |
https://hdl.handle.net/10356/87999 http://hdl.handle.net/10220/44512 |
_version_ |
1681034138537164800 |