Correlation-based multi-view representation learning for information fusion

• Main Author: Wu, Zuobin
• Other Authors: Mao Kezhi
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/144264
• Institution: Nanyang Technological University

With the advancement of information technology, a large amount of data are generated from different sources. A lot of pattern recognition systems employ a fusion of multiple sources of information to enhance learning performance. The information is collected from various domains or obtained from diverse feature extractors with heterogeneous properties. Each information can be referred to as a particular view. Single-view learning may not be sufficient to describe the underlying distribution of information. A conventional way of using multi-view information is concatenating them into one single compact information to adapt to the learning setting. However, this concatenation may not be the best way to integrate different sources of data. In contrast to single-view learning, multi-view representation learning introduces one function to model a particular view and jointly optimizes all the functions to exploit the redundant views of the same input data to improve learning performance. Although there exist various multi-view representation learning methods for information fusion, how to learn a good association among multi-view data is still an unsolved problem. In this thesis, we try to solve this problem from the perspective of correlation-based multi-view representation learning and feature fusion strategies because the correlation structure is important in multi-view representation learning. The motivation of correlation-based multi-view representation learning is introduced in Chapter 1. Various correlation-based multi-view representation learning and deep representation learning models are reviewed in Chapter 2. Three correlation-based multi-view representation learning models are proposed in Chapter 3, Chapter 4 and Chapter 5, respectively. Chapter 6 elaborates a feature regrouping algorithm for correlation-based multi-view representation learning. These models are proposed to solve the problems in the existing correlation-based multi-view representation learning methods. Traditional correlation-based multi-view representation learning approach may not preserve the complete and effective discriminative structure underlying multi-view representations. In Chapter 3, we propose intra-class and extra-class discriminative correlation analysis (IEDCA) to provide more discriminative and complete representations through correlation-based multi-view representation learning. IEDCA explores both the pair-wise correlation like canonical correlation analysis (CCA) based feature fusion approaches and the correlation across different features within the same class. Most representation generated by correlation-based multi-view representation learning may still contain redundancies due to the correlation criterion. In Chapter 4, to explore the correlation underlying different views and jointly optimize the learned representation to remove the irrelevant redundancies brought by the correlation-based multi-view representation approach, we incorporate the proposed IEDCA in Chapter 3 into minimum Redundancy Maximum Relevance (mRMR) and further improve the learning performance. The nonlinear representation produced by kernelized correlation-based multi-view representation learning approach is limited by a fixed kernel. Moreover, the training time scales poorly with the size of the training data. In Chapter 5, we combine the flexibility of deep representation learning with the canonical correlation analysis from multiple views of data and propose deep supervised multi-view CCA (DMCCA). DMCCA can learn flexible nonlinear representations by passing multi-view data through multiple stacked layers of nonlinear transformation. In most correlation-based multi-view representation learning approach, the natural groupings of features are directly used. The complementary and mutual information existed in multi-view representations may not be fully utilized. In Chapter 6, we propose a feature regrouping approach for correlation-based multi-view representation learning to reduce the variances of learned representations and enhance the learning performance. In Chapter7, we have the summary and future directions of this PhD work.