Adaptive Similarity Component Analysis in Nonparametric Dynamic Environment

• Main Author: Sojodishijani, Omid
• Format: Thesis
• Language: English
• Published: 2011
• Online Access: http://psasir.upm.edu.my/id/eprint/19970/1/ITMA_2011_9_ir.pdf; http://psasir.upm.edu.my/id/eprint/19970/
• Institution: Universiti Putra Malaysia

Pattern classification and recognition in low-rank distance metric dealing with nonparametric changes is an underlying problem in dynamic environment applications. Data arrives from operational field in a stream model and similarity-based classification algorithms must identify them with acceptable performance. Although, there are adaptive forms of independent feature extraction methods such as principle component analysis (PCA), linear discriminant analysis (LDA) and independent component analysis (ICA) to transform the training patterns to low dimensional space and/or improve the classifiers accuracy, they suffer from nonparametric changes in data over time. This study is devoted to design a data-driven linear transformation to increase the performance of similarity-based classifiers in the presence of nonparametric changes of data over time. For this purpose, a nonparametric multiclass component analysis technique in nonstationary environments is introduced. This generative model enables adaptive similarity-based classifiers to classify time-labeled inquiry pattern with superior accuracy in a low dimensional feature space. In this thesis, an optimal transformation matrix is used to transform the time-labeled instances from original space to a new feature space in order to maximize the probability of selecting the correct class label for incoming instance by similarity-based classifiers. For this purpose, the most probable location of incoming instance for each class is estimated. Then, an optimal transformation matrix is computed by maximizing the information gain at the estimated points. By restricting the transformation matrix to a nonsquare matrix, the dimensions of feature space will be linearly reduced. Experimental results on real and synthesized datasets with real and artificial changes demonstrate the performance of the proposed method in terms of accuracy and dimension reduction in dynamic environments. In the case of real datasets, the proposed method yields 12.16% average misclassification error while the average misclassification error for five different methods GAM, TSY, NWKNN, DWM and FISH is 19.54%. Also, the results of experiments on synthesized datasets show that the proposed method yields 32.83% average misclassification error while average misclassification error of five different methods is 38.78%. From a dimensionality reduction evaluation aspect, the average misclassification error of the proposed method in low-rank feature space is 9.6% and same error rate for three other well-known feature extraction methods is 21.21%. The novelty of the proposed approach resides in the possibility to reduce the dimensions of feature space and simultaneously increase the accuracy of similarity-based classification method in an adaptive fashion in the nonparametric dynamic environment. Consequently, the proposed adaptive feature extraction technique and neighborhood-based classifier family are tightly integrated in an adaptive K-nearest neighbor classifier.