Novel progressive learning technique for classification problems

• Main Author: Venkatesan, Rajasekar
• Other Authors: Er Meng Joo
• Format: Theses and Dissertations
• Language: English
• Published: 2017
• Subjects: DRNTU::Engineering::Electrical and electronic engineering
• Online Access: http://hdl.handle.net/10356/72680
• Institution: Nanyang Technological University

The human brain is complex and is arguably the most sophisticated cognitive computer which can perform multifarious tasks rapidly and accurately. Developing a mental system to mimic the human brain and to enable machines to learn on their own without the need for extra programming each time is an active area of research and a feat yet to be achieved. Progressive learning is an effective learning model which is demonstrated by the human learning process. It is the process of learning continuously from direct experience. In machine learning, there are different categories of classification problems, namely (1) Single-label classification, which includes binary and multi-class classification and (2) Multi-label classification. Inspired by human learning, the research work has three key objectives. The first key objective of this research is the development of a label-independent generic classifier that is capable of performing binary, multi-class and multi-label classification. The second key objective of this research is developing an extreme learning machine based advanced learning technique capable of exhibiting progressive learning behavior for classification problems. And finally, the third key objective is the integration of progressive learning technique to the label-independent classifier thereby resulting in a generic classifier capable of dynamic learning of new classes and also capable of addressing all the aforementioned types of classification problems. The progressive learning technique based on the progressive learning paradigm exhibited by human learning process requires the classification technique to be independent of the number of class constraint and capable of learning several new classes on the go by retaining the knowledge of previous classes. It is realized by modifying the network structure by itself upon encountering a new class and updating the network parameters in such a way that it learns the new class by retaining the knowledge learnt thus far. The existing online sequential learning methods do not require retraining when a “new data sample” is received, but it fails when a “new class of data” which is unknown to the existing knowledge is encountered. Progressive learning technique overcomes this shortcoming by allowing the network to learn multiple new classes’ alien to existing knowledge, encountered at any point of time. In the sequence of batch and online learning paradigms, the next logical extension is progressive learning. Machine learning classification can be categorized into single-label classification (binary and multi-class) and multi-label classification. Several machine learning classifiers have been developed and are available in the literature for each of the classification types. But the major limitation of all the classifiers in the literature is that, the classifiers are limited only to the particular type of classification problem for which it has been trained. There exists no classifier that is capable of universally addressing all the aforementioned types of classification problems. Inspired from human learning, a new label-independent/universal classifier which is capable of performing binary, multi-class and multi-label classification is developed. The final outcome of the thesis is the development of human-learning inspired progressively learning universally generic classifier. It is achieved by integrating the progressive learning feature onto the label-independent classifier. The newly developed classifier based on the extreme learning machine exploits its inherent high speed training and testing. Thus, the developed classifier can be used to address binary, multi-class and multi-label classification problems with dynamic class constraints accurately and efficiently.