Sensor-based human activity recognition via zero-shot learning

• Main Author: Wang, Wei
• Other Authors: Miao Chun Yan
• Format: Theses and Dissertations
• Language: English
• Published: 2019
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/104645; http://hdl.handle.net/10220/50145
• Institution: Nanyang Technological University

Sensor-based activity recognition aims to recognize human activities from the sensor readings. In recent years, with the advance of the sensor technology and the development of the machine learning algorithms, it obtains much progress. Activity recognition is a classification problem. Thus, existing works on sensor-based activity recognition are generally based on supervised-classification algorithms, such as SVM, HMM, CRF, kNN, and decision tree. In these works, for each class to recognize, sufficient labeled training instances belonging to this class are needed. In practical applications, the process of obtaining labeled sensor readings is time consuming and costly. The activity classes having sufficient sensor readings are generally classes predefined during data collecting-and-labeling in the model learning phase. However, in many applications, the activities needed to recognize could contain not only the predefined classes, but also some previously unseen classes which do not have labeled training instances during model learning. In this thesis, we focus on sensor-based activity recognition including the unseen classes. Under this problem setting, the training instances belong to a set of activity classes (referred to as the seen classes), while the testing instances can come from the previously unseen activity classes (referred to as the unseen classes). For problems under this problem setting, as there are no labeled training instances belonging to the unseen classes, the zero-shot learning methods are used. We focus on three problems under this setting. Firstly, we study the problem of unseen activity recognition for one person. In this problem, the classes covered by the training and the testing instances are disjoint. We propose a method, referred to as Nonlinear Compatibility Based Method (NCBM), to address this problem. In our method, we adopt the binary attribute space as the semantic space, and use it to involve the semantic information. Then, we learn a compatibility function between the feature space and the semantic space. The recognition of the testing instances is achieved with the learned compatibility function and the unseen-class-prototypes in the semantic space. We conduct extensive experiments, and the experimental results show the effectiveness of our method. Secondly, we study the problem of activity recognition across smart homes with different binary sensors and label spaces. We propose a method, referred to as Binary Sensor Semantic and Time Information Method (BSST), to address this problem. In this problem, the deployed binary sensor networks and the activities needed to recognize in different smart homes are different. In our method, we utilize the characteristics of the binary sensors, the semantic information about the sensors and the activities, and the time information. Sensor readings in different smart homes are extracted into a common feature space, and each activity class has a defined prototype in the semantic space. Then, we learn a classification model in one smart home environment and use it to perform activity recognition in a new smart home environment. During model learning, no information about the new smart home environment is needed. The experimental results show the effectiveness of our method. Lastly, we study the problem of multi-resident activity recognition including unseen classes in smart homes. We propose a method, referred to as Multi-Resident with Unseen-Activity-Class Recognition Method (MRUA), to address this problem. In our method, the activity recognition for each resident is regarded as a learning task, and all tasks are learned jointly with multi-task learning techniques. The zero-shot learning techniques are also adopted. Thus, the unseen classes can be recognized. Through extensive experiments, we show the effectiveness of our method.