STranGAN: Adversarially-learnt Spatial Transformer for scalable human activity recognition

• Main Authors: Faridee, Abu Zaher Md, Chakma, Avijoy, MISRA, Archan, Roy, Nirmalya
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Adversarial learning; Domain adaptation; Generative modeling; Learnable data augmentation; Wearable sensing; Numerical Analysis and Scientific Computing; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/6946; https://ink.library.smu.edu.sg/context/sis_research/article/7949/viewcontent/Strangan_av.pdf
• Institution: Singapore Management University

We tackle the problem of domain adaptation for inertial sensing-based human activity recognition (HAR) applications -i.e., in developing mechanisms that allow a classifier trained on sensor samples collected under a certain narrow context to continue to achieve high activity recognition accuracy even when applied to other contexts. This is a problem of high practical importance as the current requirement of labeled training data for adapting such classifiers to every new individual, device, or on-body location is a major roadblock to community-scale adoption of HAR-based applications. We particularly investigate the possibility of ensuring robust classifier operation, without requiring any new labeled training data, under changes to (a) the individual performing the activity, and (b) the on-body position where the sensor-embedded mobile or wearable device is placed. We propose STranGAN, a framework that explicitly decouples the domain adaptation functionality from the classification model by learning and applying a set of optimal spatial affine transformations on the target domain inertial sensor data stream by employing adversarial learning, which only requires collecting raw data samples (but no accompanying activity labels) from both source and target domains. STranGAN’s uniqueness lies in its ability to perform practically useful adaptation (a) without any labeled training data and without requiring paired, synchronized generation of source and target domain samples, and (b) without requiring any changes to a pre-trained HAR classifier. Empirical results using three publicly available benchmark datasets indicate that STranGAN(a) is particularly effective in handling on-body position heterogeneity(achieving a 5% improvement in classification F1 score compared to state-of-the-art baselines), (b) offers competitive performance for handling cross-individual variations, and (c) the affine transformation parameters can be analyzed to gain interpretable insights on the domain heterogeneity.