Transfer learning enhanced AR spatial registration for facility maintenance management

• Main Authors: Chen, Keyu, Yang, Jianfei, Cheng, Jack C. P., Chen, Weiwei, Li, Chun Ting
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2022
• Subjects: Engineering::Electrical and electronic engineering; AR Spatial Registration; Facility Maintenance Management
• Online Access: https://hdl.handle.net/10356/161072
• Institution: Nanyang Technological University

Augmented reality (AR), which requires a spatial registration technique, has proved to greatly improve the efficiency of facility maintenance management (FMM) activities. Being one of the most promising techniques for indoor localization, Wi-Fi fingerprinting has been widely used for AR spatial registration. However, localization accuracy of Wi-Fi fingerprinting decreases over time due to dynamics of environmental factors. Readings from different mobile devices can also affect the accuracy negatively. In this paper, a transfer learning technique named transferable CNN-LSTM is proposed for improving the robustness of Wi-Fi fingerprinting while implementing AR in FMM activities. Convolutional neural network (CNN), embedded with long short term memory (LSTM) networks, is utilized to predict the location of unlabeled fingerprints. Multiple kernel variant of maximum mean discrepancy (MK-MMD) is adopted to reduce the distribution difference between the source domain and the target domain, so that the location of the newly collected unlabeled fingerprints can be predicted accurately. As shown in the experimental validation, the transferable CNN-LSTM can achieve an accuracy of 97.1% in short-term (without significant environmental changes) spatial registration, 87.8% in long-term (with significant environmental changes) spatial registration, and around 90% in multi-device spatial registration, indicating a higher accuracy and better robustness over other conventional approaches.