Appliances interaction activities recognition using smartphones

• Main Author: Soh, Rynard Jun Yang
• Other Authors: Soh Yeng Chai
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Engineering
• Online Access: https://hdl.handle.net/10356/176927
• Institution: Nanyang Technological University

In this endeavour, the primary focus is on detecting the activation of energy-related devices by users, coupled with the recognition of human activities to enhance the former. The project entails an exploration into the activation patterns of various household appliances such as televisions, washing machines, vacuum cleaners, and more, utilising their distinctive audio signatures. Leveraging machine learning algorithms, specifically convolutional neural networks for event recognition, the project aims to process and differentiate these signatures, enabling the identification of both individual and combined activities and events. Furthermore, the project will incorporate gyrometers and accelerometers to discern user motion, facilitating the deduction of associated activities. Additionally, the project will leverage the smartphone camera, employing transfer learning techniques to enhance image recognition capabilities. This approach will contribute to the identification and categorisation of objects or activities captured by the camera within the project's defined scope. This comprehensive approach underscores the project's commitment to robustly identifying energy device activation, user activities, and object recognition for a more nuanced understanding of user interactions.