Audio intelligence & domain adaptation for deep learning models at the edge

• Main Author: Ng, Linus JunJia
• Other Authors: Gan Woon Seng
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/152683
• Institution: Nanyang Technological University

Identifying urban noises and sounds is a challenging but essential problem in the field of machine listening. It enables and provides a realistic use case for detecting noises in residential areas - from noise complaints to detecting sounds or unusual noises that may indicate possible emergencies. To mitigate noise issues in an estate is not an easy task using machine learning approach due to data scarcity and the lack of labeled data, where the acquisition of labeled data is often difficult, costly, and time-consuming. In this work, we leverage an end-to-end IoT system coupled with deep learning models to detect critical urban sound information at the edge. Wireless acoustic sensor nodes (WASN) are deployed in several residential areas to validate their feasibility in detecting noise events of interest, where real-time edge analytic is performed. We explore methods to address the domain shift caused by novel acoustic conditions that are introduced due to environmental influences in different deployed locations, evaluating the environmental sound classifiers in a WASN setup, and the extent it affects the performance of the sound classifiers in different locations with different microphones. We have collected and annotated audio data set in Singapore for training, validating, and testing purposes. Our experimental results show that the proposed method is able to address the mismatch introduced by the domain shift. The proposed method and future research in this work will enhance model robustness in adapting to new deployed environments and minimize the manpower time required to acquire and annotate audio data.