Voice detection with spiking neural network for smart sensor applications

• Main Author: Hu, Jinhai
• Other Authors: Goh Wang Ling
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/145056
• Institution: Nanyang Technological University

A novel methodology that embeds machine learning algorithm on microcontrollers for real time training and processing, or in other words, online training on edge devices is described in this report. Due to memory limitation and single time dimensional processing of microcontroller unit (MCU), the Arduino Uno and Nano are engaged in this project. With that, traditional Artificial Intelligence (AI) algorithms were modified to suit the new working environment. Firstly, an improved Spiking Recurrent Neural Network (SRNN) based on Spiking Convolutional Neural Network (SCNN) is designed to realize better real time voice signal, and an accuracy of 99.8% in voice digit recognition on Matlab software platform was achieved. However, Long Short-Term Memory (LSTM) Neural Network in SRNN trained all weights in floating 32-bit applications. Thus, for hardware convenience, a dynamic bias and advanced processing LSTM is designed and tested in floating 32-bit, fixed 1-bit and fixed 2-bit system, each achieved final loss of 0.0425, 0.0446 and 0.0453 in mean square error (MSE), respectively. Not only software programming was innovated in this project, hardware implementation had also been realized using different components. A multi-layer perceptron neural network that deployed 16% of the Arduino Uno flash memory was established to recognize a 7-segment LED displayer, and finally achieved a mere 0.0004 error, thus verified that simple neural network can built on an ATMEGA328P processor, an 8-bit AVR microcontroller with 32K Bytes in-system programmable flash memory. Moreover, a voice command recognition had been achieved 97.5% accuracy based on pre-well-trained neural network and Fast Fourier Transform (FFT) in a TensorFlow microprocessor library. Finally, a single spike neuron was simulated by one Arduino Uno board to mimic neuron’s performance under different cell state surroundings. Also, A simple Spiking Neural Network (SNN) made up by two series connected spike neurons, where each neuron was represented by an Arduino Uno board, was able to reproduce the excitation wave of series spiking neurons, All experiments in this final year project had verified that the proposed SNN for smart sensor applications is an interesting and valuable topic worth continuing for online training on edge devices.