Event camera based action recognition and falling detection

• Main Author: Lu, Shilin
• Other Authors: Alex Chichung Kot
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/157591
• Institution: Nanyang Technological University

Event cameras are sensors that are quite different from traditional cameras, since they asynchronously respond to the brightness changes of each pixel and their output is a data stream containing the location, timestamp and polarity of the brightness changes. The most widely used method to process event data is spiking neural networks (SNNs). In this dissertation, two kinds of popular SNNs are introduced, including Spiking ResNet and Spike-Element-Wise (SEW) ResNet. The latter one is an improved version of the former one and addresses the drawbacks of the former one. Based on observation, some networks and techniques in video action recognition work well with SNNs; therefore, Temporal Segment Network (TSN) is applied to SEWResNet to further improve the performance. There is a series of experiments conducted on four public event-based datasets, and the experimental results show the SEWResNet combined with TSN is able to achieve higher test accuracy.