Cooperative inference and learning for internet-of-things with limited resources

• Main Author: Wang, Yuan
• Other Authors: Tay Wee Peng
• Format: Theses and Dissertations
• Language: English
• Published: 2019
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/103663; http://hdl.handle.net/10220/49989
• Institution: Nanyang Technological University

In the era of big data and Internet-of-Things (IoT), ubiquitous smart devices continuously sense the environment and generate large amount of data. So that inference and learning techniques play key roles in IoT, which can create knowledge and unveil important information from the sensory data. An IoT network with only centralized processor or cloud-based remote server may lead to heavy data traffic and long system latency. Such unfavorable consequences have been gradually becoming major concerns which hinder further applications of IoT technologies. By applying the concept of edge computing, computation and data storage resources are distributed closer to the end nodes in the IoT networks. These intelligent end nodes can be organized into distributed or decentralized local networks that support cooperative inference and learning, which is promising to reduce the data volume to be uploaded to the central processor, shorten the response time, and improve robustness and scalability. However, in typical IoT networks, resources like computing power, communication bandwidth and energy budget is rather limited. Therefore it is essential to explore approaches for the inference and learning that can make the most of local resources and reduce unnecessary consumption. In this thesis, we study cooperative inference and learning schemes for the IoT networks with only limited resources. We address this topic by developing appropriate distributed least-mean-square (LMS) algorithms for inference and learning from linear models and cooperative deep learning-based face recognition pipeline for inference and learning from nonlinear models. To be more speci c, this thesis covers the following aspects: Firstly, we consider a wireless sensor network (WSN) in the context of IoT where every nodes have a common learning objective, performing distributed LMS estimation cooperatively by means of exchanging local intermediate estimates. To preserve the merits of local cooperation and save energy budget as well as communication resources at the same time, a diffusion strategy with event-based communication policy is proposed, in which nodes exchange information with their neighbors only when a criterion is satis ed. By doing so, the diffusion network can significantly reduce communication overhead while being able to achieve satisfactory steady-state mean-squared deviation (MSD) performance. Secondly, we consider a WSN consists of multiple clusters of nodes where only the nodes belong to the same cluster share the same learning objective. In this scenario, inappropriate cooperation among nodes belongs to different clusters may worsen learning accuracy and waste the limited resources. To avoid this issue, we propose a multitask diffusion strategy whose mean stability is independent of the inter-cluster cooperation, based on which we also develop optimization schemes in order to allow every nodes to optimize their inter-cluster cooperation with neighbors in different clusters. As such, the entire diffusion network are able to achieve better learning performance compared to the case where no inter-cluster cooperation is utilized among clusters. Finally, in addition to online learning from linear data model, cooperative learning from nonlinear model is also studied. To be specific, we designed a face recognition pipeline for IoT surveillance application by using convolutional neural network (CNN)-based deep learning techniques. The pipeline is implementable with low-cost embedded IoT devices, and is able to achieve short-enough image-to-result latency. We showed that local cooperation among these low-cost embedded devices can lead to enhanced learning performance. The results imply that deep learning tasks can be realized in local IoT network with limited resources instead of uploading raw data to a power-hungry central processor resides remotely or on the cloud by spending a lot communication bandwidth, which could also reduce the system latency at the same time.