Tensor computing for big data analytic

• Main Author: Ong, Jenn Bing
• Other Authors: Ng Wee Keong
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/152350
• Institution: Nanyang Technological University

Tensors have been around since the end of nineteenth century with the development of differential calculus. Tensor research and applications have spanned many areas ranging from mathematics and physics in the early days to psychometrics and linguistics later in around 1960s, and more recently to signal processing and machine learning in the 1990s. Tensor is multi-dimensional extension of matrix, it emerges as important mathematical tool because measurements from experimental studies usually come from more than two dimensions (e.g., space, time, sensors data, and locations), treating these arrays as matrices might lose important structural information such as multi-dimensional correlation structure during data analysis. Tensor decomposition is an established technique for multi-way data analysis such as blind source separation, feature extraction, higher-order statistical analyses, etc. In general, matrix and tensor decomposition have different properties (e.g., uniqueness and interpretability) despite their similarity in concept. Tensor network decomposes higher-order tensors into sparsely-interconnected low-order core tensors, which captures the complex correlation structure in parsimonious manner. Tensor network has been used mostly for data imputation and compressed computation due to the lack of physical interpretability, however, it is promising for big data processing and high-dimensional numerical computing due to their natural support for parallel distributed and compressed computation. The rationale behind this thesis is to explore new applications of tensor network computing in the big data deep learning era. The contributions of this thesis are summarized as follows: 1) protect big data privacy using randomized tensor network decomposition and dispersed computation, 2) provide theoretical and empirical analysis of adversarial perturbations in deep learning using tensor analysis, and 3) perform blind source separation for human movements sensing using streaming WiFi channel state information. For big data privacy applications, our primary intuition comes from observing the ability of tensor network to compute wide-range of multi-linear operations using the low-order core tensors without the need to reconstruct the original tensor, which overcomes the curse-of-dimensionality by modeling large parameter space in parsimonious manner using the tensor network representations. The tensor distributed computing is implemented in the multi-party computation setting, this enhances the privacy of big data computation by randomized tensor information dispersal. Compared to existing encryptions and data splitting techniques, randomized tensor network computation does not require centralized servers for management and hence completely removes the single point of failure in big data privacy protection. Adversarial perturbation on deep learning models deviates the inferred output to desired state by the adversary by slightly modifying the input data. Adversarial perturbations have disastrous impact to mission-critical and safety-critical applications such as autonomous vehicles. Many research studies have shown that adversarially-perturbed road signs and adversarial patch can easily cause many deep learning models to misclassification, e.g., detects a speed sign instead of the actual stop sign. Higher-order tensor network decomposition is utilized to provide theoretical analysis of the sensitivity of deep learning models subject to complex correlation structure in the input data, empirical evidence is provided to support the theoretical analysis and adaptive algorithm based on tensor network is proposed to detect strong and static adversarial perturbations. Human movements are modulated in the surrounding WiFi signals, which can be extracted using data processing and analysis pipeline. Vital sign such as respiration and heartbeat are important predictors of human health status. In this study, we are interested to extract respiration signals from WiFi channel state information (CSI) for occupancy detection and healthcare monitoring. However, the motion dynamics of CSI subcarriers centered at different frequencies cannot be written as linear mixtures of the respiration signals. Thanks to the complementary CSI amplitude and phase information for respiration detection, we propose a complex system made up of the complementary CSI amplitude and phase as the real and imaginary parts, respectively, and model the complex CSI time series of different subcarriers as linear superposition of respiration signals. Stochastic and deterministic separation techniques are then used to extract the respiration source signals from the noisy, multi-modal CSI streams for stationary-person detection and monitoring in quasi-static environments.