Disentangling latent space of variational autoencoder with distribution dependent guarantees for out-of-distribution detection and reasoning

• Main Author: Rahiminasab Zahra Reza (Zahra Rahiminasab)
• Other Authors: Arvind Easwaran
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/172959
• Institution: Nanyang Technological University

Cyber-physical systems (CPS) have diverse applications, especially in a safety-critical setting, such as autonomous cars (AV). In safety-critical systems, any mistake can lead to non-compensable results, such as losing individuals. Therefore, ensuring the safety of such systems is vital. Many safety-critical CPS use machine learning (ML) models to accomplish their goals, such as in object detection, image segmentation, etc. As a result, there is a need for approaches to examine the safety of generated predictions by such ML models. One of the presented problems in examining the safety of ML models is the outof- distribution (OOD) problem. In the OOD problem, the objective is to identify test samples that are drawn from distributions different from the distribution of training samples. For example, in an object detection application, if the ML model is trained with day images and the test sample is a night image, the test sample should be identified as OOD. Solving the OOD problem itself can be divided into two subproblems: OOD detection and OOD reasoning. In OOD detection, the goal is to identify whether or not the sample is OOD. In contrast, in OOD reasoning, the objective is to explain the OOD behavior. Approaches based on one-class classifiers have poor performance on real-world datasets with multi-label data and overlapping partitions. For example, in the nuScenes dataset, each image has multiple labels such as pedestrian presence, weather, etc. An image belonging to the low rain intensity partition can also belong to the no-pedestrian partition. As a result, in our first attempt, we create an ensemble of β-Variational autoencoders (EBVAE) as OOD detectors that can be trained with multi-label data. A β-variational autoencoder (β-VAE) model maps each input to lower dimensional latent representation and reconstructs the image based on learned representation. Each β-VAE in the ensemble learns a representation corresponding to one generative factor. Generative factors are critical factors in the image that are necessary for image reconstruction. This thesis focuses on meteorological (such as rain intensity) and background generative factors. Addressing the OOD problem for these factors is challenging as these factors affect all the pixels of the image and can be dependent. In each β-VAE, we identify the most sensitive representation dimension for each generative factor. Disentanglement is establishing one-to-many maps between generative factors and their representative latent dimensions. Disentangling the latent space of VAE or its variants is a key step for OOD reasoning when we use one VAE to learn known generative factors corresponding to the meteorological features and background of an image. Disentangling the latent space of VAE is only possible with bias on the model or supervision for data. In our second attempt to resolve the OOD detection and reasoning problem for multi-label data, we use one VAE to decrease OOD inference overhead. We enforce bias on the OOD model through hyperparameter tuning to disentangle generative factors. We call obtained model hyperparameter based disentangled β-VAE (HPVAE). We use change point detection approaches to consider the effect of time dependency between data samples on OOD detection and reasoning. HPVAE based solution generates one-to-many maps between generative factors and their corresponding latent dimensions that can differ during training and inference time. As a result, in our third effort to solve the OOD detection and reasoning problem, we augment disentanglement to the training process by adding disentanglement constraints as regularization terms to the loss function. We use matchpairing weak supervision for training the VAE with disentangled latent space. In the match-pairing setting, samples are divided into groups in which samples from the same group have the same value or range of values for specific generative factors. Achieving total disentanglement, even with supervision, is impossible in practice due to the presence of unknown generative factors, dependencies between different generative factors, etc. The disentanglement constraints are formed by fuzzy logic since using fuzzy logic helps to formalize partial disentanglement. We call the trained model by this approach weakly supervised logic variational autoencoder (WDLVAE). Finally, we propose the disentangled distilled VAE (DDV) to reduce the model size while preserving disentanglement properties. The motivation behind introducing this framework is to reduce required resources when we deploy OOD reasoners and detectors on resource-constrained devices. For model compression, we use student-teacher knowledge distillation. To ensure the disentanglement is preserved during model compression, we define the problem as a constrained optimization problem with disentanglement constraints. To provide a theoretical guarantee for disentanglement during distillation, we analyze the optimality of obtained solutions and use generalization bounds. We also evaluate our approach empirically by deploying the compressed model on a resource-constrained device.