Dimensionality reduction in deep neural networks - Wee, Keane Jin Yen (Nanyang Technological University, 2025)
Deep Neural Networks (DNNs) are powerful Artificial Intelligence (AI) techniques that enable machines to learn complex patterns from high-dimensional data. Due to the increasing complexity of data points and parameters in the model, DNNs experience the curse of dimensionality, where the model's efficiency exponentially decreases as the dimensionality increases. To solve this, dimensionality reduction techniques are often applied to feedforward DNNs to retain essential information and improve computational performance. However, the underlying mechanisms and explainability of these techniques remain insufficiently understood. This project aims to improve the explainability of feedforward DNNs and its usage of dimensionality reduction techniques. Specifically, we explore linear techniques such as Principal Component Analysis (PCA) and Singular Value Decomposition (SVD), alongside the nonlinear t-distributed Stochastic Neighbour Embedding (t-SNE), to evaluate how the feature representations evolve across network layers. We further assess the separability and structure of these representations using clustering methods including k-means algorithms and hierarchical clustering.