Deep learning-based concrete image analysis and generalization capability

• Main Author: Qian, Hanjie
• Other Authors: Xie Lihua
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Engineering; Deep learning; SEM; Concrete; Micro-structure
• Online Access: https://hdl.handle.net/10356/180162
• Institution: Nanyang Technological University

Concrete is one of the most widely used materials in the world, playing a fundamental role in modern human civilization. The analysis of concrete's microstructure contributes to the design and improvement of concrete compositions. Scanning electron microscopy (SEM) imaging is the most common and direct method for analyzing the microstructure of concrete. However, previous research on SEM image analysis has encountered several challenges: low accuracy and efficiency, lack of quantitative research, poor generalization capability, and dependency on labels. The aim of this thesis is to develop a highly automated and accurate method for analyzing concrete SEM images while improving generalization capability through transfer learning. To achieve this, we propose analysis algorithms based on deep learning and design methods to enhance generalization capability and alleviate the reliance on labels. One long-standing problem in the analysis of concrete SEM data is the heavy reliance on researchers' personal experience or pixel-based shallow features, which can only perform simple qualitative analysis and lack automation and precision. In this thesis, we first establish a deep learning-based SEM data classification framework, followed by an investigation into the impact of pre-trained models using transfer learning techniques. The results demonstrate that our approach is more robust and achieves higher accuracy compared to traditional methods. Moreover, further improvements in model performance can be achieved by pre-training on different concrete SEM datasets. The limited efficiency and accuracy of previous algorithms for SEM data analysis have hindered quantitative research in analyzing SEM data. Additionally, the presence of significant noise and irregular boundaries in SEM images also poses considerable challenges for quantitative analysis. To solve these problems, we propose an improved segmentation algorithm, specifically focusing on edge segmentation in SEM images, effectively enhancing the accuracy of SEM image segmentation. We also compare our results with chemical analysis, validating the feasibility of the proposed algorithm. Even with accurate algorithms, the issue of generalization capability cannot be avoided in the analysis of SEM data. When deploying models trained on one SEM dataset to new scenarios, such as different compositions or regions, a significant decrease in performance is often observed. Since the training data and test data belong to different distributions, and directly applying a trained model can be highly risky. Therefore, it is necessary to reduce the differences between data from different sources to ensure algorithm applicability. In this thesis, we propose an algorithm that utilizes an autoencoder structure to selectively exclude training data that deviate from the target data distribution, retaining data that are more representative of the target distribution. This approach enables the training of more generalized models. Results on SEM data demonstrate that our proposed algorithm achieves better performance by removing a portion of the training data. Furthermore, experiments on a popular visual dataset further demonstrate the effectiveness of our method not only for SEM data but also for general tasks. Another generalization capability issue related to models trained on SEM data is the unavailability of labels in new scenarios, which may have data from a different distribution. In such cases, obtaining labels becomes unfeasible. However, traditional research has often overlooked this constraint and instead selected the best results based on the model's performance on new data. To address this challenge, we propose a label-free model evaluation method that combines deep feature information and model parameter information. This method enables the evaluation of different algorithm performances and determines when to stop iteration to avoid overfitting. We conducted extensive experiments on SEM data and popular visual datasets, demonstrating the effectiveness of our method in evaluating the performance of an algorithm, not only for SEM data but also for general data. In summary, this thesis addresses the long-standing issues of low accuracy, low efficiency, poor generalization capability and label dependency in previous studies based on concrete SEM data. We present more accurate and automated algorithms by considering the characteristics of SEM data. Additionally, we enhance generalization capability by proposing an algorithm to reduce the differences between SEM data distributions, significantly improving performance on different domain data. To reduce the reliance on labels, we introduce a label-free evaluation method. These methods are not only effective for the analysis of concrete SEM data but also help with the practical application of general deep learning algorithms.