Using deep learning for quality control in cyber-manufacturing

• Main Author: Dai, Wenting
• Other Authors: Alexei Sourin
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/155249
• Institution: Nanyang Technological University

Even though automation has been applied in most manufacturing production lines, the inspection tasks including 1) defect identification, 2) defect categorization, and 3) defect localization or segmentation are still done by human operators in many factories. The poor accuracy of labour and time-intensive human visual inspection encourages the application of automatic optical inspection. Many approaches were proposed in this area, including both rule-based image processing approaches and machine-learning-based approaches. They gained good results. However, the rule-based approaches are often invented for only one specific component. It will be tedious if inspectors are required to design complicated rules for each component when there are plenty of components in one product. Regarding machine learning-based approaches, they have shown some extend of generalization ability, but the difficulty is in gaining sufficient data with labels for model training, which is also the obstruction of implementing many excellent supervised learning algorithms in the real world. Therefore, it was hypothesized that these problems can be addressed by the learning-based algorithms that can be generalized for inspection of various components and work with the limitation of data, especially the labelled data. Based on these two principles, unsupervised learning (including clustering and anomaly detection), and active learning are focused on in this thesis. Three different works have been done. The first one proposed an active learning framework for the automatic classification of manufacturing defects. This framework starts with the training of a classifier only on a small, labelled subset of training data. The small training set is then enlarged step-by-step by combining K-means clustering with active user annotations of selected representative samples. Accordingly, the trained classifier becomes accurate. Evaluations on a real manufacturing dataset have shown that the proposed method achieved high accuracy while requiring only minimal user annotations. For further minimizing the labelling work in industrial applications, a novel Self-supervised Pairing Image Clustering (SPIC) network was proposed to categorize defects, which produces clustering prediction in an advanced pair classification network. For training this network, a self-supervised pairing module was built to form balanced training pairs without label information. Besides, information-theoretic regularizations constrain the training to avoid trivial solutions. Experimental results indicate that SPIC outperforms the state-of-art approaches on manufacturing datasets–NEU and DAGM, as well as MNIST, Omniglot, CIFAR-10, and CIFAR-100 datasets. The last work solves the unsupervised defect detection and segmentation problem. A novel multi-stage image resynthesis approach was proposed, which is based on repairing suspicious regions of defective images. Each pixel of the image will be coarsely generated by its context in the proposed approach. Then, while excluding all safe pixels, the proposed method repairs suspicious regions that have large deviations to the original input image. After several iterations of reparation, the defects can be detected in the final residual maps between inputs and the repaired outputs. Besides an excellent capability of repairing defects, it also achieved better detection and segmentation performances than the state-of-art benchmarks using MVTec dataset and a real-world equipment surface dataset. In summary, this thesis mainly focuses on designing deep learning approaches that can assist quality control in cyber-manufacturing but work with limited supervision, which is critical for efficient resource usage especially at the beginning of inspection where the available dataset usually consist of much more normal samples than various abnormal samples. The thesis describes novel approaches to address different inspection questions and achieve state-of-the-art performance in both public datasets and real-world manufacturing datasets.