Online deep transfer learning applied to building quality assessment robots

• Main Author: Liu, Lili
• Other Authors: Chen I-Ming
• Format: Theses and Dissertations
• Language: English
• Published: 2019
• Subjects: Engineering::Mechanical engineering::Robots
• Online Access: https://hdl.handle.net/10356/104468; http://hdl.handle.net/10220/50012
• Institution: Nanyang Technological University

Post-construction quality assessment is critical to the building projects. It is labour intensive and time consuming. The results of the assessment depend on the examiner performing the assessment and are therefore subjective – people may have various opinions about an assessment and people may make mistakes; therefore, different examiners may give different results. Recent development in artificial intelligence techniques has enable design of an automated system for building quality assessment, to increase objectivity and accuracy, and reduce labour costs. This motivated current research, thereby establishing an automated post-construction quality assessment system for detecting various types of defects, such as cracks, finishing defects and hollowness. Compared to traditional methods, the system greatly reduces labour costs and provides a fast, objective and accurate assessment. In the proposed system, transfer learning for convolutional activation feature (TLCAF) networks, active-TLCAF (A-TLCAF) and online-TLCAF networks are employed for task automation. In the TLCAF network, faster R-CNN test mode is used as the base model for the proposal of region of interest (ROI), and a deep transfer learning (DTL) network is employed for model training and defect classification; finally, non-maximum suppression (NMS) and threshold adjustment are performed for defect detection. The active TLCAF (A-TLCAF) network allows users to actively intervene the labelling work of the top-N ROIs, and fine-tune the networks using the newly labelled images. Compared with TLCAF, A-TLCAF can improve the detection accuracy. To improve learning speed and to achieve incremental learning without forgetting existing knowledge, an online deep transfer-learning network is also proposed. The network is termed as online-TLCAF, whereby YOLO is used as the underlying network to deliver generic objects, convolutional neural networks are employed for extraction of features of visual defects, and broad learning algorithm is used for incremental learning. The system provides generalization capabilities for function approximation and simplifies the final structure using singular value decomposition (SVD). Compared with TLCAF, online-TLCAF has two improvements: 1). the ROI proposal network is replaced by an automated object proposal, which eliminates the need for ROI labelling work; 2). the linear classifier in TLCAF is replaced by an online learning system. The online-TLCAF network proposed in this study provides an incremental learning for high-dimensional dynamic image/video streams. Extensive experiments were conducted in the CONQUAS room, the test bed and our self-built data set, and the results were used to validate the developed automated post-construction quality assessment system. Various learning algorithms have been developed to illustrate the power of the proposed framework. The new method is satisfactory in evaluating various image-based defects. Compared to shallow structures, online-TLCAF provides greater flexibility for image/video-based object detection. Compared to traditional manual inspections, this automated system is suitable for large area inspections and increasing efficiency and reliability.