Image processing and machine learning for IC image analysis and hardware assurance

• Main Author: Cheng, Deruo
• Other Authors: Gwee Bah Hwee
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Electrical and electronic engineering::Integrated circuits; Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/148478
• Institution: Nanyang Technological University

Integrated Circuit (IC) image analysis consists of several major steps including IC delayering, IC imaging, image annotation, schematic generation, and functional analysis. IC image analysis is one of the highly reliable approaches towards hardware assurance, such as failure analysis, functional verification, intellectual property protection and hardware trojan detection. The conventional approaches to IC image analysis and hardware assurance are generally human dependent with software assistance, which have become intractable in view of the increasing complexity of modern ICs. Research efforts have been devoted to automated IC image analysis and hardware assurance without human intervention. This thesis pertains to the investigation and proposal of image processing and machine learning techniques for highly automated IC image analysis and hardware assurance, with emphasis on examining the tradeoffs over the accuracy, robustness, required human intervention, and computational efficiency of the proposed techniques. This thesis proposes three categories of techniques (image processing, classical machine learning and deep learning) for IC image analysis and hardware assurance. To achieve high computational efficiency and competitive accuracy with minimal human intervention, we proposed two novel image processing techniques, namely the Global Template Projection and Matching (GTPM) method and the Tanimoto Convolution and Morphological Decision (TCMD) model, for transistor interconnection retrieval. The proposed GTPM method performs template matching on the query image at global scale instead of local regions as in the conventional approaches. It does not require an iterative scanning process, granting a higher computational efficiency. With experiments on a set of delayered IC images, the proposed GTPM achieves a mean intersection-over-union accuracy of 82.07% and a mean pixel accuracy of 88.54% on retrieving transistor interconnections, which are over 2 times higher than the reported image processing techniques. We have also demonstrated that our proposed GTPM method can be adapted to different IC image sets with minimal manual data labeling. Further to the proposed GTPM method, the proposed TCMD model incorporates prior domain knowledge on the templates of IC images into the feature extraction process to eliminate the expensive filter learning process. The required human efforts on data labeling and model training are significantly reduced. The proposed morphological decision strategy also increases the accuracy and robustness of the proposed TCMD model. Through evaluations on a set of delayered IC images, the proposed TCMD model is shown to be 3 times faster than the reported learning-based techniques while achieving comparable accuracy. To achieve high accuracy and high robustness with moderate human intervention and computational complexity, we proposed two classical machine learning models. We first proposed a Hybrid K-means Clustering and Support Vector Machine (HKCSVM) method to detect the via and metal line positions for the retrieval of metal layer interconnections. The proposed HKCSVM method embodies K-means clustering to provide candidate regions and support vector machine to provide the final positions of vias and metal lines at pixel-level, in contrast to the individual techniques which only provide the position of vias and metal lines at region-level. From experiments on a set of delayered IC images, the proposed HKCSVM method achieves an F-score of 99.82% on via detection, a mean intersection-over-union accuracy of 94.44% and a mean pixel accuracy of 95.83% on metal line segmentation, which are both higher than the reported techniques. For IC images at the poly layer with higher semantic complexity, we further proposed a poly line shape library comprising of a set of basic geometric shapes for describing various poly lines. Thereafter, we proposed a Hierarchical Multi-Classifier System (HMCS) to detect the contacts and to segment the poly lines by learning a classifier for each basic shape in the poly line shape library. The trained multi-classifiers are applied to IC images to detect the basic shapes, which are subsequently conjoined to form complete poly lines through a novel geometry-based probability model. The proposed HMCS achieves approximately 1.5 times higher accuracy on both contact detection and poly line segmentation in comparison to the reported techniques. We further investigated deep learning models to aim for superior accuracy and robustness to the aforementioned techniques, upon the availability of manual data labeling and computing capability. We first proposed a Fully Convolutional Network (FCN) with a recursive training strategy to improve the pixel-level precision on circuit components detection in delayered IC images and to mitigate annotation errors for IC image analysis. To further improve the training efficiency of the deep learning models which adopts backpropagation with iterative parameter tuning, we proposed a Semi-analytic Sparse Neural network (SSNnet) that incorporates a sparsely connected representation stage with a novel spatial sparse connection and a fully connected classification stage with an analytic optimization. The proposed SSNnet is optimized analytically with one-time learning that requires neither gradient computation nor iterative parameter tuning, making its training phase more efficient than the backpropagation-based models. In summary, this thesis has proposed two image processing techniques, two classical machine learning techniques, and two deep learning techniques for highly automated IC image analysis and hardware assurance. The proposed image processing techniques require minimal human intervention for achieving comparable accuracy with high computational efficiency. The proposed classical machine learning techniques achieve high accuracy and high robustness with moderate human intervention and computational complexity. The proposed deep learning techniques instead are capable of achieving superior accuracy and robustness with higher requirements on data labeling and computing resources.