Automatic 3D reconstruction of tumor using 2D H&E images

• Main Author: Ho, Adriel Jin Liang
• Other Authors: Cai Yiyu
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering; Engineering::Mechanical engineering
• Online Access: https://hdl.handle.net/10356/148915
• Institution: Nanyang Technological University

The analysis of normal and disease processes, especially when it involves structural changes or those in which the spatial relationship of disease features is significant, could benefit greatly from three-dimensional (3D) reconstruction and analysis of tissue at microscopic resolution. However, the process of 3D reconstruction is challenging because the solutions to produce an accurate model are often technical, complex, and time-consuming to learn. In this paper, we proposed an automatic 3D reconstruction method to reconstruct tumors from its Haematoxylin and Eosin (H&E) stained serial-sectioned tissues using the Python programming language to shield users from the difficult challenges that 3D reconstruction pose. In recent years, as advancements are made in the area of digital pathology, digital histopathology slides are more easily accessible. This encourages people to find innovative and effective ways to do diagnose patients. Through the stacking of 2D digital slides, 3D volumes can be created, enabling a deeper understanding of the specimen structure and easy interpretation of the growth patterns of a tumor. In addition to the advancement made in the field of digital pathology, the field of computer science and computer-aided engineering has also made huge technological leaps in areas that are relevant to 3D reconstruction technology such as image registration, image segmentation, and 3D modeling. Combining the technologies together allows the 3D reconstruction of a tumor from its 2D images to be an automatic, relatively computationally inexpensive, and quick process. In this project, we present a new pipeline to do automatic 3D reconstruction using the python programming language and great visualization software. The new pipeline follows the sequence of first segmenting the images, followed by registering the images and then the reconstruction. This project uses image segmentation and image registration techniques to create an accurate stack of 2D images for 3D reconstruction, and these techniques are implemented in Python 3. This project uses Fiji, an image processing program, an extension of ImageJ (also an image processing program) to visualize the 3D volume. This project also uses Hausdorff distance and Euclidean distance to validate the image registration implementation. The Hausdorff distance is highly resistant to circumstances involving many feature points, pseudo-feature points, and noise, and it, therefore, has a lower computational complexity. In addition to getting to see the structure of the tumor, the area and volumetric information of the tumor was obtained from the implementation of the proposed methods which is essential to pathologists to allow them to determine the type of treatment to recommend better.