Optimization of Convolutional Neural Networks for Detection of Childhood Pneumonia using Neural Network Pruning

• Main Author: Qui,, Christian Michael
• Format: text
• Published: Archīum Ateneo 2021
• Subjects: n/a
• Online Access: https://archium.ateneo.edu/theses-dissertations/520
• Institution: Ateneo De Manila University

Pneumonia, a bacterial or viral infection of the lungs that causes the inflammation of the air sacs, is one of the leading causes of mortality of chil- dren in the world. Chest X-rays, one of the golden standard tool in determin- ing pneumonia, is mainly used to detect malignancy in the lungs. However, the process of analyzing may be very expensive for hospitals and medical centers, and time-consuming for the radiologist. Inter-observer variability with the diagnosis is very high since disagreements between radiologists is frequent. Considering that the design of convolutional neural networks makes it suited to process spatially distributed input such as images, the application of convolutional neural networks trained with chest X-rays to automate the diagnosis of pneumonia is viable. Although the use of con- volutional neural networks would be very helpful, the large amount of pa- rameters of the deep network and computation cost may render it unusable and inefficient in low power systems. Therefore, there is a need to optimize convolutional neural networks to reduce its size in memory, and to decrease the time of inference. This study initially develops optimized models using structured and unstructured pruning methods applied to three well known architectures in literature using a childhood pneumonia dataset: (1) VGGNet, (2) ResNet, (3) and DenseNet. These optimized models were (1) compared against each other, (2) to their original versions, and (3) to two state of the art lightweight v architectures: MobileNet and ShuffleNet. However, results showed that setting weights to zero whether in an unstructured or structured manner is insufficient to reducing the amount of resources consumed by the neural network. This study applies a physical pruning pipeline that builds on top of the already implemented filter pruning method by the removal of the ze- roed out filters. In order to determine the sparsity to optimize the ac- curacy to speed trade-off, an overall performance measure was used. To further improve its performance, a global weight pruning step was added to the pipeline. Although it performs comparably in terms of accuracy, speed, and size to implementations in literature, and against state of the art lightweight architectures, further refinement in the reconstruction of the convolutional layers is needed to minimize accuracy loss.