Image retrieval with deep learning

• Main Author: Tan, Joe Chin Yong
• Other Authors: Lin Weisi
• Format: Final Year Project
• Language: English
• Published: 2017
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: http://hdl.handle.net/10356/72791
• Institution: Nanyang Technological University

For many computer vision problems, the deep neural networks are trained and validated based on the assumption that the input images are pristine (i.e., artifact-free). However, digital images are subject to a wide range of distortions in real application scenarios, while the practical issues regarding image quality in visual system have been largely ignored. In this thesis, a study of performing image retrieval with deep learning via TensorFlow and the VGG Net is first reported; then we conduct an evaluation of which for image retrieval under quality distortions in terms of Gaussian Blur, Gaussian Noise and JPEG Compression. The features of the pristine image database are first extracted, then retrieval is performed using pristine query images to the database to attain the baseline mean average precision (mAP). The query images are distorted with the 3 methods mentioned with different values of sigma, variance and quality. Blur in images can occur when the camera is out of focus. Noise in images usually happens when shooting in low-light environments and JPEG compression takes place when the quality value is low. All the distorted query images are used to perform retrieval to see the effects of distortion query images to the performance of retrieval. Among the different distortion methods, Gaussian Noise drastically affects the performance, Gaussian Blur affects the performance linearly to the increasing value of sigma, and JPEG Compression does not affect much unless very low quality value is used. Actions can be done to fine-tune the feature extractor with distorted images to see whether it will be more resilient to these distortion methods. Further studies can be made on how reversing the distortion with techniques like image sharpening and noise reduction affects the performance.