Effective salience-based fusion model for image retrieval

• Main Author: Mansourian, Leila
• Format: Thesis
• Language: English
• Published: 2016
• Online Access: http://psasir.upm.edu.my/id/eprint/68606/1/FSKTM%202016%2014%20IR.pdf; http://psasir.upm.edu.my/id/eprint/68606/
• Institution: Universiti Putra Malaysia

Recently Bag of Visual Words (BoVW) has shown promising results for image annotation and retrieval tasks. In the traditional BoVW model, all visual words are collected and treated the same, regardless of whether or not they are from an important part or the background of a picture. Traditional Scale Invariant Feature Transform (SIFT) features have no spatial information; therefore, the recognition of diffcult objects requires more attention. The first objective of this thesis was to develop a new BoVW model, the Salient Based Bag of Visual Word (SBBoVW) model, to recognize diffcult objects that previous methods were unable to accurately identify. This new model collects visual words based on their importance and combines several Pyramidal Histogram of visual Words (PHOW) feature vectors from the salient, rectangular part of a picture, as well as from the whole picture, to overcome the above-mentioned problem. After implementation, it was found that this method of feature extraction affects the accuracy of the results, which were more accurate than results obtained using seven other state-of-the-art models. However, the SBBoVW model focused only on gray-scale pictures.Previous research found that integrating color, significantly improved the overall performance of both feature detection and extraction because color is an important characteristic of human vision. Based on the literature, most of the image classification strategies have been developed for gray-based SIFT descriptors. Since color content is ignored, misclassification may occur. The Dominant Color Descriptor (DCD) is the best color descriptor for region color and the focus of improvements because it is a low-dimensional or less expensive descriptor representing colors in images. The DCD uses one to eight colors for each picture, and one to four colors for each region. However, some background colors are not used in the object of an image. Therefore, the second objective of this research was to establish a new Salient Dominant Color Descriptor (SDCD) to estimate the number of colors in a salient region using an easily implemented algorithm. Based on the results, it was found that if the maximum Euclidean color distance (dmax) was set to 20, as suggested by other researchers, more accurate results were obtained.The DCD is both low-dimensional and less expensive for representing image colors compared to the previous BoVW model that concentrated on the Color Scale Invariant Feature Transform (CSIFT), combinations of color SIFTs extracted from different color spaces, and opponent-color SIFTs extracted from opponent color spaces to add color information to a SIFT. Therefore, the final objective of this research was to develop a late fusion model, the SDCD BoVW and SBBoVW model. This model fuses the SDCD BoVW, and SBBoVW models using late fusion from histograms and is a comprehensive model for color object recognition. After implementation, the final proposed model provided more accurate results than the other three state-of-the-art models mentioned here and 19 additional color feature extraction methods.