Mobile visual search information services

• Main Author: Ho, Bao Xian
• Other Authors: Yap Kim Hui
• Format: Final Year Project
• Language: English
• Published: 2015
• Subjects: DRNTU::Engineering::Electrical and electronic engineering::Computer hardware, software and systems
• Online Access: http://hdl.handle.net/10356/62067
• Institution: Nanyang Technological University

Mobile visual search (MVS) is form of search performed on a mobile phone where images are used as a basis for the search rather than text. To build the search engine, the Bag-of-Words Model was used. Images are described using local feature descriptors and occurances of each descriptor are stored in a histogram. A database is then built and stored using a scalable vocabulary tree with inverted file indexing. Query images are then compared with the database images and each database image will be given a score. The category that the query image is in is then decided by finding the category of the database image that has the highest score. The methodology adopted by past FYP and PhD students requires an active internet connection for the search to work. This means that it is not economical for the app to be used overseas where data charges are high. Therefore, one of the objectives of this project is to build a MVS app that can perform MVS without the need of an active interconnection. The other objective of this project is to build augmented reality (AR) applications based on the MVS technology. In this report, the tools used to build the apps and the process of how the applications were built will be discussed. Three apps were developed in the course of this project. One of the app developed can perform both online and offline MVS. The Nanyang Technological University (NTU) landmarks database was used to test this app and it could perform an offline search in (3.42 ± 1.09) seconds using a Samsung Galaxy Note 3 with an accuracy of 84.9%. The app developed also spots advance camera functionalities such as zooming, auto focusing and rotating buttons based on the orientation of the phone. Two AR MVS applications that can perform online mobile visual search were also developed. They are a generic AR MVS app and an AR MVS cataloguing app. The generic AR MVS app spots a full functionality results page including tabs to display various metadata about the object found. The tabs were designed in such a way that it can be easily modified to cater for any databases used. This app is tested using a book cover database and it has an accuracy of 98.2% and a search time of (0.791 ± 0.196) seconds. The AR MVS cataloguing app developed is an app that will generate a catalogue of products after scanning many products. This app was tested using a commercial product database and it has an accuracy of 85.2% and a search time of (0.969 ± 0.321) seconds