Distributed k-nearest neighbor queries in metric spaces

• Main Authors: DING, Xin, ZHANG, Yuanliang, CHEN, Lu, GAO, Yunjun, ZHENG, Baihua
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Algorithm; k nearest neighbor query; Metric space; Publish/subscribe; Query processing; Databases and Information Systems; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/4095; https://ink.library.smu.edu.sg/context/sis_research/article/5098/viewcontent/Ding2018_Chapter_DistributedK_NearestNeighborQu.pdf
• Institution: Singapore Management University

Metric k nearest neighbor (MkNN) queries have applications in many areas such as multimedia retrieval, computational biology, and location-based services. With the growing volumes of data, a distributed method is required. In this paper, we propose an Asynchronous Metric Distributed System (AMDS), which uniformly partitions the data with the pivot-mapping technique to ensure the load balancing, and employs publish/subscribe communication model to asynchronously process large scale of queries. The employment of asynchronous processing model also improves robustness and efficiency of AMDS. In addition, we develop an efficient estimation based MkNN method using AMDS to improve the query efficiency. Extensive experiments using real and synthetic data demonstrate the performance of MkNN using AMDS. Moreover, the AMDS scales sub-linearly with the growing data size.