HDIdx: High-dimensional indexing for efficient approximate nearest neighbor search

• Main Authors: WAN, Ji, TANG, Sheng, ZHANG, Yongdong, LI, Jintao, WU, Pengcheng, HOI, Steven C. H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2016
• Subjects: High-dimensional indexing; Approximate Nearest Neighbor Search; Product Quantization; Spectral Hashing; Computer Sciences; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/3413; https://ink.library.smu.edu.sg/context/sis_research/article/4414/viewcontent/HDIdx_Neurocomputing_2016_pp.pdf
• Institution: Singapore Management University

Fast Nearest Neighbor (NN) search is a fundamental challenge in large-scale data processing and analytics, particularly for analyzing multimedia contents which are often of high dimensionality. Instead of using exact NN search, extensive research efforts have been focusing on approximate NN search algorithms. In this work, we present "HDIdx", an efficient high-dimensional indexing library for fast approximate NN search, which is open-source and written in Python. It offers a family of state-of-the-art algorithms that convert input high-dimensional vectors into compact binary codes, making them very efficient and scalable for NN search with very low space complexity.