SAP: Improving continuous top-K queries over streaming data

• Main Authors: ZHU, Rui, WANG, Bin, YANG, Xiaochun, ZHENG, Baihua, WANG, Guoren
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Partitioning algorithms; Maintenance engineering; Monitoring; Temperature sensors; Heuristic algorithms; Complexity theory; Databases and Information Systems; Data Storage Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/3666; https://ink.library.smu.edu.sg/context/sis_research/article/4668/viewcontent/streamdata_paper.pdf
• Institution: Singapore Management University

Continuous top-k query over streaming data is a fundamental problem in database. In this paper, we focus on the sliding window scenario, where a continuous top-k query returns the top-k objects within each query window on the data stream. Existing algorithms support this type of queries via incrementally maintaining a subset of objects in the window and try to retrieve the answer from this subset as much as possible whenever the window slides. However, since all the existing algorithms are sensitive to query parameters and data distribution, they all suffer from expensive incremental maintenance cost. In this paper, we propose a self-adaptive partition framework to support continuous top-k query. It partitions the window into sub-windows and only maintains a small number of candidates with highest scores in each sub-window. Based on this framework, we have developed several partition algorithms to cater for different object distributions and query parameters. To our best knowledge, it is the first algorithm that achieves logarithmic complexity w.r.t. k for incrementally maintaining the candidate set even in the worstcase scenarios.