Data stream mining

• Main Author: Wan, Li
• Other Authors: Ng Wee Keong
• Format: Final Year Project
• Language: English
• Published: 2009
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems::Database management
• Online Access: http://hdl.handle.net/10356/17010
• Institution: Nanyang Technological University

The data stream mining problem has been studied extensively in recent years, due to the greatease in collection of stream data. The essential to a data stream mining algorithms is that we can only read data once. Unfortunately, most of traditional data mining algorithms do not have such single-scan property. Usually, data stream is considered as semi-in¯nite. It is impossible to store all the past data with limited resources. Thus, mining high dimensional data streams is a challenging task. In this report, we are going to propose some interesting observations on feature quality stream(FQS), which is obtained from data stream in real time, and a frame- work to analyze such stream. The analysis results of FQS are used to reduce the dimension of data streams. We will also propose a data stream mining framework called MR-Stream. It is a e±cient data stream clustering framework with the following properties: (1) computes and updates synopsis information in constant time; (2) allows users to discover clusters at multiple resolutions; (3) determines the right time for users to generate clusters from the synopsis in- formation; (4) generates clusters of higher purity than existing algorithms; and (5) determines the right threshold function for density-based clustering based on the fading model of stream data. MR-Stream can be extend to solve classi¯cation problem. The classi¯cation results ob- tained from the online component of MR-Stream framework are in realtime. The result given by MR-Stream is presented as a probability distribution table over di®erent classes.