An integrated data mining system to automate discovery

Many data analysts require tools which can integrate their database management packages (e.g. Microsoft Access) with their data analysis ones (e.g. SAS, SPSS), and provide guidance for the selection of appropriate mining algorithms. In addition, the analysts need to extract and validate statistical...

Full description

Saved in:
Bibliographic Details
Main Authors: CHUA, Cecil, CHIANG, Roger Hsiang-Li, LIM, Ee Peng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2000
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/1005
https://ink.library.smu.edu.sg/context/sis_research/article/2004/viewcontent/Integrated_Data_Mining_System_pv.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:Many data analysts require tools which can integrate their database management packages (e.g. Microsoft Access) with their data analysis ones (e.g. SAS, SPSS), and provide guidance for the selection of appropriate mining algorithms. In addition, the analysts need to extract and validate statistical results to facilitate data mining. In this paper, we describe an integrated data mining system called the Linear Correlation Discovery System (LCDS) that meets the above requirement. LCDS consists of four major sub-components, two of which, the selection assistant and the statistics coupler, are discussed in this paper. The former examines the schema and instances to determine appropriate association measurement functions (e.g. chi-square, linear regression, ANOVA). The latter invokes the appropriate statistical function on a sample data set, and extracts relevant statistical output such as ?2, and R2 for effective mining of data. We also describe a new validation algorithm based on measuring the consistency of mining results applied to multiple test sets.