A Model Driven Approach to Imbalanced Data Sampling in Medical Decision Making

• Main Authors: Yin H., Tze-Yun LEONG
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2010
• Subjects: Imbalanced data learning; Model driven sampling; Random sampling; Synthetic Minority Over Sampling (SMOTE); Databases and Information Systems; Health Information Technology
• Online Access: https://ink.library.smu.edu.sg/sis_research/2988; http://dx.doi.org/10.3233/978-1-60750-588-4-856
• Institution: Singapore Management University

Classification is an important medical decision support function that can be seriously affected by disproportionate class distribution in the training data. In medical decision making, the rate of misclassification and the cost of misclassifying a minority (positive) class as a majority (negative) class are especially high. In this paper, we propose a new model-driven sampling approach to balancing data samples. Most existing data sampling methods produce new data points based on local, deterministic information. Our approach extends the idea of generative sampling to produce new data points based on an induced probabilistic graphical model. We present the motivation and the design of the proposed algorithm, and compare it with two representative imbalanced data sampling approaches on four medical data sets varying in size, imbalance ratio, and dimension. The empirical study helped identify the challenges in imbalanced data problems in medicine, and highlighted the strengths and limitations of the relevant sampling approaches. Performance of the model driven approach is shown to be comparable with existing approaches; potential improvements could be achieved by incorporating domain knowledge. © 2010 IMIA and SAHIA. All rights reserved.