Outlier detection in complex categorical data by modeling the feature value couplings

• Main Authors: PANG, Guansong, CAO, Longbing, CHEN, Ling
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2016
• Subjects: Databases and Information Systems; Data Storage Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/7146; https://ink.library.smu.edu.sg/context/sis_research/article/8149/viewcontent/272.pdf
• Institution: Singapore Management University

This paper introduces a novel unsupervised outlier detection method, namely Coupled Biased Random Walks (CBRW), for identifying outliers in categorical data with diversified frequency distributions and many noisy features. Existing pattern-based outlier detection methods are ineffective in handling such complex scenarios, as they misfit such data. CBRW estimates outlier scores of feature values by modelling feature value level couplings, which carry intrinsic data characteristics, via biased random walks to handle this complex data. The outlier scores of feature values can either measure the outlierness of an object or facilitate the existing methods as a feature weighting and selection indicator. Substantial experiments show that CBRW can not only detect outliers in complex data significantly better than the state-of-the-art methods, but also greatly improve the performance of existing methods on data sets with many noisy features.