Comparison of robust estimators for detecting outliers in multivariate datasets

• Main Authors: Sharifah Sakinah, Syed Abd Mutalib, Siti Zanariah, Satari, Wan Nur Syahidah, Wan Yusoff
• Format: Conference or Workshop Item
• Language: English
• Published: IOP Publishing 2021
• Subjects: Q Science (General); QA Mathematics
• Online Access: http://umpir.ump.edu.my/id/eprint/35199/1/Comparison%20of%20robust%20estimators%20for%20detecting%20outliers%20in%20multivariate%20datasets.pdf; http://umpir.ump.edu.my/id/eprint/35199/; https://doi.org/10.1088/1742-6596/1988/1/012095
• Institution: Universiti Malaysia Pahang

Detecting outliers for multivariate data is difficult and does not work by visual inspection. Mahalanobis distance (MD) has been a classical method to detect outliers in multivariate data. However, classical mean and covariance matrix in MD suffer from masking and swamping effects. Masking effects happened when outliers are not identified and swamping effects happened when inliers are identified as outliers. Hence, robust estimators have been proposed to overcome these problems. In this study, the performance of a new robust estimator named Test on Covariance (TOC) is tested and compared with other robust estimators which are Fast Minimum Covariance Determinant (FMCD), Minimum Vector Variance (MVV), Covariance Matrix Equality (CME) and Index Set Equality (ISE). These five robust estimators' performance is being tested on five real multivariate datasets. Brain and weight, Hawkins-Bradu Kass, Stackloss, Bushfire and Milk datasets were used as these five real datasets are well-known in most outlier detection studies. Results show that TOC has proven to be able in detecting outliers, does not have a masking effect and has the same performance as other robust estimators in all datasets.