COMPARISON STUDY OF SUPPORT VECTOR MACHINE AND RELEVANCE VECTOR MACHINE IN BINARY CLASSIFICATION

• Main Author: Gabriel Novieanto Wijaya, Juan
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/81657
• Institution: Institut Teknologi Bandung

This study addresses the comparison of Support Vector Machine (SVM) and Relevance Vector Machine (RVM) in the context of classification problems. SVM is a machine learning algorithm that works by finding a hyperplane that maximizes the margin between two data classes, while RVM uses a Bayesian approach to produce a more sparse model with only a few relevance vectors. Relevance vectors are data points in the RVM model selected through a Bayesian learning process to form a sparse and efficient predictive model, similar to support vectors in SVM but chosen based on a probabilistic framework. This study aims to provide an in-depth understanding of these two algorithms, including their mathematical derivations, and to evaluate the performance and efficiency of each method. Through a series of simulations on various datasets, this research finds that RVM tends to produce better predictive accuracy than SVM for most kernels used, except for the radial basis function (RBF) kernel. Additionally, the number of vectors produced by RVM is relatively fewer compared to SVM, indicating that RVM is more efficient in terms of resource usage. However, the model fitting process of RVM is slower compared to SVM due to the Bayesian approach requiring iterations to find relevance vectors. The results of this study show that although RVM is less popular than SVM, it has advantages in producing sparser and more efficient models, as well as better accuracy in some cases. Therefore, RVM has great potential for application in various fields, including the medical domain where probabilistic interpretation and computational efficiency are crucial.