DEVELOPMENT OF KNOWLEDGE BASED LINKS PREDICTION SYSTEM IN SPARK DISTRIBUTED SYSTEM

• Main Author: Thamrin Andrew H Siho, Timothy
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/51443
• Institution: Institut Teknologi Bandung

DBPedia, Freebase, and other knowledge bases have sparse connectivity, where it is a major challenge in predicting linkages on existing entities. Some recommendation systems use complex models to perform link predictions. In the midst of the rapid development of the internet in the world, accompanied by ever larger data movements, the existing knowledge base is also getting bigger. So, it is also necessary to build a parallel and distributed model so that development can be carried out more quickly and efficiently. Therefore, the final project regarding the construction of a knowledge base link prediction system is carried out. In this study, it was determined how to encode entities and also assess relationships in parallel and distributed by utilizing various libraries on Spark such as Word2Vec, TF-IDF, vector multiplication, and several other processing carried out using a map on Spark. This study uses xLearn as a factorization machine. Based on the research conducted, it was found that the system built successfully carried out all the model building processes in the Spark distributed system. The system can build models more efficiently and generate link predictions with FMR 27 and FMRR 0.56 values.