Paraphrase Identification in Vietnamese Documents

• Main Authors: Ngo, Xuan Bach, Tran, Thi Oanh, Nguyen, Trung Hai, Tu, Minh Phuong
• Format: Article
• Language: English
• Published: IEEE 2018
• Subjects: Paraphrase Identification; Semantic Similarity; Support Vector Machines; Maximum Entropy Model; Naive Bayes Classification; K-Nearest Neighbor
• Online Access: http://repository.vnu.edu.vn/handle/VNU_123/61163
• Institution: Vietnam National University, Hanoi

In this paper, we investigate the task of paraphrase identification in Vietnamese documents, which identify whether two sentences have the same meaning. This task has been shown to be an important research dimension with practical applications in natural language processing and data mining. We choose to model the task as a classification problem and explore different types of features to represent sentences. We also introduce a paraphrase corpus for Vietnamese, vnPara, which consists of 3000 Vietnamese sentence pairs. We describe a series of experiments using various linguistic features and different machine learning algorithms, including Support Vector Machines, Maximum Entropy Model, Naive Bayes, and k-Nearest Neighbors. The results are promising with the best model achieving up to 90% accuracy. To the best of our knowledge, this is the first attempt to solve the task of paraphrase identification for Vietnamese.