Chinese poem generation using natural language processing

• Main Author: Wang, Jihan
• Other Authors: Ling Keck Voon
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/169119
• Institution: Nanyang Technological University

The objective of this project is to investigate and construct multiple NLP-based models to generate Chinese poetry. Initially, an experiment utilizing LSTM was conducted to evaluate the significance of character selection, whereby the performance of the LSTM model was compared with and without the input dictionary which contains the frequently used words. The outcome shows that the model which starts to generate poems from the commonly used words has a better performance than the other, illustrating the importance of character selection. Subsequently, to enhance the performance and investigate the function of the Transformer, we extended the dataset and trained a Transformer model. The Transformer model's performance was then compared against the LSTM model and two prominent applications. The outcome shows our model has a better performance than others. Additionally, auxiliary functions were developed, including a sequence-to-sequence model to facilitate classical and modern Chinese translation for users to better comprehend classical Chinese, and a BERT model for keyword extraction to aid users' understanding of the text. Finally, an elegant UI was created and linked to the NLP models.