Question answering model using deep learning with attention mechanism and its applications

• Main Author: Chew, Aaron Weng Kit
• Other Authors: Chen Lihui
• Format: Final Year Project
• Language: English
• Published: 2019
• Subjects: DRNTU::Engineering::Electrical and electronic engineering
• Online Access: http://hdl.handle.net/10356/77955
• Institution: Nanyang Technological University

This project is about the experimental study and implementations of Question & Answer(Q&A) systems for possible applications. Q&A is sub-domain under Natural Language Processing (NLP) that focuses on machine understanding and answering of human questions based on a given context. Background knowledge on traditional NLP methods and components reviewed and discussed. A comprehensive study was done on current state-of-the-art Q&A models and their useful components like the attention mechanism, the transformer, pointer networks. A performance comparison was made between available online implementations of the models and their published scores before being used as the implementation benchmark for SQuAD. Three revised models were subsequently proposed to explore if any improvements could be made on Q&A related tasks. BERT was considered for the expansions to the three different models namely Singlish BERT, Multilingual BERT and Q&A in a System, as BERT had the most generalised language model that could be tweaked to meet specific tasks. Despite having no improvements made, those three models were used as a platform to understand the concepts and implementations of the individual components in each model for integration with others for potential Q&A related applications. Prototypes of some applications have been developed to show the ideas and potentials of those systems. This report highlights the reviews, methodologies, implementation details used in the experimental studies, followed by discussions and analysis of the results obtained. These findings are important groundwork and as compilation of the important concepts that are necessary to understanding the NLP/Q&A domain. It also highlights a clear direction on what can be further improved backed by comprehensive tests and results