Collecting and annotating videos that teach MS Excel

• Main Author: Lin, Paing Khantt
• Other Authors: Li Boyang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Data
• Online Access: https://hdl.handle.net/10356/171941
• Institution: Nanyang Technological University

This project focused on collecting data for the development of an Artificial Intelligence (AI) system capable of operating Microsoft Excel. The approach involved downloading Excel tutorial videos on YouTube and annotating specific events within the videos, such as mouse clicks and keystrokes, to distinguish intentions and procedures. The process of data annotation was complex due to the diverse instructional styles across different YouTube channels. Challenges arose in defining clear intentions and procedures, with some channels presenting content in a lengthy and intricate manner. Despite the challenges, the project successfully collected a substantial dataset, ensuring that only videos with manually added captions were considered to maintain data quality. Annotations were done in a concise and modular manner to enhance the dataset's usability for future machine learning endeavours. The project's findings underline the complexity of analysing instructional videos. Future work could involve expanding the annotated dataset and refining the annotation process to improve the AI system's training and performance.