Collecting and annotating videos that teach MS Excel

This project focused on collecting data for the development of an Artificial Intelligence (AI) system capable of operating Microsoft Excel. The approach involved downloading Excel tutorial videos on YouTube and annotating specific events within the videos, such as mouse clicks and keystrokes, to dis...

Full description

Saved in:
Bibliographic Details
Main Author: Lin, Paing Khantt
Other Authors: Li Boyang
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2023
Subjects:
Online Access:https://hdl.handle.net/10356/171941
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
Description
Summary:This project focused on collecting data for the development of an Artificial Intelligence (AI) system capable of operating Microsoft Excel. The approach involved downloading Excel tutorial videos on YouTube and annotating specific events within the videos, such as mouse clicks and keystrokes, to distinguish intentions and procedures. The process of data annotation was complex due to the diverse instructional styles across different YouTube channels. Challenges arose in defining clear intentions and procedures, with some channels presenting content in a lengthy and intricate manner. Despite the challenges, the project successfully collected a substantial dataset, ensuring that only videos with manually added captions were considered to maintain data quality. Annotations were done in a concise and modular manner to enhance the dataset's usability for future machine learning endeavours. The project's findings underline the complexity of analysing instructional videos. Future work could involve expanding the annotated dataset and refining the annotation process to improve the AI system's training and performance.