Annotating videos that teach MS Excel and predicting mouse / keyboard actions

• Main Author: Tan, Genson Yao Jie
• Other Authors: Li Boyang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Large language models; Prompt engineering
• Online Access: https://hdl.handle.net/10356/175233
• Institution: Nanyang Technological University

This research paper explores the extraction of specific sentences from natural language as a foundational step towards building an Artificial Intelligence system for automating Microsoft Excel. The focus is on leveraging language models with the capability to extract intention and procedure sente...