Predicting MS Powerpoint mouse/keyboard actions
This project explores the application of Generative Pre-trained Transformer (GPT) models, specifically GPT-2 and GPT-3, for predicting the textual instructions corresponding to user actions in Microsoft PowerPoint, such as mouse movements and keyboard inputs. Through extensive experimentation and im...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/175104 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Institution: | Nanyang Technological University |
| Language: | English |
| Summary: | This project explores the application of Generative Pre-trained Transformer (GPT) models, specifically GPT-2 and GPT-3, for predicting the textual instructions corresponding to user actions in Microsoft PowerPoint, such as mouse movements and keyboard inputs. Through extensive experimentation and implementation, we were
able to observe how soft prompting with GPT-2 and in-context learning with GPT-3 exceed baseline performance established through hyperparameter tuning of the GPT-2 model. This achievement is particularly notable in two domains: the prediction of user intentions and the prediction of procedural instructions. Hence, this study underscores the efficacy of these techniques in augmenting the capabilities of the employed models. By illustrating the potential of AI-driven solutions to streamline interactions with software applications, this work sets a foundation for a shift in user experience within productivity tools, driven by seamless, natural language commands. |
|---|