Hand Gestures Interaction in Augmented Reality (AR) Learning Application

• Main Author: Law, Irene Xin Lin
• Format: Final Year Project Report
• Language: English; English
• Published: Universiti Malaysia Sarawak, (UNIMAS) 2023
• Subjects: Q Science (General); T Technology (General)
• Online Access: http://ir.unimas.my/id/eprint/44053/1/Irene%20Law%20Xin%20Lin%20%2824pgs%29.pdf; http://ir.unimas.my/id/eprint/44053/2/Irene%20Law%20Xin%20Lin%20%28fulltext%29.pdf; http://ir.unimas.my/id/eprint/44053/
• Institution: Universiti Malaysia Sarawak

Augmented Reality (AR) is a technology that has become popular in this recent years. It can mix the virtual objects generated by computers into the real world. Hand gesture recognition is the gesture from human body language which can be recognized by the computer. AR and hand gesture recognition had faced numerous problems such as problems in target recognition, lack of interaction of Augmented Reality (AR), and high cost of hardware used. The project Hand Gesture Interaction in Augmented Reality (AR) Learning Application will be developed to solve this problem. The objective that will be used to solve the problem is to identify hand gesture techniques for learning applications, to design an interactive AR learning application using hand gestures, and to develop a prototype of a learning application with AR using hand gesture techniques. The methodology to develop the application of hand gesture interaction in AR learning is agile methodology. The process of the AR hand gesture by using ManoMotion is stated in the agile methodology. The outcome of the application is the user can use this application to learn astronomy courses such as Sun and Moon through AR. The user also can use the hand gesture to modify the size of the AR to make their learning more interactive. The testing results of the gesture detected to trigger the action reveal that the highest percentages are observed for Right Grab is 36% and Right Pick is 21% while Left Grab attained a rate of 29%, and Left Pick exhibited the lowest percentage at 14%. The limitations of the application include a limited set of hand gestures, the accuracy of detecting hand gestures being affected by lighting conditions and background noise, non-standard hand gesture performance, and hardware limitations. Future work includes exploring new hand gestures and integrating alternative SDKs, improving the ability to recognize various lighting conditions, enhancing hand detection accuracy, and leveraging smartphones with better hardware conditions