DEVELOPMENT OF VIDEO-BASED EYE TRACKING SYSTEM USING PRETRAINED CONVOLUTIONAL NEURAL NETWORK MODEL FOR COMPUTER CURSOR MOVEMENT

• Main Author: Manuel, Ivan
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/75403
• Institution: Institut Teknologi Bandung

For some individuals with limitations, such as those suffering from neurodegenerative diseases or disabilities, moving their body muscles is challenging. These limitations often act as barriers for them to lead their daily lives. Eye tracking technology can help solve problems for anyone who faces limitations in using their hands or mouth to interact with a computer. Companies like Tobii have been developing this technology for a long time, with one of its functions being computer usage. Hence, the purpose of this writing is to understand how to develop a video-based eye tracking system utilizing pretrained convolutional neural network models to move the computer cursor. This research was conducted using an eye tracker device mounted on a table, specifically with the computer's camera. Since this research builds an eye tracking system based on video, the camera used to capture eye images is the computer's webcam. The user's face image is detected using the MediaPipe library, which has pretrained convolutional neural network models to recognize facial landmark points. Then, the midpoint between the left and right eyelid edges is selected as the reference point for iris movement. The difference in distance between the eye's midpoint and the iris's midpoint is processed as data during the research. The system starts with a calibration program that asks the user to look at several points on the screen. Both of the user's eyes are tracked, and their average values are taken as the processed data. This data is collected and processed to obtain a linear regression model with the data as input and the target point as output. The linear regression model is then used by the system to predict the direction of eye movement on the screen. The combination of utilizing the PyAutoGUI library and the linear regression model results in a computer cursor movement system using eye tracking. The eye gaze direction on the screen is further refined with an average filter, which takes the average value from the last ten frames. Then, the system is tested by asking the user to move a small circle to the center of nine large circles scattered on the screen sequentially. Each target circle undergoes five trials, and the result of each trial is the coordinate of the small circle's center. The results of these trials are analyzed for accuracy and precision. The research system achieved an accuracy of 14.95 mm and a precision of 13.46 mm.