DEVELOPMENT OF COMPUTER VISION AND INTERNET OF THINGS (IOT) MODELS ON IOT-BASED EARLY WARNING AND MONITORING SYSTEMS IN LOW INFRASTRUCTURE ENVIRONMENTS IN COAL MINES

• Main Author: Ghifari, Vito
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/85261
• Institution: Institut Teknologi Bandung

Workplace safety in mining environments is of paramount importance. Workers in coal mining operations, particularly heavy equipment operators, perform their tasks in shifts over relatively long periods each day. This poses safety risks in the operation of heavy machinery, especially when operators are unable to stay focused on their tasks, a condition referred to as deviation. Deviation conditions, such as drowsiness and smartphone use while driving, violate mining regulations and increase the risk of workplace accidents. Against this backdrop, a capstone research project was developed to create an IoT-based early warning and monitoring system to supervise drivers. This system works to reduce the risk of accidents by alerting drivers when signs of deviation are detected through image inputs. By integrating with a server and dashboard, the system enables driver monitoring via a web application. In this study, a deviation detection model and server communication were implemented on a mini-computer called Jetson Nano. From the experiments conducted during the development of the solution, the best model for detecting deviations through images was the YOLOv8n object detection model. Additionally, the development of deviation detection for driver facial features in cases of microsleep and yawning was supported by the Mediapipe library. The results showed that the average deviation detection accuracy achieved by this solution was 72%. For server connectivity, the MQTT protocol was used to handle limited network conditions. Overall, the computer vision and IoT system on the Jetson Nano performed well, with room for further development to improve deviation detection accuracy.