Image recognition using artificial intelligence (human-object interaction detection in workplace)

• Main Author: Lin, Sining
• Other Authors: Yap Kim Hui
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/157460
• Institution: Nanyang Technological University

The rapid evolution in deep learning for Computer Vision has accelerated the application of image analytics in many of our daily and work lives. In the Smart Factory context, image analytics can enable a more effective production monitoring process by interpreting the activities in the scenarios, playing a vital role in digital transformation. In recent years, a new rising field, Human-Object Interaction (HOI) detection, is established to progress image analytics for semantic understanding. Given its capability of comprehending the interaction relationship between a specific human-object pair, the project specifically focused on applying the HOI detection to learn and recognize the production activities in the factory workplace. The project built a new industrial dataset, the Human Interaction with Factory Objects (HIFO) dataset, to introduce 8 factory objects with 9 sets of new interaction categories that commonly occur in the context. A two-stage Spatially Conditioning Graph (SCG) model, distinguished by combining spatial and appearance features, was used to perform the HOI detection in the new workplace scenarios. The project used transfer learning, and implemented proposed improvement techniques including negative instance suppression and Online Hard Example Mining (OHEM) loss function, to fine-tune the pre-trained SCG model on the dataset. The best performance achieved 61.51% mAP for the test dataset. Lastly, a graphical user interface (GUI) was designed for user-ends to visualize the results of the detections. In the last part of the report, conclusions were drawn from the discussion of quantitative and qualitative evaluations. Some recommendations were proposed to improve the SCG model detection in the factory workplace settings in future work. Overall, the SCG model achieved strong performance in HOI detection in the new proposed industrial dataset, implicating its practical applications for the Smart Factory context.