In-the-loop emotion recognition system for Human Machine Interaction (HMI)

• Main Authors: Ghazali, Aimi Shazwani, Sidek, Shahrul Na'im
• Format: Book
• Language: English
• Published: IIUM Press, International Islamic University Malaysia 2017
• Subjects: TJ Mechanical engineering and machinery
• Online Access: http://irep.iium.edu.my/61941/1/61941_In-the-loop%20emotion%20recognition%20system%20for%20human%20machine.pdf; http://irep.iium.edu.my/61941/; http://iiumpress.iium.edu.my/bookshop
• Institution: Universiti Islam Antarabangsa Malaysia

In the 21st century, there will be more machines developed either to complement or totally replace the jobs previously done by human since the machine can operate with high precision and accuracy. However many machines do not have ways to incorporate and respond to the emotion of the user. The efficacy of the system is further reduced if the machine has to take commands from human in order to operate. Thus, it is essential for a machine to understand the users’ feeling and react accordingly especially for Human Machine Interaction (HMI) applications. The existing emotion recognition systems have two major weaknesses which are the identification is limited to certain group of people and the hassle to wear the sensor by the user. Thus, an emotion recognition system is developed based on the machine learning technique that can be used throughout a wider range of human population as well as it is hassle free as there will not be any sensors attached to the body of the human subject. The audio-visual stimulants are used to invoke the desired emotions which are happy, sad and nervous taken from video-sharing website which consists of a ten minutes video for each session. After each session, the subject is given a set of questionnaire with Likert Scale of 4 to evaluate the audio-visual stimuli effectiveness to invoke the required emotion. The identification of the emotion is deduced from the measured electromagnetic (EM) signals radiated from the human body by a handheld device called Resonant Field Imaging (RFITM). From the dataset obtained consisting ten points of interests (POIs) on the human body, the signals are fed into Bayesian Network (BN) to classify the emotion under study. The classification of the EM dataset results accord 86% precision and 90.7% accuracy by using BN. A dedicated Graphical User Interface (GUI) is developed to display the corresponding classified emotion and as a medium to pass the coded emotion to the hybrid automata system. The hybrid automata system is selected as a framework to embody emotion in controlling the rehabilitation robot platform. The results obtained from the dataset show promising trends where the emotion recognition system is able to classify the type of emotions with high accuracy and the hybrid automata system is feasible in controlling the movement of the rehabilitation platforms’ end effector through a series of offline and online experiments. The limitation of the developed system is there are only three emotions under study due to the natural emotion of the post stroke patient who is undergoing rehabilitation therapy