MODELING PROSODIC FEATURE FOR EMOTION RECOGNITION IN INDONESIAN SPOKEN LANGUAGE

• Main Author: Mirza Fathan Al Arsyad, M
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/72129
• Institution: Institut Teknologi Bandung

Information conveyed through spoken language is not only conveyed through words, but also through fundamental frequency (pitch), intensity (volume), speaking rate and rhythm, and timbre, collectively known as prosody. A neural network-based model is developed to create an emotion classification system for Indonesian speech. Prosody as a feature in speech has long been a subject of research in various places. There have been many studies on modeling prosodic features for various purposes, such as automatic speech recognition, emotion identification, dialogue act classification, and many other studies that explore prosodic features in conversation. To build an emotion recognition model, a corpus containing various segmented Indonesian language speeches from various audio sources is used. Then feature extraction is performed on the corpus using feature sets such as eGeMAPS and INTERSPEECH 2009. A neural network-based model is then created using the extracted features. Prosodic features are used to train the system, and experimental results show an f-measure value of 0.39 for the system using the eGeMAPS feature set, which is used as the baseline for the study. The experiment then yielded a highest result of 0.568 using the eGeMAPS feature set which was reduced to 14 prosodic feature sets, and the model was built using SMOTE optimization to handle class imbalance in the dataset. Various other neural network optimization methods were also explored in this study, such as dropout layer, early stopping, and principal component analysis as an effort to reduce the dataset dimension.