Multi-Features Audio Extraction for Speech Emotion Recognition Based on Deep Learning

The increasing need for human interaction with computers makes the interaction process more advanced, one of which is by utilizing voice recognition. Developing a voice command system also needs to consider the user's emotional state because the users indirectly treat computers like humans in general. By knowing the type of a person's emotions, the computer can adjust the type of feedback that will be given so that the human-computer interaction (HCI) process will run more humanely. Based on the results of previous research, increasing the accuracy of recognizing the types of human emotions is still a challenge for researchers. This is because not all types of emotions can be expressed equally, especially differences in language and cultural accents. In this study, it is proposed to recognize speech-based emotion types using multi-feature extraction and deep learning. The dataset used is taken from the RAVDESS database. The dataset was then extracted using MFCC, Chroma, Mel-Spectrogram, Contrast, and Tonnetz. Furthermore, in this study, PCA (Principal Component Analysis) and Min-Max Normalization techniques will be applied to determine the impact resulting from the application of these techniques. The data obtained from the pre-processing stage is then used by the Deep Neural Network (DNN) model to identify the types of emotions such as calm, happy, sad, angry, neutral, fearful, surprised, and disgusted. The model testing process uses the confusion matrix technique to determine the performance of the proposed method. The test results for the DNN model obtained the accuracy value of 93.61%, a sensitivity of 73.80%, and a specificity of 96.34%. The use of multi-features in the proposed method can improve the performance of the model's accuracy in determining the type of emotion based on the RAVDESS dataset. In addition, using the PCA method also provides an increase in pattern correlation between features so that the classifier model can show performance improvements, especially accuracy, specificity, and sensitivity.

@article{Gondohanindijo2023,
  title     = {Multi-Features Audio Extraction for Speech Emotion Recognition Based on Deep Learning},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {14},
  number    = {6},
  year      = {2023},
  publisher = {The Science and Information Organization},
  author    = {Jutono Gondohanindijo and Muljono and Edi Noersasongko and Pujiono and De Rosal Moses Setiadi},
  doi       = {10.14569/IJACSA.2023.0140623},
  url       = {https://doi.org/10.14569/IJACSA.2023.0140623}
}