Developing a speech emotion recognition model using RNN-LSTM technique

Abstract

The growing significance of emotionally intelligent systems in domains such as healthcare, education, and customer service has intensified research activities in speech emotion recognition (SER). This study presents the development of a deep learning-based SER model utilising a Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM) units to improve the accuracy and robustness of emotion detection and recognition within speech signals. Four publicly available emotional speech datasets Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), CREMA-D, Surrey Audio-Visual Expressed Emotion (SAVEE), and Toronto Emotional Speech Set (TESS) were integrated to form a diverse and balanced corpus of 12,798 audio samples covering seven emotion categories. The speech dataset underwent rigorous pre-processing such as noise reduction, normalisation, and data augmentation and feature extraction using Mel-Frequency Cepstral Coefficients (MFCCs) and Mel-Spectrograms to capture spectral and temporal dynamics. The model architecture incorporated bidirectional LSTM layers with dropout regularisation, trained using a PyTorch framework and optimised with the Adam algorithm. Evaluation metrics such as accuracy, precision, recall, and F1-score were used to assess performance of the model. The developed model achieved superior generalisation compared to traditional machine learning approaches such as support Vector Machine (SVM), K-Nearest Neighbours (KNN), artificial neural network (ANN), demonstrating its ability to effectively capture long-term dependencies and temporal variations in speech data. The study’s findings confirm the efficacy of RNN-LSTM architectures in SER tasks, with a validation accuracy of 89.50%, and contribute to the advancement of affective computing by enhancing human–computer interaction through emotion-aware systems.