A multimodal evaluation approach for cost-effective emotion classification - Pinto, Gisela Antunes (2020)
The identification of emotions can improve people’s emotional regulation strategies and interaction with their multiple contexts of life. There are several studies on emotional classification systems. However, the vast majority investigate only the use of one or more isolated signals. The fact that several studies explain how informative individual signals are and how their combination works would allow developing more economical, informative, and objective systems to detect, process, and interpret emotions. In this work, Electrocardiogram (ECG), Electromyogram (EMG), and Electrodermal Activity (EDA) were processed in order to find a physiological model of emotions. Both, the unimodal and multimodal approaches, were used to analyze which signal or combination of signals can best describe an emotional response, using a sample of 55 healthy individuals. The method was divided into: (1) pre-processing; (2) extraction and selection of features; (3) classification using Random Forest and Neural Network. The results suggest that the ECG signal is the most effective for the classification of emotions. However, the combination of all signals provides the best emotion identification performance, with all signals providing crucial information for the system. This physiological model of emotions has important clinical and research implications. It provides relevant information on the value and weight of physiological signals for emotional classification, which can lead to critical assessment, monitoring and processing, and emotional regulation.