Are we all good actors? A study on the feasibility of generalizing speech emotion recognition models

Speech is one of the most natural way for human to express emotions. Emotions play also a pivotal role in human-machine interaction allowing a more natural communication between the user and the system. This led in recent years to a growing interest in Speech Emotion Recognition (SER). Most of the SER classification models are based on specific domains and are not easily generalizable to other situations or use cases. For instance, most of the datasets available in the literature are acted utterances collected from English adults and thus not easily generalizable to other languages or ages. In this context, defining a SER system that can be easily generalised to new subjects or languages has become a topic of relevant importance. The main aim of this article is to analyze the challenges and limitations of using acted datasets to define a general model. As preliminary analysis, two different pre-processing and features extraction pipelines have been evaluated for SER models able to recognize emotions from three well-known acted datasets. Then, the model that achieved the best performance was applied to new data collected in more realistic environments. The training dataset is Emozionalmente, a large italian acted dataset collected using a crowdsourced platform from non-professional actors. This model was tested on two subsets of the Italian speech emotion dataset SER_AMPEL, to evaluate the performance in a more realistic context. The first subset comprises audio clips from movies and TV series performed by older adult dubbers, while the second one consists of natural conversations among individuals of different ages. The analysis of performances and results have highlighted the main difficulties and challenges in generalize a model trained on acoustic features to new real data. In particular, this preliminary analysis has shown the limits of using acted dataset to recognize emotion in real environment.