Application of Affective Computing in the Analysis of Advertising Jingles in the Political Context

—Affective computing is an emerging research area focused on the development of systems with the ability to recognize, process and simulate human emotions in order to improve a user’s experience in an interactive system. One of the possible fields of application of affective computing is marketing and advertising, where the application of emotion analysis techniques on the opinions made by users in different contexts has been evidenced, being a challenge the analysis of different types of multimedia content, such as advertising jingles. Thus, in this article we propose as a contribution the development of an emotion analysis study on the advertising jingles of the main candidates for mayor of Cartagena-Colombia in the period 2020-2023. For the development of this study, the acoustic properties of arousal and valence present throughout the audio track of each advertising jingle were taken into consideration, in such a way that through these properties it is possible to classify an audio fragment into an emotion belonging to the circumflex model or Russell model. To perform the segmentation of the audio track and the extraction of the acoustic properties of arousal and valence, we developed the MUSEMAN tool, which allows determining the emotions fluctuation in the advertising jingle audio track.


I. INTRODUCTION
One of the most effective means of promoting products, services and even political campaigns is advertising jingles. These are broadcast on radio networks, television channels, web portals and social networks in order to promote the use of a product or service or motivate the intention to vote for a candidate [1]. Given that advertising jingles comprise an audio component, it is possible to analyze the emotionality present during the audio track using the methods and techniques provided by affective computing [2]. Research in affective computing focuses on systems being able to interpret the emotional state of a human and obtain or provide an appropriate response in real time to the identified emotion [3]- [7], which can be used to improve the interaction of a user with a service, content, or specific product [8].
A common approach to detecting emotions in acoustic content is the circumflex model, or Russell's model, which proposes to represent emotional states in two dimensions [9]- [11]: The horizontal dimension (x-axis), called valence, expresses the degree of positivity or negativity of the emotion (pleasure/displeasure), and the vertical dimension, called arousal, indicates the degree of excitement of the emotion (arousal-relaxation) [12]. Emotions can therefore be defined in regions within the emotional plane as a combination of valence and activation [13], [14]. In detecting emotions in music, the arousal value represents the intensity and activity in the course of an audio track, so that fast tracks and loud sounds will have a high arousal value, while tracks with a softer sound and slower tempo will have a low arousal value. Valence thus represents the acoustic property that describes the positivity present in an audio track, i.e. tracks with a positive valence are associated with emotions such as joy, euphoria, and happiness, while tracks with a negative valence correspond to emotions such as sadness, depression, or anger [15]. In this way, from the musical properties of arousal and valence of an audio track, by mapping them in the emotional plane it is possible to obtain the trigonometric angle and the emotion associated with the Russell model, thereby avoiding taxonomic ambiguity [16].
Based on the above and taking advantage of the advantages provided by the acoustic properties of arousal and valence, in this article we proposed as a contribution the development of a study based on emotion analysis on the advertising jingles of the candidates for mayor of Cartagena-Colombia for the period 2020-2023, in which the extraction and segmentation of the audio of each jingle was performed in order to obtain the emotion associated with each segment taking into account the Russell model and with the aim of obtaining the trace of emotions of each jingle. To carry out the above process, the MUSEMAN (Music Emotion Analyzer) tool was developed in Java language, which allows automating the above-mentioned analysis process using free libraries and technologies (FFmpeg, openEAR and JFreeChart). The FFMpeg tool allows the extraction and segmentation of the audio track into fragments of 5 seconds or less. The openEAR tool was used in the background and allows obtaining the acoustic variables of arousal and valence for a given audio fragment, which allow classifying each fragment in Russell's two-dimensional model. Finally, the JFreeChart library was used to generate the emotion trace graph for each jingle, as well as to generate the spider web graph with the distribution of emotions for each jingle. Thus, the present proposal has as a contribution with respect to other works of the state of the art the application of affective computing in the analysis of advertising jingles in the political context, as well as the automation of the process of analysis of emotions in multimedia audio contents. The www.ijacsa.thesai.org approach proposed in this article is intended to support content creators in the objective design of the emotions communicated in multimedia advertising content, in order to generate greater interest in the audience.
The article is organized as follows: Section 2 describes a set of related works that were taken into account for the development of the present research; Section 3 presents the materials and methods used for carrying out this research study; In Section 4, the results of the analysis performed on the advertising jingles of the candidates for mayor of Cartagena de Indias using the MUSEMAN tool, are presented; and finally, Section 5 reports the conclusions and future work derived from this research.

II. RELATED WORK
In previous work, the following proposals are found. In 2021, Sánchez-Barragán et al. [17] propose a music content recommendation system, which has as input the affective profile of the user determined from the sentiment analysis of the comments made through the social network twitter. In 2020, Russo et al. [16] presented a system based on images of cochleograms for the detection of affective musical content, first using Russell's model in combination with a convolutional neural network to extract the relevant musical characteristics. In 2019, Chmulik et al. [12], evaluated different selected acoustic characteristics using Russell's model (2-D emotional model). For comparison, they applied two-dimensional cepstrum (TDC), MIRToolbox (Matlab), Linear Prediction (LP) analysis, on a support vector regression (SVR) model. Also in 2019, Panwar et al. [18], studied the impact of emotions transmitted by music broadcast by radio channels, by mapping a function to transform music from radio channels into emotions. For the above, Thayer's plane of arousal and valence was used [19] and a linear regression model was applied to predict the values of music on the emotional plane. In addition, the mapping of emotions was performed at different times in various cities in the United States, obtaining a window for marketing with music and thus directing the different advertising messages in specific slots. In 2019, García et al. [20] , an emotion analysis study based on data mining was carried out on the tweets generated in Mexico related to the 2017 earthquake. In addition to detection of the primary emotions, the study sought to identify the evolution of emotions over time and the patterns of propagation by comparing two datasets, one containing tweets with the word -earthquake‖ and the other that included a mention of the emergency mechanisms. The study was able to conclude that the sentiments most widely propagated independent of the dataset were anger and joy. In 2019, Chanchí and Cordoba [2], a study of emotions and sentiments was carried out on the speech of the signing of the peace agreement. The analysis was performed by extracting the emotions during the audio track, making use of the arousal and valence variables in each subsegment of the track. Sentiment analysis was performed by studying the text of the speech through the service provided on the ParallelDots platform. This study was able to determine concordance between the motionality of the audio track and the polarity of the text of the speech. In 2018, Mäntylä et al. [21], a review of the literature was carried out on 6996 Scopus papers on the subject of sentiment analysis, making use of data mining techniques. The study concluded that work carried out at the beginning of the 21st century was focused on studies of public opinion. However, the theme of sentiment analysis acquired a greater diffusion from 2004 with the evolution of the web and the content generated and published by users in this. According to the results of the study, sentiment analysis has currently been dispersed to different topics such as the analysis of opinions of products or services, analysis in social networks, reactions to disasters, voting intentions in elections, medicine, and engineering software. In 2013, Mosquera-Cabrera [22], an exploration of music as a phenomenon of psychological interest and its participation in the development of emotional experiences is carried out. Through the exploration of different sources, the authors make a description of works that integrate neuroscience, the study of higher psychological functions and musical stimuli to elucidate the mechanisms that allow defining the role of music in the manifestation of positive emotions. The work concludes that thanks to the benefits of music, it becomes a tool to support the human being in different settings, in which it seeks to positively influence personal, social, and intellectual development. In this sense, among the main benefits of music in human beings, the authors found: facilitating the expression of emotions, energizing the body and mind, relieving fears and anxieties, psychophysical relaxation, and decreased perception of pain.
From the review of previous works, it is possible to identify that at the level of affective computing, sentiment analysis techniques have been widely spread in the study of opinions made in contents belonging to social networks in different application contexts, being a challenge the application of affective computing in the analysis of multimedia advertising content. In this sense, within this article it was proposed as a contribution the application of affective computing techniques in the analysis of emotions on advertising multimedia content in the political context, taking advantage of the advantages provided by the acoustic variables of arousal and valence in the determination of emotions from Russell's two-dimensional model. Similarly, considering that the existence of automated tools for the analysis of emotions in multimedia content has not been evidenced, this article proposes an automated tool called MUSEMAN, which allows the segmentation and analysis of the audio tracks of advertising jingles to obtain the emotions transmitted in each multimedia content.

III. MATERIALS AND METHODS
This section presents the jingles considered for the present study, as well as the conceptualization about the musical recognition of emotions and the methodology used for the conduction of the developed study.

A. Jingles Considered
The present study was carried out considering the jingles published on social networks of the candidates who obtained the 6 highest votes in the elections of October 27, 2019; with William Dau, the winner, followed by William García, Yolanda Wong, Sergio Londoño, Fernando Araujo and Jesús Hernández. The jingles were obtained from YouTube and Facebook and transformed into audio and video formats using online encoding services. Table I shows the description of the (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 13, No. 4, 2022 556 | P a g e www.ijacsa.thesai.org videos from which the audio tracks considered for the present study were taken.

B. Music Emotion Recognition
Music emotions recognition (MER) uses several approaches: 1) The categorical one predicts an emotion using a classification with labels; 2) The dimensional approach establishes the emotions of a song, representing them on a dimensional plane with numerical values in X and Y [11], [19]; and 3) A dynamic process, to predict the emotion of each short segment of a song, thus obtaining the variation of musical emotion [23]. The steps followed to conduct a recognition of musical emotions may vary, depending on the classification process to be followed, but in general terms it can be summarized as follows [24]: 1) Speech acquisition the data is acquired and makes sure to have the corpus to use for the training datasets. 2) Noise removal -improving the quality of the data to be processed is vital for proper data processing, for this you can use filtering algorithms, spectral restoration and speech-structure based. 3) End point detection -the separation of non-speech sounds: recognition depends also on this separation being carried out; 4) Windowing -voice segments (word) are limited in a window, to help in the extraction of characteristics, for which the Mel frequency cepstral coefficient (MFCC) method is used; 5) Feature Extraction -the relevant characteristics for the prediction model to be built are selected, methods such as Hidden Markov Models and Vector Support Machines are used.
In the present work, the MUSEMAN tool was developed in the Java language, which allows the segmentation and extraction of the acoustic properties of arousal and valence associated with the segments of an audio track. These acoustic properties allow the classification of a certain audio segment within the circumflex model (Russell model) in such a way that the tool obtains the fluctuation of emotions throughout an audio track. Specifically, for the classification of emotions, the trigonometric angle θ formed with respect to the x-axis is obtained according to Equation 1 [14]. The angle θ obtained determines the quadrant and the emotion to which the analyzed audio segment belongs. (1) The extraction of acoustic characteristics is carried out in the background by using the openEAR library [25], which comprises a set of tools with functionalities for the recognition of emotions, such as audio recording, feature extraction, classification, result evaluation and previously trained models. It is free software available under the GNU General Public License.

C. Experimental Methodology
To carry out this work, Iterative Research Pattern -IRP [26] were used, which has four phases as presented in Fig. 1. Next, Table II shows the decomposition of each phase into activities.
The different activities of the methodology are presented in Table II.

IV. RESULTS
This section presents the software architecture of the MUSEMAN tool and the results of the emotion analysis on the selected advertising jingles using the proposed tool.

A. MUSEMAN Software Tool
This section describes the software architecture of the tool for the classification of emotions from analysis of the acoustic properties of arousal and valence, adapted and improved from [27]. In addition, the definitive interfaces of the MUSEMAN tool are presented. Fig. 2 outlines the functional architecture of the MUSEMAN tool, which was developed in the Java language and has three functional layers: View, Analysis and Storage.
In the view layer is the graphical interface of the tool, where the audio file to be analyzed is loaded, which can be in .mp3 or .wav format. Once the file has been loaded, the emotionality analysis begins from the analysis layer, in which the file is segmented into 5-second fragments with the help of the FFmpeg library, in such a way that temporary files are generated audio on the storage layer. As the audio fragments are generated, the acoustic properties of arousal and valence are obtained, making use of the free openEAR library. With the acoustic properties of arousal and valence obtained, it is possible to classify each fragment within the circumflex model or Russell model, from obtaining the trigonometric angle formed by these two variables. In this work the model of emotions proposed in [14] was used, which has five emotions: happy, excited, angry, sad and relaxed, each of which have a coverage range of 72. As emotions are calculated, they are presented in the view layer and kept in the storage layer in a vector (time stamp and emotion), in such a way that the vector will have as many emotions as fragments are obtained from the loaded audio track. When the analysis process on the audio track has finished, it is possible to generate from the view layer a report with the results of the analysis in a csv file, consulting the emotions and the time stamps in the emotions vector. Additionally, when the report is generated, two types of graphs are also obtained: one that shows the fluctuation of emotions and their distribution along the track analyzed. These graphs were generated with the help of the Java JFreeChart library. Fig. 3 presents the graphical interface of the MUSEMAN tool, which was developed in the Java language (using the swing library components) and has three tabs: -Emotions Analysis‖, -Emotions Trace‖ and -Emotions Distribution‖.  (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 13, No. 4, 2022 558 | P a g e www.ijacsa.thesai.org From Fig. 3, in the -Emotions Analysis‖ tab, the audio file is loaded using the -Load audio‖ button, then when pressing the -Process‖ button, the segmentation of the audio track starts (FFmpeg library), to obtain the variables arousal and valence (openEAR library), and calculate the emotion associated with each fragment. In the text area on the left, in Fig. 3, the value of the variables arousal and valence is displayed in real time, as well as the emotion of each 5-second segment throughout the audio track. When the analysis process is complete, the percentages for each emotion and the dominant emotion on the audio track are displayed in the text area on the right. Also in Fig. 3, it can be seen that, for the case of the loaded audio file, it is obtained that the ‖excited‖ emotion is the one with the highest percentage in the audio track with a value of 50%. In Fig. 3, the -Generate Report‖ button can be seen, with which a .csv file is obtained with the time stamp of each segment and the emotion corresponding to them. Once the report has been generated, in the tab ‖Trace Emotions‖ a graph is obtained that shows the fluctuation of emotions along the audio track, as can be seen in Fig. 4. This graph is made by using from the JFreeChart library.
In Fig. 4, the fluctuation of emotions in the course of the loaded audio track in Fig. 3 is shown, where it can be seen that the track fluctuates mostly between excited (2) and angry (3) emotions, with a shift towards happy emotion (1).
In Fig. 5 the -Emotions Distribution‖ tab of the MUSEMAN tool is shown, where a radial graph shows the distribution of emotions in the audio track, that is, the number of times an emotion is present in the audio. Thus, for example, it is observed that for the track loaded in Fig. 3, there are six instances corresponding to the excited emotion, five instances for the angry emotion and one instance associated with the happy emotion.
Finally, it is appropriate to specify that the tool requires the use of the FFmpeg and openEAR libraries in the background. It was therefore necessary to deploy the tool on the Linux operating system, in its Lubuntu distribution.

B. Emotions Analysis Carried Out on Advertising Jingles
From the previously constructed tool presented, the analysis of the emotions on the audio associated with the advertising jingles of the candidates for mayor of Cartagena de Indias for the period 2020-2023 was carried out. From the audios associated with the videos presented in Table I and using the MUSEMAN tool, the graph of the fluctuation of emotions of the jingles of the candidates was obtained. In Fig. 6, it can be seen that the horizontal axis (x) shows the duration in seconds of the track, while the vertical axis (y) shows the emotions identified at a certain moment of the track: 1 happy, 2 excited, 3 angry, 4 sad, and 5 relaxed.
In Fig. 6, it can be seen that the horizontal axis (x) shows the duration in seconds of the track, while the vertical axis (y) shows the emotions identified at a certain moment of the track: 1 happy, 2 excited, 3 angry, 4 sad, and 5 relaxed. In Fig. 6 it can be seen that the jingles associated with:  Candidates Araujo, García and Wong remain or broadcast their entire duration in an excited state (2).
 Candidate Dau spends half the time in the excited state (2) and alternates the rest of the time between happy (1), sad (4) and relaxed (5) emotions.
 Candidate Londoño is associated with the angry emotion (3) most of the time with two fluctuations towards the excited (2) and sad (4) states.
 Candidate Hernández remains most of the time in the sad state (4), with a fluctuation towards the relaxed state (5). www.ijacsa.thesai.org The above results can be seen more clearly in Table III, which shows the way in which emotions are distributed as a percentage in each audio track.
Additionally, in Table III, it can be seen that the track of:  Candidate Dau's dominant emotion is the excited state (50%) with fluctuations towards happy, relaxed and sad states, taking into account that in a large part of the track there is an emphasis on some acts of corruption in the city, ending with an invitation to vote for the candidate.  For candidate Londoño, the dominant emotion is that of the angry state (75%) with fluctuations towards the excited and sad states. The above highlights that for most of the track the candidate denounces the acts of corruption of past administrations, ending with an invitation to vote for his proposals.
 Candidate Araujo has the excited state (100%) as the dominant emotion, explaining the fact that the candidate throughout the jingle is accompanied by people talking in a euphoric way about their proposals for change.
 Candidate Hernández's dominant emotion is the sad state (87.5%) with a fluctuation towards the relaxed state, which is explained by the fact that for most of the track the candidate reflects on the situation of the city with a slow tone of voice and ending with the invitation to vote for his proposals.

V. DISCUSSION OF THE RESULTS
In this article we presented as a contribution the application of affective computing techniques in the emotional analysis of advertising jingles in the political context, taking advantage of the benefits of multimedia content and specifically audio and music to communicate emotions. In this sense, the acoustic properties of arousal and valence were used to determine the emotion associated with an audio fragment belonging to a jingle in Russell's two-dimensional model [11], [15]. Thus, the approach proposed in this work provides an objective contribution to the creators of advertising content for the design of the emotions to be communicated in an advertising jingle, enriching the traditional methods of content creation.
For the development of the proposal presented in this article, an automated tool was built for the analysis of emotions in advertising jingles called MUSEMAN, which allows the fragmentation of the audio of a jingle in segments of 5 seconds, to each of which the variable arousal and valence is determined, so that the tool allows to obtain statistically the percentage distribution of emotions in the jingle and graphically the variation of emotions over time. It is worth mentioning that in order to obtain the acoustic properties of each of the audio fragments; the proposed tool makes use of the openEar library in the background. Thus, with respect to other state-of-the-art proposals, including the openEar library itself, the MUSEMAN tool automates the process of analysis of advertising jingles, having as an added value the segmentation processes, as well as the statistical and graphic analysis of the emotions present in the analyzed jingle, which allows supporting content creators in the design of the emotions to be communicated in an advertising jingle.
One of the limitations of this study is that it focuses on the emotional analysis of advertising jingles in the political context from the analysis of acoustic variables, so that this study can be complemented in the future by analyzing the text www.ijacsa.thesai.org corresponding to the message expressed in the audio, for example by using sentiment analysis techniques, in order to determine the level of polarity of the message. Thus, a possible challenge in the analysis of emotions in advertising jingles could be the use of fuzzy logic based systems that receive as input the polarity values of the text and the emotionality of the audio to determine as output an enriched emotionality level for the analyzed jingle.

VI. CONCLUSION
In this work, an analysis of emotions was carried out on the advertising jingles of the candidates for mayor of Cartagena de Indias for the period 2020-2023. The study was carried out from the analysis of the audios associated with each of the jingles, by determining the musical properties of arousal and valence at different moments of the audio track, in such a way that from these properties it was possible to establish an emotion in the plane of the circumflex, or Russell, model. This study intends to serve as a reference regarding the application of affective computing in the design of the messages and emotions transmitted in advertising jingles.
The approach proposed in this work represents a contribution to the state of the art regarding the application of affective computing in the analysis of commercial advertising jingles and in the political context, taking advantage of the benefits provided by the acoustic properties of arousal and valence, in terms of determining the emotions of an audio fragment according to Russell's model. In this sense, the study developed in this article was supported by a tool called MUSEMAN, which allows the automation of the process of analysis of emotions in acoustic multimedia content, through the segmentation of the advertising jingle analyzed in a set of fragments to each of which the associated emotion is determined from the acoustic properties of arousal and valence, in order to obtain the trace of emotions that make up the jingle. Thus, the tools and libraries considered in the proposed tool can serve as a reference for the extrapolation of this research in the analysis of multimedia content in other application areas.
The analysis of emotions from the study of musical properties such as arousal and valence corresponds to a nonsubjective technique, since it depends on acoustic characteristics such as time, intensity, volume, rhythm, etc. This makes this method useful both for studying the design of the messages transmitted in advertising jingles and for analyzing the perception or level of user satisfaction.
The MUSEMAN tool proved to be adequate in terms of the analysis of the emotions present in an audio track, since it allows to carry out the segmentation processes, obtaining the arousal and valence properties, calculating the emotions associated with each segment, generating reports and display of follow-up and distribution graphs. In this sense, the MUSEMAN tool takes advantage of the advantages provided by the FFmpeg and openEAR libraries, acting in the background on the Linux operating system.
The study carried out showed that the candidates used different communication strategies in their advertising messages. On the one hand, three candidates transmitted the emotion of euphoria in their messages through the use of adapted songs (García and Wong) and through the socialization of the proposals in the streets of Cartagena (Araujo). On the other hand, the three remaining candidates recounted stories with a reflection on the problems of corruption and the problem of inequality in the city, relying on emotions such as anger and sadness.
As a future work derived from the present research, we intend to complement and contrast the results obtained by applying sentiment and text analysis techniques to the messages included in the advertising jingles.