Currently, social media marketing strategies seek to position brands within the hearts of their customers, where the main experience of value is emotion [1
] (Smith and Bolton, 2002; Mauri, et al., 2011). This is why the involvement of the senses is fundamental to influencing the emotional state of social media users [3
] (Prescott, 2017). One of the most widely used resources to achieve this emotional impact on users is GIFs. Their effectiveness has been analyzed in several studies that have shown their ability to generate emotionality [4
] (Bourlai and Herring, 2014; Bakhshi et al., 2016; Gygli and Soleymani, 2016).
In fact, nowadays, communication and marketing professionals are looking for tools that allow them to measure the effectiveness of their campaigns, in terms of emotionality. The most frequently used research techniques are based on content analyses of comments made by a brand’s followers on a social network [7
] (Driscoll, 2015; Turnbull and Jenkins, 2016; Scheinbaum, 2017; Kim and Kim, 2018). Less common is the use of neuroscience techniques to measure emotionality based on users’ unconscious responses to a given stimulus.
Consequently, the present study proposes a combination of emotion measures (emotional valence, basic emotions and engagement) to assess the effectiveness of GIFs as generators of emotional experiences on social networks. In addition, neuroscience techniques were used to observe physiological and cognitive responses (implicit measures), as well as to perform sentiment analyses (explicit measures) of Instagram comments.
3. Research Questions and Hypotheses
The following are the research questions and hypotheses derived from previous literature.
Are there differences between implicit and explicit measures of emotional valence in Instagram users?
The measure of explicit valence of Instagram users’ response to GIFs will be more positive than the measure of implicit valence.
What might account for these possible differences?
The greater the user engagement with GIFs, the smaller the difference between implicit and explicit measures of emotional valence.
The longer the comment, the smaller the difference between implicit and explicit measures of emotional valence.
The greater the proportion of emojis in the comments on GIFs, the greater the difference between implicit and explicit measures of emotional valence.
Are explicit comments and biometric tools equally effective in identifying the emotions felt by subjects?
Biometric tools are more accurate predictors for assessing the basic type of emotion that is aroused in response to GIFs.
In order to test these hypotheses, this research was divided into two phases focused on inferring the emotional valence derived from a selection of GIFs posted on Instagram (Figure 1
). In the first phase, techniques from consumer neuroscience or neuromarketing were applied to obtain a measure of implicit emotional valence and engagement. The second phase complemented the first by analyzing the explicit emotional valence based on the semantic analysis of the comments on each GIF. Finally, a comparison was made between the emotional data obtained implicitly and explicitly.
4. Materials, Methods and Results
4.1. Phase 1. Experimental Study of Neuromarketing Applied to GIFs
In this first phase, neuromarketing devices (face coder, GSR and eye tracker) are applied to analyze and quantify the emotional valence, the engagement generated and the type of basic emotion caused by 18 Instagram GIFs selected for being used by renowned brands and for having a high number of likes. The selected GIFs are images in movement with an approximate duration of 4 s. All the images are high-quality, some taken outdoors and others indoors. Concerning audiovisual treatment, some contain a filmed scene and others constantly repeat a moving image. The visual contents are varied: people, objects and animals, both in the foreground and in the background.
The valence variable allows us to identify the sign of the emotion (positive or negative). The engagement variable indicates the emotional state the person is in when viewing the stimulus, and is extracted through the combination of emotional valence and activation. The type of basic emotion recorded indicates which specific emotions have been experienced through the recording of facial expressions (happiness, surprise, anger, disgust, fear and sadness).
The three variables provide the measure for each subject and each GIF, and these individual data are added to the aggregate values for the group, thus obtaining a quantitative value for each variable for each GIF.
Participants were selected randomly, with the inclusion criterion being regular use of at least one social network.
The sample size was 30 participants. This sample size was considered representative with a probability of error of less than 1% [41
] (Cohen, 1992; Sands, 2009; Hensel et al., 2017). The size is adequate to provide sufficient knowledge about the stimulus for the purpose of the research. Furthermore, the distribution of the sample was based on the Interactive Advertising Bureau and the Elogia study (2018) [44
], which described the profiles of social network users. Therefore, the composition was 33% for each age range (16–30; 31–45; 46–55), with 53% women and 47% men.
The stimulus used in this experimental phase was a video of 2 min and 26 s, made from a random combination of 18 GIFs, incorporating distractors between each of them. In order to avoid presentation bias, three different videos were edited with the elements randomly ordered.
The selection of the GIFs was based on a review of all GIFs posted on Instagram (1 March–10 April 2019) by the top 100 international brands (2017 Best Global Brands ranking—Interbrand, 2017 [45
]). The 18 GIFs with the highest number of likes were selected.
All subjects viewed one of the three versions of the video. During the presentation of the stimulus, facial micro-expressions, skin conductance level and pupil direction were recorded.
The GIFs were presented via a laptop (Windows 10 operating system) and a 24-inch monitor. In addition, an external webcam was used to record facial expressions and pupil direction.
4.1.4. Measuring Tools
Face coder. This records the data from the decoding of the person’s face through software that analyzes the image provided by the webcam. The software developed by the company INTERACTÚA+ was used.
The Face Coder tool provided six concrete emotions (joy, surprise, sadness, fear, anger and disgust), with their respective value, recorded from facial micro-expressions that were generated in the presence of each GIF. Subsequently, the values of these emotions registered in all participants were aggregated, obtaining a single value for each emotion per GIF.
Previous research in communication and marketing stimuli using this tool for the analysis of emotion type has been considered as the empirical background (McDuff, El Kaliouby and Picard, 2012; Bellman, Wooley and Varan, 2016; Goyal and Singh, 2018; Mundel et al., 2018) [46
Galvanic Skin Response (GSR). This records skin conductance levels through electrodes placed on the fingers of the subject’s hand. The eSense® device was used.
Previous studies involving GSR to analyze unconscious responses to marketing stimuli were taken into account in the design (Weibel et al., 2019; Walla, Koller, Brenner and Bosshard, 2017; Guerreiro, Rita and Trigueiros, 2015; Reimann, Castano, Zaichkowsky and Bechara; 2012) [50
Eye-tracker. This has been included in the experimental study to ensure that the participant is viewing the stimulus. It consists of a device that tracks the eye and records the subject’s gaze while viewing a stimulus.
4.1.5. Data Analysis
All recorded data were first processed in Excel in order to clean them and make them manageable in the SPSS statistical software.
Specifically, the data recorded by the facial emotion identification tool (face coder) determine the valence of the emotions (positive or negative) and the type of emotion according to the classification of the six basic universal emotions (Ekman, 1993) [14
]. The recording of the level of skin conductance (GSR) during the visualization of the stimulus was used to obtain the level of arousal. From these two records, the engagement variable was calculated, which indicates the emotional state in which the participants found themselves while they were watching the video, as it combines the type of emotion with the intensity of this emotion. In addition, as a control measure, the position of the pupil (eye tracking) was monitored, which made it possible to verify the stimulus that was really generating the emotion.
Emotional valence and engagement were tested for normality and homogeneity of variances. The basic emotion type was quantified to get an approximation of how much each emotion was experienced upon exposure to each GIF.
4.1.6. Ethical Issues
The study was approved on 20 April 2018 by the Ethics Committee of the Universidad Internacional de la Rioja, Spain, under the number LABNMKT-001-2018.
The ethical protocols of the World Medical Association Declaration of Helsinki (1964) were followed. In addition, informed consent was obtained from all participants, assuring data confidentiality.
After signing the informed consent form, each participant was placed in front of the computer. Both the face coder and the GSR were checked to ensure that they were calibrated and working correctly, and the baseline measurement was taken. Then, one of the three videos with the 18 stimuli was randomly played. Once the video had finished, they were thanked for their participation and accompanied to the exit. Each test lasted about 6 min in total.
In order to obtain the baseline of the GSR tool, sensors were placed on the non-dominant hand of the participants in such a way that it began to collect values prior to the presentation of the stimulus, thus establishing the skin conductance baseline of each participant. At the same time, the computer equipment and the tool software were synchronized. On the other hand, the eye tracker and Face Coder tools shared the same software and implementation. The first step was to regulate the webcam to achieve optimal lighting conditions and get the face of the participant centered on the screen. Then, the calibration software allowed us to obtain the baseline. This involved following a point that moved across the screen and picking up the point where the pupil is located.
The Shapiro–Wilk normality test (see Table 1
) shows that the variables implicit emotional valence (VIE
) and engagement (Eg) are close to normal (VIE
= 0.779 > 0.050; Eg: p
= 0.772 > 0.050). Furthermore, their variances are homogeneous, according to the results obtained in Levene’s test of equality of variances (VIE: p
= 0.873 > 0.050; Eg: p
= 0.818 > 0.050). The measures of the variables analyzed are shown in Table 1
The 18 GIFs selected reflect the six basic emotions. The emotion that appears most frequently is sadness, followed by anger, as shown in Table 2
4.2. Phase 2. Sentiment Analysis of GIF Content on Instagram
In this phase, a quantitative methodology was implemented and a content analysis was carried out, using data mining and sentiment analysis. To do this, the comments on the Instagram GIFs selected in Phase 1 were obtained.
The explicit emotional valence of the comments was quantified (negative or positive), both the one derived from the text and the one derived from the emojis in the comments (negative and positive), as well as the length of the comment itself (including both text and emojis) and the proportion of emojis in the comments.
A total of 1420 comments were analyzed. These came from each of the 18 GIFs used in Phase 1. Up to 100 comments were extracted from each GIF (15 April–1 May 2021). If they had less than 100 comments, all existing comments were extracted. Similarly, if they had more than 100 comments, only the first 100 were used.
4.2.2. Measuring Tools
Sentiment analysis of comments. To obtain information on the length and composition of the comments, a program written in Python 3 was used. This program takes as input an excel document, generated through the Export Comments application, with the comments on each GIF (the openpyxl library was used). The comments were cleaned up using a function that removes strange punctuation symbols and tags that correspond to other users. The Emoji library was used to identify the emojis present in the comments. Then, a function was created to obtain the number of words, the average lengths of the comments, and the percentage of words and emojis in the comment. With all the information obtained, a new Excel document was generated (using openpyxl) that includes the following statistics: average number of words in the comments, average number of emojis and percentage of emojis versus words for each GIF.
The Twinword API (https://www.twinword.com/
; accessed on 31 May 2021) was used. This natural language processing (NLP) interface, based on the Python programming language, detects the intentionality of sentences and paragraphs. The measure it provides is the overall score of the analyzed text. Thus, values lower than −0.05 are considered negative, values higher than 0.05 are considered positive, and all values in between are considered neutral. In addition, thanks to this application, a list of the most emotionally charged words and adjectives was obtained from the comments.
Detection of the basic emotions in the comments. Through Twinword’s Emotion Analysis API, the basic emotions in each of the 18 GIFs were identified.
Analysis of the emotionality of emojis
. The emotional valence of emojis was obtained using the Emoji Sentiment Ranking (http://kt.ijs.si/data/Emoji_sentiment_ranking/
; accessed on 31 May 2021), developed and validated by Kralj et al. (2015) [29
Content analysis. We developed our own counting application to obtain the maximum length of the comments, as well as their composition (proportion of text and emojis).
4.2.3. Procedure and Data Analysis
Compilation of comments on each GIF
Using the URLs of the Instagram GIFs, up to 100 comments were collected for each of the 18 GIFs under study. Using Export Comments (https://exportcomments.com/
; accessed on 31 May 2021), an Excel file was obtained for each GIF. If it had more than 100 comments, the tool extracted 100; if this number was lower, all comments were extracted.
Subsequently, all comments were translated into English as the sentiment analysis tool only evaluates content in this language. This was done via Google Translator (https://translate.google.com/?hl=fr&tab=TT
; accessed on 31 May 2021).
Analysis of the explicit emotionality of each GIF and calculation of the variable VEE
The analysis of the emotionality of the textual content was carried out with Twinword. An emotion value was obtained for each comment in the GIF, calculated by averaging the total emotion value of each GIF, which was in the range −1 to 1.
The emotional valence of the emojis was then obtained by means of the Emoji Sentiment Ranking.
In order to obtain one single standardized value for the explicit valence measure, all text and emoji values were taken together and divided by the number of comments. This resulted in the variable Explicit Emotional Valence (VEE).
Analysis of the Composition of Comments and Calculation of Variables LgC and Pemj
In order to count the length (LgC) and the proportion of emojis in the total content of the comments (Pemj), we used our own counting application, designed in Python language. Thus, we obtained the average value of the length of the comments in each GIF and the percentage of emojis used in their comments.
Analysis of the Differences between Explicit and Implicit Emotionality and Calculation of the Variable VD (Table 3
The variable Emotional Valence Difference (EVD) was obtained by the difference between the values recorded explicitly (comments) and implicitly (biometric tools):
Implicit Measure of Valence Versus Explicit Measure of Valence
The first step was to calculate normality for the VIE
through the Shapiro–Wilk test (Table 4
). The distributions were found to approach normality for both VIE
> 0.050). In addition, the assumptions of homogeneity of variance were met using Levene’s test for equality of variances (p
= 0.873 > 0.05).
Since the normality and homoscedasticity criteria were met, a t-test for independent samples was performed to compare the average of implicit valence and explicit valence. The results, as shown in Table 5
, reveal statistically significant differences between VIE
= 0.148, SD = 0.036) and VEE
= 0.407, SD = 0.030; t
(34) = −7.178, p
= 0.000). Therefore, emotional valence was significantly higher when users expressed themselves through comments on Instagram than when the emotional valence aroused by GIFs was recorded through biometric tools. Consequently, H1 is confirmed. Moreover, in both cases the average emotional valence was positive.
Differences between Implicit and Explicit Measures of Emotional Valence for Each GIF
A new continuous variable is defined: the difference of the emotional valence between the implicit and the declared measures in each GIF (VD). This value was calculated for each GIF. Additionally, in order to obtain which variables could explain this difference between the two measures of valence, different correlations between different variables were studied through Pearson’s bivariate correlation coefficient. Specifically, these variables were engagement (Eg), comment length (LgC) and proportion of emojis in the comments of the GIFs (Pemj).
Relation between Valence Difference (VD) and Engagement (Eg)
In the relation between VD and Eg, the correlation showed a statistically significant inverse relation with a negative sign, as shown in Table 6
= −0.546, p
> 0.050). This indicates that the more engagement a GIF generates, the smaller the difference in emotional valence measured explicitly (through comments) and implicitly (biometric tools), thus confirming H2.
Relation between Valence Difference (VD) and Comment Length (LgC)
A possible significant and positive relation between VD and LgC has been hypothesized. However, the results indicate that there was no statistically significant relation between VD and LgC (r = −0.380, p = 0.120), rejecting H3. Therefore, the greater difference found between the psychophysiological and the stated measure of valence was not due to the length of the comment.
Relation between the Valence Difference (VD) and the Proportion of Emojis (Pemj)
With respect to the relationship between VD and Pemj
, as hypothesized, a significant and positive correlation was found (r
= 0.631, p
> 0.010), as shown in Table 7
. This indicates that the greater the proportion of emojis used in comments, the greater the difference that will be found between the psychophysiological and the stated measure of valence. Therefore, H4 was accepted.
Comparison between the Emotions Generated by the GIFs According to Whether They Were Measured Implicitly or Explicitly.
In order to find out which type of measurement is more effective as a predictor, understanding effectiveness in this case in terms of precision, the specific emotions that gave rise to both implicit and explicit emotional valence were obtained (see Table 8
As a result, while the biometric tools detected the six basic emotions (happiness, surprise, anger, disgust, fear and sadness) in the 18 selected GIFs, when analyzing the comments, it was not possible to detect all the emotions. Only the emotion happiness appeared in all 18 GIFs, while more negative emotions such as anger or disgust were not detected in any of them. In fact, most of the words used in the comments alluded to positivity (see Figure 2
). Given that the biometric tools were more accurate in detecting the six emotions, we accept H5.
The present study reflects the emotional impact that can be achieved through the use of GIFs in social networks [4
], (Bourlai and Herring, 2014; Gürsimsek, 2016; Miltner and Highfield, 2017), specifically Instagram. Moreover, the use of different implicit and explicit techniques in the two phases of the study highlights the need to use biometric and self-reported tools in a complementary manner [31
] (Hernández-Fernández, Mora and Hernández, 2019; Lajante et al., 2020) in order to eliminate biases—biases that undoubtedly occur if only the analysis of the comments declared by the subjects is taken into account.
By comparing implicit and explicit emotional valence, we find that social network users tend to express comments with a significantly higher (more positive) emotional charge than what they actually feel (before processing the stimulus rationally). Biometric tools can therefore help to provide a more accurate analysis.
This finding raises the question of what might account for the differences between the two valence measurements. Thus, the possible influences of engagement, the length of the comments, or the higher proportion of GIFs on the difference in valence are investigated.
As for the relation between the differences in valence and engagement, a significant inverse relation is evident. This indicates that the greater the engagement with the GIF, the smaller the difference between the valences. This relation can probably be explained by understanding engagement as commitment to the brand. This commitment could mean that users are less vulnerable to external influences, so that the emotions expressed would be more in line with what the subjects really feel towards the brand. In contrast, with a lower level of engagement, the emotions declared are under greater external influence: conditioning due to being a quality brand [39
] (Rúa-Hidalgo et al., 2021), a sense of belonging, prestige, etc. In short, well-known brands need to surprise in order to activate users; when this does not happen, less engagement is generated (Rúa et al., 2021) [39
]. However, it remains possible that the comments on social networks continue to be positive, moving even further away from the emotion that is really felt.
On the other hand, the length of the comment is not found to influence the differences between implicit and explicit measures of valence. This suggests that the emotionality of the words contained in the comment matters more than the length of the comment itself.
The proportion of emojis has not been found to be a reliable predictor when assessing the emotional level of the social network user. A higher proportion of emojis in comments generates a greater distortion between implicitly and explicitly measured valence. This could be due to the fact that text provides greater possibilities for expressing emotions felt (greater semantic richness), while with emojis the range of expression is smaller, and more limited in its emotional gradation.
Finally, in the comparison of the type of emotion recorded by biometric tools and in the declared comments, a higher number of emotions were detected when recording data with neuromarketing devices.
We suggest that, when there is public exposure, through comments on social networks users express more positive emotions (happiness and surprise), avoiding more negative ones such as anger or disgust [17
] (Reinecke and Trepte, 2014; Waterloo, Baumgartner, Peter and Valkenburg, 2018). It is possible that the external influences to which we as social beings are subjected (need to belong, social judgement, status, etc.) are the cause. However, these influences disappear when the implicit emotion is recorded through neuromarketing tools, making it possible to identify a greater and more precise variety of emotions.
All of the above indicates that using the comments received on social networks as the only indicator of the level of success of a communication action can lead to a false sense of success for well-known brands.
It also shows that the implicit measurement of emotion is more effective, both in quantifying the emotional level of subjects and in identifying the basic type of emotion that subjects experience, thus achieving greater richness and precision in the analysis.
7. Limitations and Future Lines of Research
A limitation of the research lies in the use of Google Translator to transcribe sentiment analysis comments into English. This basic translation tool can cause the context to be lost in translation; however, less than 11% of the comments needed to be translated, and in most cases, they were short, easily translated comments.
In addition, as the GIFs were always from well-known brands, the level of emotionality in the comments may be different from if they were used in the context of non-known brands.
It would be interesting to carry out future research to study whether the use of other types of stimuli in social networks (stories, photos or videos) can achieve similar results to those achieved with GIFs.