Disinformation in Social Networks and Bots: Simulated Scenarios of Its Spread from System Dynamics

Abstract

Social networks have become the scenario with the greatest potential for the circulation of disinformation, hence there is a growing interest in understanding how this type of information is spread, especially in relation to the mechanisms used by disinformation agents such as bots and trolls, among others. In this scenario, the potential of bots to facilitate the spread of disinformation is recognised, however, the analysis of how they do this is still in its initial stages. Taking into consideration what was previously stated, this paper aimed to model and simulate scenarios of disinformation propagation in social networks caused by bots based on the dynamics of this mechanism documented in the literature. For achieving the purpose, System dynamics was used as the main modelling technique. The results present a mathematical model, as far as disinformation by this mechanism is concerned, and the simulations carried out against the increase in the rate of activation and deactivation of bots. Thus, the preponderant role of social networks in controlling disinformation through this mechanism, and the potential of bots to affect citizens, is recognised.

1. Introduction

The academic community has shown widespread interest in understanding how disinformation spreads in virtual media, including social networks, e.g., [1,2,3,4,5,6,7], due to the potential of disinformation to trigger various problems for governments, citizens, and other social actors [2]. Thus, the state approach has attributed multiple consequences to disinformation on social networks, such as: the polarisation of citizens’ opinions [4], the destruction of the credibility of traditional media [8], the mobility of citizens to prevent the development of public policies [9], among others.

Recently, the spread of misinformation has been growing exponentially [1] as a result of the massive use of social networks. An example of this was the case of COVID-19, when the Russian media RT and Sputnik accused NATO and the United States of America of creating the virus in order to destabilise the Chinese economy, and this information was widely spread on social networks such as Facebook, Twitter and Tik Tok [3,10], or, in the case of the vaccines developed for COVID-19, where the anti-vaccine movement sought to attribute effects such as autism and possible genetic malformations to their use, triggering mistrust on the part of the population and preventing the control of the virus and the mitigation of its transmission [11]. In view of these examples, one of the main problems for social actors, in particular states, as well as the academic community, is the lack of awareness of the existence of this type of information and the lack of understanding of the strategies used by the disinformation agent to ensure the propagation of misinformation on social networks [12,13,14].

This article focuses on the second problem area. Thus, it is evident from the literature that disinformation can be propagated by the direct intervention of individuals either consciously or unconsciously, as well as by automated accounts known as bots [15]. These bots are present in all social networks, however, in some of them their presence is more noticeable. An example of this is Twitter, where, given the importance of this network in issues related to politics, it has been estimated that between 9% and 15% of active accounts are bots [15,16,17,18].

Due to the proliferation of bots used in social networks to misinform and the interest in this mechanism, the literature has focused on establishing the role of bots in the spread of misinformation, with a focus on describing case studies, e.g., [15,18,19,20,21,22,23,24], however, the development of models to understand the patterns of propagation of disinformation caused by this mechanism is still in its beginnings, with most of the advances being generalist and not specialised in bots such as the works of Lazer et al. [24], Shao et al. [25]; Vosoughi et al. [26] and Shao et al. [27]. The limited amount of work is due to the difficulty of finding the initial sources of disinformation [4], as well as the absence of robust and easily accessible tools for identifying bots, and thus correctly identifying their activities [15].

In this context, the aim of this article was to model and simulate scenarios of disinformation propagation in social networks caused by bots based on the dynamics of this mechanism documented in the literature. This will strengthen the understanding of the phenomenon of disinformation through bots by making it possible to establish patterns of behaviour in the system and to evaluate the effects of the various decisions made by the actors involved in disinformation, especially in relation to social network policies.

It is important to highlight that, although there is no unified meaning of disinformation due to the large number of definitions and intermediate terms found in the literature, particularly in the literature from Anglo-Saxon countries, (e.g., fake news, misinformation, etc.), this work assumes disinformation to be any deviant information that is intended to distort and mislead a target audience in a predetermined way [3,28]. In this way, disinformation refers to a wide range of content, including fake news, misinformation, misleading content, hate speech and deliberate misreporting, among others [29,30,31]. Additionally, disinformation is not only about the message itself, but as a practice it has the potential to discredit the messenger and true information due to its close relationship with multiple social sectors, especially politics.

This article is structured in five main sections. The first one broadens the conceptualisation of bots and their features; the second one establishes the behaviour of the disinformation caused by this mechanism in social networks and introduces the diagram of causal loops with their respective dynamic hypotheses; the third section shows the methodology used while the fourth section describes the results with emphasis on the mathematical model developed and the simulations defined in the methodology; finally, the fifth section presents the conclusions of the study.

2. Theoretical Framework and Dynamic Model

2.1. What Are the Bots?

The use of the word bots corresponds to an abbreviation of the Anglicism “software robot” [15], which has permeated the disinformation literature, including that of Latin American origin. Thus, the bots as machines are usually automated to some degree, either independently or with human intervention. The bots as machines are put at the service of external agents, in this case diplomacy. In this sense, this type of software can be categorised as either beneficial or malicious [32]. In the case of those that are beneficial, they are programmed by companies to improve the attention and service provided to their customers [33], or as in the case of the social network Twitter, some are used to inform the time, as documented by Yang et al. [32], or to support the raising of charitable resources [34].

For those called malicious, in some cases they oversee malware distribution or spam generation, but in social networks they have another purpose, which is to automatically produce content and interact with humans on social networks, trying to emulate them and possibly alter their behaviour [35,36,37]. According to the Academic Society for Managment and Communication [38] these bots have three strategies for influencing the behaviour of online users on social networks:

• Smoke screening: bots use context-related hashtags on social networks to distract online users from the main point of debate in a discussion.
• Misdirecting: the use of hashtags related to the topic of discussion to talk about other non-corresponding plots to divert the attention of the target audience from the disinformation.
• Astroturfing: the bot tries to influence public opinion, e.g., in a political debate, by creating the impression that a large majority is in favour of a certain position.

Although the strategies described above are the basis of the strategies used by the malicious bots, it is necessary to recognise that, along with the emergence of new technologies, these tend to evolve and new strategies emerge, which is why a general view of the behaviour of the system generated by this type of software is required, rather than the study of each of the strategies themselves.

2.2. Disinformation, Bots and Causal Loop Diagrams

Disinformation as a social phenomenon that occurs in social networks is not the result of chance, but of a strategic analysis developed by the disinforming agent, as related by Guzmán and Rodríguez-Canovas [2]. Therefore, the disinforming agent seeks to attract the largest possible target population in order to convert it into a population susceptible to being disinformed [39], which is where the various mechanisms for disinformation, such as bots, emerge [5]. With the linking of the population susceptible to disinformation, we proceed to the propagation of the message [40] to consolidate the disinformed population, which for the purposes of this study will be understood as the individual or subject who was exposed to the message of the disinforming agent.

Considering the above as a basis, the role of bots as a mechanism for spreading disinformation begins with their activation, where the disinformation agent establishes how many he or she wishes to have, however, maintaining a fixed number over time is difficult, given the detection mechanisms that social networks have to eliminate or block this type of account [16,41]. However, the interaction of these fake accounts is usually done with publications based on powerful hashtags, comments and by sharing content, being standardised in terms of the number of characters when providing a machine, as well as in the frequency and time of publications [42]. In view of the number of the target population, which will become a susceptible and post-informed population, bots are characterised by making exclusive use of the organic reach of the account, so that as the number of followers grows, this reach tends to decrease, requiring a greater number of bots to impact a greater number of the population [2]. Figure 1 shows the causal loop diagram that represents the behaviour of the studied phenomenon.

Finally, it should be noted that the spread of disinformation is like the way a disease is spread, where there is a population of infected people (bots), who seek to spread the disinformation message in a susceptible population, hence the previous models developed are based on the SIR model (Susceptible—Infected—Recovered), e.g., [43,44]. Thus, previous studies have sought to complement the SIR model by analysing specific disinformation mechanisms, as well as models that integrate several of these mechanisms, (e.g., bots, trolls, paid outreach, etc.). These models include the SIRaRu model, which allowed us to understand the behaviour of disinformation in homogeneous and heterogeneous communities [45], the SEIR model (Susceptible—Exposed—Infectious—Recovered), the SIR model for dynamic and complex social networks [46], among others. Hence, the model presented here both in Figure 1 and in the subsequent sections is based on the SIR model.

3. Methodology

The aim of this article was to model and simulate scenarios of disinformation propagation in social networks caused by bots based on the dynamics of this mechanism documented in the literature, so the main technique used for the development of the model was System Dynamics, considering Bala et al. [47] and Bianchi [48] as theoretical references. In this sense, the choice of modelling technique is based on the complexity of the disinformation caused by bots, in which various elements are involved and whose behaviour is characterised by a non-linear, multicausal and time lagged behaviour [47]. The model is based on the existing literature on the problem under study, for which the steps established by Bala et al. [47] were followed and summarised below:

• Construction of the flow diagram and levels of the model: this corresponds to the physical structure of the system in a defined period and allows the generation of the model’s patterns. In this step, the variables that allow the system’s behaviour to be represented are defined.
• The writing of differential equations: they represent the causes and effects of the system and allow its operationalisation.
• Parameter estimation: they assign the values of the variables that make it possible to simulate and obtain the system’s behaviour. For the present work, this stage was based on multiple sources, especially the study developed by Himelein-Wachowiak et al. [15].
• The testing of the internal consistency of the model: this corresponds to the evaluation of the behaviours generated by the model and which, for the present case, is based on the theory exposed in Section 2.2.

With the proposed model, we proceeded to develop the simulations presented in

Table 1. Computer simulations.

Code	Simulation	Modified Parameters
Sim—1	Reference simulation	NA
Sim—2	Increased activation rate	$TAB=0.8$
Sim—3	Increased deactivation rate	$TDB=0.1$
Sim—4	Beginning of disinformation	$RD=30$

, for which modifications were made to the parameters established in the initial model (

Table 2. Variables required for model development and initial parameters.

Variables	Conceptualización	Unidades	Parámetro Inicial
$Pob{O}_t$	Target population: a population with specific characteristics that was defined by the agent to be susceptible to disinformation	People	1,000,000
$Pob{S}_t$	Susceptible population: the population that is subscribed to one of the disinformation agent’s accounts.	People	0
$Pob{D}_t$	Disinformed population: the population that visualised or had some contact with the disinforming message.	People	0
$Bot{s}_t$	Bots: number of bots in the social network used by the disinformation agent at a given time.	Bots	5
$Bots{D}_t$	Deactivated bots: corresponds to the number of bots that were deactivated by the algorithm or social network staff.	Bots	0
$TCpobC$	Target population growth rate: is the ratio of growth of the population $PobO$ for a specified $t$.	%	0.1%
$AO$	Organic reach: percentage of publications viewed by the Bots through the distribution of the algorithm. This rate is defined according to the number of $Pob{S}_t$ contacted.	%	GRAPH (TIME) (0, 0.0017), (10001, 0.0004), (100001, 0.00015), (300000, 0.00015), (400000, 0.00015), (500000, 0.00015), (600000, 0.00015), (700000, 0.00015), (800000, 0.00015), (1000000, 0.00015), (10000001, 0.00008)
$TEC$	Effective contact rate: defined as the percentage of people who subscribe to the disinformation agent’s account.	%	0.01%
$RD$	Disinformation delay: This corresponds to the initial $t$ at which message starts its propagation. This lag was developed by using the delay function.	Days	90
$TAB$	Bot activation rate: the percentage of bots that are activated in each $t$.	%	50%
$METB$	Goal bots: number of bots that the uninformed agent wishes to have active.	Bots	300
$TDB$	Bot deactivation rate: the percentage of bots that are deactivated by the social network at a given $t$.	%	1.973%
$RTDB$	Delay deactivation of bots: This corresponds to the initial $t$ at which the deactivation of the bots is initiated. This lag was developed by using the delay function	Days	98

); it should be noted that in the execution of the simulations, only the parameter indicated in Table 1 was modified, and the other parameters retained their initial values. The description of the modified parameters is presented in Table 2.

The analyses of the simulations were carried out descriptively, and in order to determine the existence of statistically significant differences in the behaviour of the system, the medians of the PobD level were compared (see Table 2). The Kolmogorov-Smirnov statistic was applied to check whether the data fit a normal distribution (p-value > 0.05), and it was found that the data did not follow a normal distribution. Thus, to establish the difference in averages between the behaviour of the system with the initial parameters and the modified parameters, the Wilcoxon test was used, considering this difference with a p-value < 0.05.

Finally, the computational work on the model and the simulations were implemented in Stella Architect software version 1.9.5. The following model settings were considered: $t_i=0$, $t_f=360$, $\mathsf{\Delta }t=4$, where $t$ represents time in days; additionally, Euler was used as the integration method. SPSS software version 25 was used for the statistical analyses.

4. Results

The results are presented in two sections. The first one describes the model proposed with the capacity to replicate the behaviour of the system based on the evidence from the literature review; and the second one describes the simulations obtained and the corresponding statistical analyses.

4.1. Model

Figure 2 represents the flow and level diagram [1], which is based on the SIR model. Thus, it consisted of five level variables, five flow variables and 10 auxiliary variables. The green section represents the process of disinformation of the target population, and the blue section how the activation and deactivation of bots behaves.

The proposed model explains the behaviour of the phenomenon studied here, as long as the following assumptions are met:

• Between the moment of the constitution of the population susceptible to disinformation (PobS) and the beginning of the spread of disinformation there is a delay.
• There is a limited number of bots that the disinformation agent wishes to place on the social network.
• The existence of a delay for the deactivation of bots.

Under the technical conditions of non-negativity of the level variables, (i.e., their domain is restricted to 0 or positive numbers) and that $t=0,1,2\dots ,360$, the system was represented by the following differential equations.

Target population:

$$Pob{O}_t=\left[Pob{O}_{t-1}+\left(Pob{O}_{t-1}\times TCpobC\right)-\left(Pob{O}_{t-1}\times AO\times TEC\times Bots\right)\right] dt$$

(1)

Susceptible population:

$$Pob{S}_t=\left[Pob{S}_{t-1}+\left(Pob{O}_{t-1}\times AO\times TEC\right)-\left(Pob{S}_{t-1}\times f\left(X_t,X_{t-\tau },dt;t\ge t_0\right)\right)\right] dt$$

(2)

where:

$$X_t=\left[Pob{S}_{t-1}\times Bots\times AO\right] dt$$

(3)

Misinformed population:

$$Pob{D}_t=\left[Pob{D}_{t-1}+\left(Pob{S}_{t-1}\times f\left(X_t,X_{t-\tau },dt;t\ge t_0\right)\right)\right] dt$$

(4)

Bots:

$$Bot{s}_t=\left[Bot{s}_{t-1}+\left(Bot{s}_{t-1}\times TAB\times \frac{METB-Bots}{METB}\right)-\left(Bot{s}_{t-1}\times f\left(Y,Y_{t-\tau },dt;t\ge t_0\right)\right)\right] dt$$

(5)

where:

$$Y_t=\left[Bot{s}_{t-1}\times TDB\times RTDB\right] dt$$

(6)

Deactivated bots:

$$Bot{s}_t=\left[Bot{s}_{t-1}+\left(Bot{s}_{t-1}\times f\left(Y,Y_{t-\tau },dt;t\ge t_0\right)\right)\right] dt$$

(7)

Table 2 presents the description of the variables and the initial parameters for the operationalisation of the model. Because of the generalist nature of the proposed model, which is applicable to any social network, the initial parameters are susceptible to modification. In this particular case, those of Twitter were used, so parameters such as organic reach, paid reach, effective contact rate, among others, must be modified for its use in other social networks.

4.2. Simulations

Compared to the results obtained in SIM-1, it was found that under the initial parameters for $t=360$ $PobO$ increased by 230,000 people, with 159,000 being effectively uninformed, which represented 15.9% of the initial $PobO$. On the other hand, for $t=90$, period in which the disinformation of $PobS$ begins, this was close to 28,200 people, increasing until $t=111$, after this day the $PobS$ begins to decrease until it reaches the value of 0. In the case of the bots for $t=133$ the highest number of activated automata was 54.8 ≈ 55 and for $t=360$ the number of deactivated bots was 158. Figure 3 shows the behaviour of the system for SIM-1.

The results for SIM-2 show that with a higher activation rate bot, the $PobO$ for $t=360$ would be equal to 1,150,000 people, with $PobD$ being 228,000, which represented an increase of 43.39% compared to SIM-1. For the bots in this simulation for $t=132$ the highest number of activated automata would be reached with a total of 81.2 ≈ 82. Similarly, due to the increase in the activation rate for $t=360$, a total of 238 bots would have been deactivated, showing an increase of 50.63% in relation to SIM-1. On the other hand, a comparison between the $PobD$ between SIM-1 and SIM-2 showed statistically significant differences with $z=-16.42$, $p\text{-}\mathrm{value}=0.00$. Figure 4 shows the behaviour of the system for SIM-2.

For SIM-3, which sought to simulate a higher rate of bots’ deactivation by social networks, a change in system behaviour was evident (see Figure 5). Thus, the disinformation target population $PobO$ for $t=360$ would be equal to 1,260,000 people with a total of 109,000 people being disinformed, decreasing the $PobD$ by 31.44 % in relation to SIM-1. Similarly, in the case of the susceptible population, there are two peaks at $t=107$ and $t=360$ (see Figure 5a), correlated with the number of active bots (see Figure 5b). Regarding the $PobD$ level between SIM-3 and SIM-1, it was determined that there are statistically significant differences with $z=-14.00$ $p\text{-}\mathrm{value}=0.00$.

Now, in the case that disinformation starts to circulate at $t=30$, it was observed that the target population of disinformation $PobO$ for $t=360$ would be equal to 1,240,000, being similar to the behaviour derived from SIM-1. For the population that managed to be uninformed, it was determined that for $t=360$ the $PobD$ was 145,000. Similarly, in the case of bots, and due to the ability of social networks to deactivate them, after $t=170$ the number of active automatons tends to stabilise. Thus, for $t=360$ a total of 23.3 ≈ 24 active bots and 147 deactivated bots were evident. On the other hand, the comparison between the $PobD$ between SIM-1 and SIM-2 established statistically significant differences with $z=-14.66$, $p\text{-}\mathrm{value}=0.00$. Figure 6 shows the behaviour of the system for SIM-4.

5. Conclusions

The objective of this study, which was to model and simulate scenarios of the propagation of disinformation in social networks caused by bots based on the dynamics of this mechanism documented in the literature, was achieved. The model presented has the capacity to replicate the behaviour of the system, being consistent with the dynamic hypotheses set out in Figure 1, complementing previous studies such as those developed by de Lazer et al. [24], Shao et al. [25]; Vosoughi et al. [26] and Shao et al. [27], in relation to the use of bots as a mechanism to propagate false information.

This model allows actors involved in disinformation to analyse in a more objective way the behavioural patterns of disinformation caused by bots for decision making, based on three assumptions. The first one relates to the delay in the start of disinformation; the second one to the limited number of bots that the disinformation agent can put in place; and the third one to the limitations of social network systems to detect and deactivate automated accounts.

With that said, the simulations developed clarify that the system of disinformation using bots is susceptible to policies that are conducive to better detection, blocking and elimination of these types of accounts. In this sense, the uninformed population is smaller, hence the responsibility of social networks to design better detection mechanisms and of citizens to report these types of accounts, to increase the effective rate of deactivation of bots. However, if the disinformation agent starts its activity early, the impact over time will not be on the amount of the disinformed population, but on the number of bots required to achieve its purpose, since the number of bots tends to stabilise over time.

From the proposed model and the simulations developed, it is necessary to recognise the role of bots in aggravating existing social problems as a result of the propagation of false information, hence the need to delve deeper into various analyses such as the evolution of this type of mechanism, the new technologies they incorporate to circumvent the security systems of social networks, the use of artificial intelligence in these, among other aspects. On the other hand, it is necessary to urge the academic community to make use of the model, to complement it and, above all, to eliminate the current barriers to the study of disinformation, such as the difficulties of access to declassified information that would allow the model to be operationalised under conditions different from those expressed in this article and which are based on secondary data from other studies.