Next Article in Journal
Chimeric Virus-like Particle-Based COVID-19 Vaccine Confers Strong Protection against SARS-CoV-2 Viremia in K18-hACE2 Mice
Next Article in Special Issue
COVID-19 Vaccination Drive in a Low-Volume Primary Care Clinic: Challenges & Lessons Learned in Using Homegrown Self-Scheduling Web-Based Mobile Platforms
Previous Article in Journal
Probable Vogt–Koyanagi–Harada Disease after COVID-19 Vaccination: Case Report and Literature Review
Previous Article in Special Issue
Uptake of COVID-19 Vaccines among Pregnant Women: A Systematic Review and Meta-Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 10
Issue 5
10.3390/vaccines10050784
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Factors Associated with COVID-19 Vaccine Acceptance in Morocco: Applying the Health Belief Model

by
Imane Berni
1,*,†,
Aziza Menouni
1,2,†,
Younes Filali Zegzouti
1,
Marie-Paule Kestemont
3,
Lode Godderis
2,4 and
Samir El Jaafari
1
1
Cluster of Competency “Health and Environment”, Moulay Ismail University, Meknes 50000, Morocco
2
Environment and Health Unit, Department of Public Health and Primary Care, Katholic Universiteit of Leuven, 3000 Leuven, Belgium
3
Institute for the Analysis of Change in Contemporary and Historical Societies, Université Catholique de Louvain, 1348 Louvain-la-Neuve, Belgium
4
IDEWE, External Service for Prevention and Protection at Work, 3001 Heverlee, Belgium
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Vaccines 2022, 10(5), 784; https://0-doi-org.brum.beds.ac.uk/10.3390/vaccines10050784
Submission received: 20 March 2022 / Revised: 12 May 2022 / Accepted: 12 May 2022 / Published: 16 May 2022
(This article belongs to the Special Issue People’s Perception on COVID-19 Vaccine and Its Accessibility)
Browse Figures
Versions Notes

Abstract

:
To identify factors that influenced Moroccans’ intention to get a COVID-19 vaccine, a cross-sectional survey among a Moroccan sample was conducted based on Health Belief Model constructs. Participants’ sociodemographic characteristics, perceived susceptibility, perceived severity, perceived barriers, perceived benefits, self-efficacy, cues to action, and intention to receive vaccine data were collected and analyzed using a structural equation model (SEM). The survey was completed by 3800 individuals; 57.2% were men, 44.5% were aged 30 to 44, and 44.6% were married. After controlling for confounders, being a woman and having a chronic disease were associated with higher acceptance of the COVID-19 vaccine. The strongest predictor for the intention of receiving a vaccine was participants who were married. Most of the HBM constructs were shown to be significantly associated with vaccine acceptance. Susceptibility and Benefits were the strongest predictors of acceptance of the COVID-19 vaccine (standardized path coefficient, SPC = 0.23), followed by Severity (SPC = 0.22). Conversely, given the negative correlation between barriers and intention (SPC = −0.08), it is necessary to maintain a high level of transparency regarding the vaccines’ safety. Our study provides guidance for an implementation of vaccination strategies, intending to bolster the overall COVID-19 immunization program.

1. Introduction

Since its emergence in December 2019, the coronavirus disease 2019 (COVID-19) pandemic has caused catastrophic damage worldwide. As the health issue continues to linger and a sense of pandemic fatigue starts to take hold [1], it appears that the entire world is looking forward to the introduction of a safe and efficient vaccine [2]. In response, more than 50 vaccines for COVID-19 are either in clinical testing or have already been approved for limited use in some countries [3]. As of February 2021, two vaccines have been approved by the Health Ministry of Morocco for the novel Coronavirus [4]. While it seems that the entire globe is anticipating the arrival of safe and efficacious vaccines, the acceptance of the COVID-19 vaccine by people remains unknown.
In this context, a basic understanding of different factors influencing people’s intention to get vaccinated against COVID-19 is crucial to achieving herd immunity and developing effective interventions that promote vaccine acceptance. One such framework is the Health Belief Model (HBM). According to this model, to adopt healthy behavior, individuals need to consider themselves susceptible to the potential disease and perceive it has serious consequences (i.e., perceived threat), believe that certain health behaviors can improve and promote their wellbeing (i.e., perceived benefit), and that the benefits of healthy behaviors outweighed the costs (i.e., perceived barriers), and that taking action would have a benefit (cues to action) [5,6]. Additionally, as COVID-19 vaccines were not generally accessible in Morocco at the time of this study, we focused on the intention to receive COVID-19 vaccines due to their high correlation with the actual uptake behavior [7].
Previous literature has focused on people’s opinions regarding vaccination programs among the general population before introducing the vaccine. However, applying a behavioral model or theories such as the Health Belief Model (HBM) that could predict the acceptance of the COVID-19 vaccine has received less attention. Two studies were found that applied HBM in the general public context [8,9]. Furthermore, there has been no research conducted in North Africa to assess people’s willingness to get vaccinated against COVID-19. In response, the present study provides valuable insights into how the Health Belief Model impacts the Moroccan general public’s intention toward COVID-19 vaccine acceptance.
Overall, this study contributes to improving awareness in two ways. First, it sheds new light on factors that predict people’s intention to receive the COVID-19 vaccine. Second, it provides more empirical data on who is more likely to get a vaccine to help authorities plan the distribution of COVID-19 vaccines.

Theoretical Framework and Hypotheses Development

Developed by Champion and Skinner [10], HBM is now used as the most popular social–psychological theory for understanding people’s behaviors in various fields. This study proposed a hypothesis and analyzed six main factors that affect the public towards COVID-19 vaccine acceptance.
Vaccination, like several others health behaviors, is used to avoid adverse health outcomes [11]. Therefore, it is reasonable that people’s perceptions of how severe and likely a disease is to affect them should influence their willingness to receive a vaccine [12]. In the context of COVID-19 vaccination, if individuals believe themselves to be vulnerable to a COVID-19 disease, think that the disease would have significant consequences, and recognize that vaccination will help to minimize susceptibility and severity, they are likely to seriously undertake healthy behaviors that they think will reduce the risk [13]. Nevertheless, lower mortality rates among younger populations may lead to lower perceived risk and hesitation to get a COVID-19 vaccine [14]. Considering this, our first hypothesis was that perceived susceptibility and severity of COVID-19 will be positively related to vaccination intention.
Perceived benefits and barriers can affect the intention of the general public to get the COVID-19 vaccination; perceived benefits of getting vaccinated were strong predictors with uptake intent, while perceived barriers were negatively correlated with vaccination intention (e.g., seasonal influenza vaccine) [9]. People who consider the vaccines to benefit themselves and their communities held a generally positive view about COVID-19 vaccines [15]. Likewise, the accelerated development of COVID-19 vaccines and cost concerns have been reported as barriers [15]. Our second hypothesis was that perceived benefits and barriers of COVID-19 vaccination are positively associated with a higher COVID-19 vaccine uptake intent.
The Health Belief Model (HBM) contains two additional elements: cues to action and perceived self-efficacy [16], as showed on Figure 1. Cues to action work as “triggers” and thus motivate individuals to change behaviors, while self-efficacy is the belief that one can carry out behavior [17]. In the context of the COVID-19 vaccine, providing correct information via the government and experts acts as triggers for promoting engagement in vaccination campaigns [18]. Correspondingly, our final hypothesis was that increased cues to action and self-efficacy would positively affect vaccination intention.

2. Methods

2.1. Study Design

This cross-sectional questionnaire was conducted from 5 to 20 January 2021, 10 days immediately before the start of the vaccination campaign in Morocco. We decided to collect data online because it was not feasible to perform a community-based national sampling survey during this particular period. Relying on the social network, an online questionnaire was spread via Facebook, WhatsApp, Instagram, Twitter, and LinkedIn using sponsored social network advertisements. We used a convenience sampling method. The eligibility criteria were as follows: (1) subjects aged 18 or above; (2) lived in Morocco. A de-duplication protocol was applied to identify multiple submissions to preserve data integrity, including cross-validation of the eligibility criteria and discrepancies in key data, as well as checking for unusually fast completion time (<15 min). The registered members clicked the link on the platform and responded to the survey voluntarily until the convenience sample covered all 12 regions in Morocco.
Research was conducted with generally accepted ethical principles and approved by the Ethics Committee of Moulay Ismail University (N: CERB-UMI 03/2020). Online consent was obtained from the participants before completing the questionnaire. No compensation was provided.

2.2. Participants

A total of 4015 participants clicked on the survey link, and 3970 individuals started the survey, among whom 30 individuals refused to provide informed consent, and 3940 participants provided informed consent and submitted the questionnaire. After excluding 120 participants who did not complete all the questions and 20 respondents younger than 18 years old, the final sample consisted of 3800 participants.

2.3. Survey Content

We designed the study instrument after reviewing related questionnaires in published literature [19,20] to incorporate most of the constructs of the Health Belief Model. The first questionnaire section pertained to general information about the participant (age, education, gender, marital status, presence of chronic disease, flu vaccine uptake history, past infection with COVID-19, and monthly household income). The second questionnaire section was designed to measure intention to receive the COVID-19 vaccine on a scale based on the Health Belief Model. Twenty-four items in seven categories were considered. The participants were asked on a Likert scale how strongly they agreed or disagreed with each statement (1 as “strongly disagree” to 5 as “strongly agree”), except for the construct of intention to receive a COVID-19 vaccine, participants indicated for each statement the likelihood that they would consider getting COVID-19 vaccines on a scale of 1 (very unlikely) to 5 (very likely).
The face and content validity of the questionnaire were examined to assure that the items could subjectively/theoretically cover the constructs [21]. For this purpose, a review panel consisting of two specialists at the Faculty of Sciences in Meknes (a professor of biostatistics and epidemiology and a professor of biological sciences) was formed to assess the research instrument. Their feedback was incorporated to enhance the questions’ readability, completeness, and clarity. The questionnaire was piloted on a small sample of participants from the general population prior to the study (n = 50). Data from this pilot sample were not used in the subsequent analysis. As a result, the clarity and appropriateness of the questions were evaluated, and the questionnaire was edited accordingly. Additionally, Cronbach’s alpha was employed to examine internal consistency reliabilities of the questions.

2.4. Statistical Analysis

Data analysis was performed using the Statistical Package for Social Sciences (SPSS) version 20 (IBM). Frequencies and percentages were used for the description of the sociodemographic data. The primary outcome variable was a dichotomous variable on acceptance of COVID-19 vaccination. Vaccine acceptance was calculated based on the self-administrated questionnaire. The dependent variable “I intend to get vaccinated” had two possible values: 1 = (“Yes, likely/very likely”); 0 = (“No, very unlikely/unlikely/neutral). Forward univariate analysis based on Pearson’s chi-square tests at a significance level of 5% was used. To determine the association between sociodemographic variables and vaccine acceptance, a multivariable logistic regression analysis was performed. The associations between vaccine acceptance and outcomes are presented as odds ratios (O.R.s) and 95% CIs, after adjustment for confounders that showed the most effect from univariate analysis, including gender, marital status, influenza vaccination status, and infection with COVID-19.
Using the maximum likelihood procedure with AMOS software, Structural Equation Modeling (SEM) was adopted to test the relationship between perceptions and the intention to receive a COVID-19 vaccine. Following Anderson and Gerbing’s [22], a two-stage analytical process was used. First, a measurement model was estimated through Confirmatory Factor Analysis (CFA) by assessing the model fit, convergent validity, discriminant validity, and reliability. Then, a structural model was used to check the best-fitting model.
To assess the fit of the model, various indices were used in the present research, including relative Chi-square (<0.5), GFI (goodness of fit index) (GFI > 0.90), AGFI (adjusted GFI) (AGFI > 0.90), CFI (confirmatory fit index) (CFI > 0.90), IFI (incremental fit index) (IFI > 0.90), RMR (root mean square residual) (RMR > 0.08), and RMSEA (root mean square error of approximation) (RMSEA > 0.08) [23,24]. Regarding the convergent validity, three indicators are evaluated: item loading, average variance extracted (AVE) and construct reliability (CR) [25]. For acceptable convergent validity, item loading and AVE estimates must have a value of 0.5 or greater, while CR estimates must have a value of 0.7 or greater [22,26]. Similarly, discriminant validity was evaluated by comparing the AVE of each construct with the average shared squared variance (ASV) and the maximum shared squared variance (MSV). In this context, AVE should be greater than ASV and MSV [25].

3. Results

3.1. Descriptive Information of the Sample

Table 1 shows the sociodemographic characteristics of the sample concerning their intention to receive the COVID-19 vaccine. A total of 3800 respondents participated in the survey. Among the survey completers, 42.8% were women, 73.8% were aged 30 to 59, and 57.3% held a Bachelor’s degree or above. Around 55.4% were married, and 42.9% were infected by COVID-19 (Table 1).
About a third of respondents reported intending to vaccinate against COVID-19 (40%). All the sociodemographic variables were significantly associated with the intention to receive the COVID-19 vaccine (Table 1).

3.2. Sociodemographic Factors Associated with Acceptance of COVID-19 Vaccine

From the binary logistic regression analysis (Table 2), and after controlling for confounders, being a woman, married, and having a chronic disease was associated with higher acceptance of COVID-19 vaccine (e.g., gender: OR, 1.24; 95% Cl, 1.09–1.42; p < 0.001). Compared with uninfected participants, those with confirmed COVID-19 were more likely to get vaccinated (OR, 1.20; 95% Cl, 1.05–1.38; p < 0.001). The strongest predictor for intention to receive a COVID-19 vaccine was participants who is married; these participants were significantly more likely than other groups to receive a COVID-19 vaccine (OR, 131; 95% Cl, 1.15–1.50; p < 0.05)

3.3. Statistics of Key Variables

The descriptive statistics of key variables regarding COVID-19 vaccine acceptance are summarized in Table 3. In the interview results, the construct “perceived barriers” (Bar) showed the highest mean value (mean = 4.53) among all constructs, while the construct “severity of COVID-19” showed the lowest mean value (mean = 3.28) (Table 3). Of the four indicators of the intention to receive the COVID-19 vaccine construct, “I am currently undecided” was the most highly rated.
The construct “self-efficacy” (SE) received a total mean score of 4.5, indicating the Moroccan population’s strong perception of self-efficacy. People seemed to have enough ability to take action for COVID-19 prevention. According to Table 3, the construct “perceived benefits” (Ben) was rated high (mean = 4.5). Most of the respondents had a high level about the indicator “COVID-19 vaccine will be effective in preventing coronavirus.” “Cues to action” also received a high score (mean = 4.31) (Table 3). “Perceived susceptibility” received a moderate score (mean = 3.90). Most respondents felt worried about themselves or their family members contracting the virus.

3.4. Predictors of Intention to Vaccinate against COVID-19

3.4.1. Measurement Model

To investigate the adequacy of the model, the full measurement model was estimated by conducting CFA. In this study, the scales used were derived from those used in the relevant literature [24,25]. According to the CFA results (Table 4), all construct standardized factor loadings ranged from 0.419 to 0.901, except for three constructs, and were all significant at p < 0.001. Additionally, the AVE and CR values of each construct used in this study were greater than 0.50 and 0.70, respectively, except for AVE of the severity of COVID-19, thereby indicating good convergent validity and reliability. Further, the AVE values were all greater than those of ASV and MSV, supporting discriminant validity. Thus, it can be stated that all model measurements used in this study were valid and reliable.

3.4.2. Structural Model

SEM analysis showed that all fit indices were within acceptable ranges, indicating that the model was well fit to the data (Table 4). The square multiple correlation (R2) coefficient of the dependent variable SB was 0.472, which indicated that the investigated constructs explained 47.2% of the variable’s variance in the model.
Furthermore, significant associations were found between intention to receive a COVID-19 vaccine and other constructs, namely Sus, Sev, Ben, Bar, SE, and CTA (Figure 2). The perceived susceptibility and perceived benefits were the most important factors influencing intention to get vaccinated in Morocco (SPC = 0.23), followed by perceived severity (SPC = 0.22), cues to action (SPC = 0.17), and self-efficacy (SPC = 0.16). The path analysis of the general public indicated that perceived barriers (SPC = −0.08) had a major negative effect on the intention to receive a COVID-19 vaccine.
Suc—perceived susceptibility; Sev—severity of COVID-19; Bar—perceived barriers; Ben—perceived benefits; SE—self-efficacy; CtA—cues to action; Int—intention to receive COVID-19 vaccine; significant at p < 0.001

4. Discussion

The current study provides insights into how various psychosocial factors are associated with COVID-19 vaccine uptake among the Moroccan population. The findings indicated that most of the HBM constructs were shown to be significantly associated with vaccine acceptance. In particular, respondents who perceived COVID-19 as serious, believed the vaccine conferred benefits, and received cues to action were significantly more likely to accept the vaccine. On the other hand, the perception of access barriers was negatively associated with their acceptance. Similar to previous research [20,27], we found that the Health Belief Model helps identify major determinants of getting the COVID-19 vaccine. These findings are useful for informing public health strategies to improve willingness and uptake of the COVID-19 vaccine among the general population.
We found a significantly positive correlation between age and intention to receive a vaccine: older participants adopted a higher level in response to COVID-19 vaccine acceptance than younger participants. Malik et al. and Wong et al. [28,29] also reported that vaccine uptake was more common among elderly individuals. Given their greater awareness of the risks associated with coronavirus and their higher risk of contracting the more severe disease and death due to COVID-19 [30], older individuals had higher acceptance of the COVID-19 vaccine. This could also explain, in our findings, why participants with chronic diseases were considerably more likely to express vaccine acceptance. In developed countries, as in developing countries such as Morocco, it is essential to regard these individuals as a top-priority group to receive the COVID-19 vaccine.
Our study corroborates research on other viruses that found that women had higher acceptance of the COVID-19 vaccine [31]. In the USA, it has recently been shown that more women receive COVID-19 vaccines than men, even though more men die of the disease [32]. However, misinformation and access challenges have increased inequities in distribution in most Indian states, and women have received fewer vaccines than men [33]. Vaccine safety has been mentioned as a source of concern [34], and indeed our analyses suggest that women can be more easily convinced than men to get vaccinated.
Past infection with COVID-19 was one of the factors affecting vaccine acceptance. Those with confirmed COVID-19 were more likely to get vaccinated than uninfected people. This is partially consistent with previous research on H1N1 [35]. Many of the respondents of our study experienced COVID-19 disease. Consequently, they thought that the disease could be difficult to control.
Despite the importance of susceptibility and severity of COVID-19 as a significant predictor of vaccine acceptance, the study results revealed that these components had a significant effect on the intention to receive vaccine compared to other constructs of the HBM, supporting hypothesis 1. This underscores that the greater the perceived susceptibility, the higher the individual’s likelihood of getting the vaccine, which would mitigate the risks of COVID-19 exposure. The possible reasons for this may be related to the ‘hypochondriac concerns” (worry about being infected and infecting family) and the panic that the disease would be difficult to control. In a similar sense, Dini et al. [36] argued that when self-protection beliefs are associated with protecting others, the altruistic behaviors of the population are reduced, and vaccine uptake is higher. However, Wong et al. [19] found that perceived susceptibility was not associated with vaccine acceptance.
On the other hand, the findings of our study also revealed that the perceived severity of COVID-19 has the most significant influence on the intention to receive a vaccine. This implies that people who perceive COVID-19 as serious will be more likely to accept the vaccine. From a psychological perspective, individuals take actions toward health if they believe that harm can be serious [37]. The severity of the health risk is a key feature in convincing people to get vaccinated. In fact, the date our survey questionnaire was finished, COVID-19 caused more than two million confirmed cases and about 157955 deaths worldwide [38].This health threat is assumed as an indication to alter the public’s future behavior toward a vaccine against COVID-19. Musa Ibrahim discussed that severity includes beliefs about the disease itself and broader impacts of the disease on an individual’s social role [39]. This, in turn, may lead to higher chances of behavior to avoid these impacts. Transmission dynamic models of the COVID-19 outbreak show that containment played a major role in the spread limitation of infection in Morocco [40]. Taking this into account, the promotion of extension services and public awareness campaigns to sensitize the public as to the persistent risk of COVID-19 even after de-containment can be regarded as a key policy option [41]. In this respect, educational courses highlighting personal risks to the disease and using fear appeal messages via mass media such as television and social media are defined as making the public aware of the severity of coronavirus and promoting the COVID-19 vaccine.
Our findings partially supported Hypothesis 2, as the only perceived benefit was positively associated with the intention to receive the COVID-19 vaccine. However, we suggest a negative association between perceived barriers and the acceptance of the COVID-19 vaccine. In our study, the general public took a generally favorable opinion toward COVID-19 vaccines. They believed the vaccines effectively prevented the disease for both themselves and their families. Nevertheless, long-term side-effects, inadequate data concerning the COVID-19 vaccine, fear of the vaccine causing the COVID-19 disease, and confusion about efficacy were serious barriers to people’s vaccination intention. This finding is in concordance with previous studies conducted in other regions, showing that beliefs about COVID-19 vaccines’ benefits and barriers influenced intention to receive COVID-19 vaccine; the former was positive, while the latter was negative [15,42]. Overall, lack of knowledge of the COVID-19 vaccine and reservations about side-effects are still the key reasons for hesitancy to get the vaccine [29]. In Morocco, the safety, efficacy, and strengths of the COVID-19 vaccine adopted have been confirmed through positive findings and clinical trials conducted in Morocco and other countries [4]. Thus, building trust between the Moroccan population and health authorities, disseminating correct knowledge, and ensuring transparency in the process of vaccine approval through an effective outreach strategy could be an effective option to improve vaccine acceptance. Furthermore, public health authorities need to inform the public about expected post-vaccination symptoms as well as the rarity of major side effects.
Different from existing research [18], this study’s findings showed a significant positive association between CtA and self-efficacy and intention to get vaccinated for COVID-19, supporting Hypothesis 3. This is partially consistent with previous research on H1N1, which also shows that self-efficacy is positively related to vaccine uptake [43]—knowing that getting vaccinated to prevent contracting COVID-19 may motivate the public to adopt the behavior. Thus, the recommendation from the expert and doctor stood out as the most important cue, far stronger than others, such as from friends and family members. Therefore, since our result reveals that people have trust in health authorities and they agree to follow the advice of their doctor, the healthcare professionals should be equipped with the knowledge of vaccine development, efficacy, and provide information about the difference between selected vaccines so that they can confidently answer the concerns of the public and make them understand the rationale for getting vaccinated against COVID-19.
Our study has several limitations that can be addressed in future work. First, considering the limited resources available and the time sensitivity of the COVID-19 outbreak, we followed the snowball sampling method. The snowballing sampling technique did not use a random sample range, and the study population did not represent the general population’s actual trend. Secondly, our study was limited in scope. Our sample was obviously over-representative of men and well-educated people, limiting the generalization of our findings to the whole population. Thirdly, this study was carried out before COVID-19 vaccines were available. Thus, actual vaccine uptake was not evaluated. A further follow-up to test the current framework at a later pandemic stage could give a complete picture of vaccine hesitancy. Fourth, although incorporating elements affecting the intention to get vaccinated against COVID-19 in the model significantly improved the model’s explanatory capacity, a considerable part of the variance remained unexplained. Future research could include other variables such as attitudes toward vaccines, fears, and injunctive norms that have been relatively less examined.
As of 6 March 2022, fortunately, against the findings of this study, 14 months after the survey was conducted in January 2021, the percentage of Moroccans who had received at least one and two vaccination shots were 66.59% and 62.47%, respectively [44], exceeding the levels predicted by the outcome of this study analysis. Thus, the reason why the Moroccan community was able to overcome its initial low vaccination intentions and achieve a high vaccination achievement rate remains an issue to be unraveled by future research and constitute the last limitation of this study.
Despite these above limitations, the findings from this study have important public health implications. Overall, the findings indicate that our ability to increase uptake of available COVID-19 vaccines among females, young adults and participants with lower education levels may rely on appropriately communicating information specifically about the safety of the vaccine, how it works to protect against infection, and why it is important to protect oneself and others against COVID-19. Given the public’s confidence in an expert or doctor, it will be necessary for public health strategies to engage healthcare professionals as possible communication channels for trusted information. These strategies could be successful in increasing inoculation uptake. It is also essential to implement strategies to limit the spread of conspiracy theories.

5. Conclusions

This study is the first in Morocco to shed the light on the socio-psychological factors that determine Moroccan’s intention to receive the COVID-19 vaccine. Overall, various demographic and psychological predictors were correlated to the intention to get vaccinated against COVID-19. The results showed that perceived susceptibility and benefits were the strongest predictors of acceptance of the COVID-19 vaccine. Further, given the negative correlation between perceived barriers and vaccination intention, it is necessary to maintain a high level of transparency regarding the vaccines’ effectiveness and safety. Our study findings raise major concerns about equitable vaccination, with lower acceptance among individuals with chronic disease who most require the vaccinations. There is an urgent need to further explore and address the fears and concerns of these groups to ensure equitable access to and utilization of COVID-19 vaccines. These identified constructs provide evidence-based formulation and implementation of vaccination strategies to bolster the overall COVID-19 immunization program.

Author Contributions

Conceptualization, I.B. and A.M.; data curation, I.B.; formal analysis, I.B., A.M., Y.F.Z. and M.-P.K.; funding acquisition, A.M., L.G. and S.E.J.; investigation, I.B. and A.M.; methodology, I.B., A.M., Y.F.Z., M.-P.K., L.G. and S.E.J.; project administration, A.M.; resources, I.B. and A.M.; software, I.B. and Y.F.Z.; supervision, S.E.J.; validation, M.-P.K., L.G. and S.E.J.; visualization, I.B.; writing—original draft, I.B. and A.M.; writing—review and editing, Y.F.Z., M.-P.K., L.G. and S.E.J. All authors have read and agreed to the published version of the manuscript.

Funding

The APC was supported by the Environmental health project of the Institutional University Cooperation Program with Moulay Ismail University of Meknes, funded by the VLIR-UOS (Contract n° A2017IUC038A104).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Moulay Ismail University of Meknes (CERB03/2020, 18 March 2020).

Informed Consent Statement

Informed consent was obtained online from all subjects involved in the study before proceeding with the questionnaire.

Data Availability Statement

Not applicable.

Acknowledgments

The authors gratefully acknowledge the generosity of those who voluntarily responded to the questionnaire of this study.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Menouni, A.; Berni, I.; Abchouch, T.; Khouchoua, M.; Filali-Zegzouti, Y.; Janssen, P.; Kestemont, M.P.; Godderis, L.; El Jaafari, S. Mental Health among the Moroccan Population during SARS-CoV-2 Outbreak: MAROCOVID Study. J. Affect. Disord. 2022, 308, 343–352. [Google Scholar] [CrossRef] [PubMed]
  2. Altmann, D.M.; Douek, D.C.; Boyton, R.J. What Policy Makers Need to Know about COVID-19 Protective Immunity. Lancet 2020, 395, 1527–1529. [Google Scholar] [CrossRef]
  3. World Health Organization COVID-19 Vaccines. Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/covid-19-vaccines (accessed on 13 July 2021).
  4. Ministère de La Santé Campagne Nationale de Vaccination Contre Le Virus de La COVID-19. Available online: https://www.sante.gov.ma/Pages/actualites.aspx?IDActu=431 (accessed on 13 July 2021).
  5. Buglar, M.E.; White, K.M.; Robinson, N.G. The Role of Self-Efficacy in Dental Patients’ Brushing and Flossing: Testing an Extended Health Belief Model. Patient Educ. Couns. 2010, 78, 269–272. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Kim, S.; Kim, S. Analysis of the Impact of Health Beliefs and Resource Factors on Preventive Behaviors against the COVID-19 Pandemic. Int. J. Environ. Res. Public Health 2020, 17, 8666. [Google Scholar] [CrossRef]
  7. Gerend, M.A.; Shepherd, J.E. Predicting Human Papillomavirus Vaccine Uptake in Young Adult Women: Comparing the Health Belief Model and Theory of Planned Behavior. Ann. Behav. Med. 2012, 44, 171–180. [Google Scholar] [CrossRef] [Green Version]
  8. Jose, R.; Narendran, M.; Bindu, A.; Beevi, N.; Manju, L.; Benny, P.V. Public Perception and Preparedness for the Pandemic COVID 19: A Health Belief Model Approach. Clin. Epidemiol. Glob. Health 2021, 9, 41–46. [Google Scholar] [CrossRef]
  9. Santos, A.J.; Kislaya, I.; Machado, A.; Nunes, B. Beliefs and Attitudes towards the Influenza Vaccine in High-Risk Individuals. Epidemiol. Infect. 2017, 145, 1786–1796. [Google Scholar] [CrossRef] [Green Version]
  10. Champion, V.L.; Skinner, C.S. The Health Belief Model. In Health Behavior and Health Education: Theory, Research, and Practice, 4th ed.; Jossey-Bass: San Francisco, CA, USA, 2008. [Google Scholar]
  11. Brien, S.; Kwong, J.C.; Buckeridge, D.L. The determinants of 2009 pandemic A/H1N1 influenza vaccination: A systematic review. Vaccine 2012, 30, 1255–1264. [Google Scholar] [CrossRef]
  12. Weinstein, N.D. Testing Four Competing Theories of Health-Protective Behavior. Health Psychol 1993, 12, 324–333. [Google Scholar] [CrossRef]
  13. Dryhurst, S.; Schneider, C.R.; Kerr, J.; Freeman, A.L.J.; Recchia, G.; Bles, A.M.; Spiegelhalter, D.; Linden, S. Risk Perceptions of COVID-19 around the World. J. Risk Res. 2020, 23, 994–1006. [Google Scholar] [CrossRef]
  14. Kreps, S.E.; Kriner, D.L. Factors Influencing COVID-19 Vaccine Acceptance across Subgroups in the United States: Evidence from a Conjoint Experiment. Vaccine 2021, 39, 3250–3258. [Google Scholar] [CrossRef] [PubMed]
  15. Chu, H.; Liu, S. Integrating Health Behavior Theories to Predict American’s Intention to Receive a COVID-19 Vaccine. Patient Educ. Couns. 2021, 104, 1878–1886. [Google Scholar] [CrossRef] [PubMed]
  16. Hanson, J.A.; Benedict, J.A. Use of the Health Belief Model to Examine Older Adults’ Food-Handling Behaviors. J. Nutr. Educ. Behav. 2002, 34, 25–30. [Google Scholar] [CrossRef]
  17. Mukhtar, S. Mental Health and Emotional Impact of COVID-19: Applying Health Belief Model for Medical Staff to General Public of Pakistan. Brain Behav. Immun. 2020, 87, 28–29. [Google Scholar] [CrossRef]
  18. Wong, M.C.S.; Wong, E.L.Y.; Huang, J.; Cheung, A.W.L.; Law, K.; Chong, M.K.C.; Ng, R.W.Y.; Lai, C.K.C.; Boon, S.S.; Lau, J.T.F.; et al. Acceptance of the COVID-19 Vaccine Based on the Health Belief Model: A Population-Based Survey in Hong Kong. Vaccine 2021, 39, 1148–1156. [Google Scholar] [CrossRef]
  19. Alhalaseh, L.; Fayoumi, H.; Khalil, B. The Health Belief Model in Predicting Healthcare Workers’ Intention for Influenza Vaccine Uptake in Jordan. Vaccine 2020, 38, 7372–7378. [Google Scholar] [CrossRef]
  20. Wang, C.; Horby, P.W.; Hayden, F.G.; Gao, G.F. A Novel Coronavirus Outbreak of Global Health Concern. Lancet 2020, 395, 470–473. [Google Scholar] [CrossRef] [Green Version]
  21. Berni, I.; Menouni, A.; Ghazi El, I.; Duca, R.-C.; Kestemont, M.-P.; Godderis, L.; EL Jaafari, S. Understanding Farmers’ Safety Behavior Regarding Pesticide Use in Morocco. Sustain. Prod. Consum. 2021, 25, 471–483. [Google Scholar] [CrossRef]
  22. Anderson, R.M.; Heesterbeek, H.; Klinkenberg, D.; Hollingsworth, T.D. How Will Country-Based Mitigation Measures Influence the Course of the COVID-19 Epidemic? Lancet 2020, 395, 931–934. [Google Scholar] [CrossRef]
  23. Rezaei, R.; Safa, L.; Damalas, C.A.; Ganjkhanloo, M.M. Drivers of Farmers’ Intention to Use Integrated Pest Management: Integrating Theory of Planned Behavior and Norm Activation Model. J. Environ. Manag. 2019, 236, 328–339. [Google Scholar] [CrossRef]
  24. Bagheri, A.; Bondori, A.; Allahyari, M.S.; Damalas, C.A. Modeling Farmers’ Intention to Use Pesticides: An Expanded Version of the Theory of Planned Behavior. J. Environ. Manag. 2019, 248, 109291. [Google Scholar] [CrossRef]
  25. Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis, 7th ed.; Pearson Education Ltd.: Essex, UK, 2010; ISBN 978-1-292-02190-4. [Google Scholar]
  26. Fornell, C.; Larcker, D.F. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
  27. Carico, R.; Sheppard, J.; Thomas, C.B. Ron. Res. Soc. Adm. Pharm. 2021, 17, 1984–1987. [Google Scholar] [CrossRef] [PubMed]
  28. Malik, A.A.; McFadden, S.M.; Elharake, J.; Omer, S.B. Determinants of COVID-19 Vaccine Acceptance in the US. EClinicalMedicine 2020, 26, 100495. [Google Scholar] [CrossRef] [PubMed]
  29. Qattan, A.M.N.; Alshareef, N.; Alsharqi, O.; Al Rahahleh, N.; Chirwa, G.C.; Al-Hanawi, M.K. Acceptability of a COVID-19 Vaccine Among Healthcare Workers in the Kingdom of Saudi Arabia. Front. Med. 2021, 8, 83. [Google Scholar] [CrossRef]
  30. Department of Health and Social Care. Available online: https://www.gov.uk/government/publications/priority-groups-for-coronavirus-covid-19-vaccination-advice-from-the-jcvi/interim-advice-on-priority-groups-for-covid-19-vaccination (accessed on 13 May 2022).
  31. Sun, S.; Lin, D.; Operario, D. Interest in COVID-19 Vaccine Trials Participation among Young Adults in China: Willingness, Reasons for Hesitancy, and Demographic and Psychosocial Determinants. Prev. Med. Rep. 2021, 22, 101350. [Google Scholar] [CrossRef] [PubMed]
  32. Hawlader, M.D.H.; Rahman, M.L.; Nazir, A.; Ara, T.; Haque, M.M.A.; Saha, S.; Barsha, S.Y.; Hossian, M.; Matin, K.F.; Siddiquea, S.R.; et al. COVID-19 Vaccine Acceptance in South Asia: A Multi-Country Study. Int. J. Infect. Dis. 2022, 114, 1–10. [Google Scholar] [CrossRef]
  33. Guha, N. India’s COVID Gender Gap: Women Left behind in Vaccination Drive. The Guardian, 28 June 2021. [Google Scholar]
  34. Schwarzinger, M.; Flicoteaux, R.; Cortarenoda, S.; Obadia, Y.; Moatti, J.-P. Low Acceptability of A/H1N1 Pandemic Vaccination in French Adult Population: Did Public Health Policy Fuel Public Dissonance? PLoS ONE 2010, 5, 10199. [Google Scholar] [CrossRef]
  35. Ludolph, R.; Nobile, M.; Hartung, U.; Castaldi, S.; Schulz, P.J. H1N1 Influenza Pandemic in Italy Revisited: Has the Willingness to Get Vaccinated Suffered in the Long Run? J. Public Health Res. 2015, 4, 559. [Google Scholar] [CrossRef] [Green Version]
  36. Dini, G.; Toletone, A.; Sticchi, L.; Orsi, A.; Bragazzi, N.L.; Durando, P. Influenza Vaccination in Healthcare Workers: A Comprehensive Critical Appraisal of the Literature. Hum. Vaccines Immunother. 2018, 14, 772–789. [Google Scholar] [CrossRef] [Green Version]
  37. Glanz, K.; Rimer, B.K.; Viswanath, K. Health Behavior and Health Education: Theory, Research, and Practice; John Wiley & Sons: Hoboken, NJ, USA, 2008. [Google Scholar]
  38. WHO. Coronavirus Disease 2019 (COVID-19) Situation Reports. 2020. Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports (accessed on 26 April 2020).
  39. Musa Ibrahim, A. Using Health Beliefs Model as an Intervention to Non Compliance with Hypertension Information among Hypertensive Patient. Int. Organ. Sci. Res. J. Hum. Soc. Sci. 2015, 20, 11–16. [Google Scholar] [CrossRef]
  40. Hammoumi, A.; Qesmi, R. A First Study on the Impact of Containment Measure on COVID-19 Spread in Morocco. medRxiv 2020, 140, 110231. [Google Scholar] [CrossRef]
  41. Berni, I.; Menouni, A.; Zegzouti, Y.F.; Kestemont, M.-P.; Godderis, L.; El Jaafari, S. MAROCOVID: Snapshot Monitoring of Knowledge and Perceptions of Safety Behaviors during the COVID-19 Outbreak in Morocco. Int. J. Environ. Res. Public Health 2021, 18, 5745. [Google Scholar] [CrossRef] [PubMed]
  42. Maraqa, B.; Nazzal, Z.; Rabi, R.; Sarhan, N.; Al-Shakhra, K.; Al-Kaila, M. COVID-19 Vaccine Hesitancy among Health Care Workers in Palestine: A Call for Action. Prev. Med. 2021, 149, 106618. [Google Scholar] [CrossRef] [PubMed]
  43. Gargano, L.M.; Painter, J.E.; Sales, J.M.; Morfaw, C.; Jones, L.M.; Murray, D.; Wingood, G.M.; DiClemente, R.J.; Hughes, J.M. Seasonal and 2009 H1N1 influenza vaccine uptake, predictors of vaccination, and self-reported barriers to vaccination among secondary school teachers and staff. Hum. Vaccines 2011, 7, 89–95. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Ministère de la Santé Covidvax.live: Live COVID-19 Vaccination Tracker—Voir les Vaccinations en Temps Réel! Available online: http://covidvax.live/fr/location/mar (accessed on 15 March 2022).
Vaccines 10 00784 g001 550
Figure 1. Theoretical framework.
Figure 1. Theoretical framework.
Vaccines 10 00784 g001
Vaccines 10 00784 g002 550
Figure 2. Standardized path coefficients of the factors that directly and indirectly affect intention to receive COVID-19 vaccine in Morocco.
Figure 2. Standardized path coefficients of the factors that directly and indirectly affect intention to receive COVID-19 vaccine in Morocco.
Vaccines 10 00784 g002
Table
Table 1. Frequency of intention to receive COVID-19 vaccine by sociodemographic characteristics.
Table 1. Frequency of intention to receive COVID-19 vaccine by sociodemographic characteristics.
CharacteristicsTotal Number (%)I Intend to Get VaccinatedNot Intending to Vaccinatep-Value
Total, No (%)3800 (100)1521 (40)2279 (60)
Gender
  Men2175 (57.2)829 (38.1)1346 (61.8)0.003
  Women1625 (42.8)692 (42.6)933 (57.4)
Age group
  18–29806 (21.2)36 (42.9)460 (57.1)0.046
  30–441690 (44.5)661 (39.1)1029 (60.9)
  45–591114 (29.3)430 (38.6)684 (61.4)
  >60190 (5)84 (44.2)106 (55.8)
Marital status
  Others1693 (44.6)617 (36.4)1076 63.6)0.000
  Married2107 (55.4)904 (42.9)1203 (57.1)
Educational level
  >Primary school617 (16.2)228 (37)389 (63)0.033
  Secondary school1003 (26.4)377 (37.6)626 (62.4)
  Bachelor’s degree963 (25.3)409 (42.5)554 (57.5)
  Master’s and PhD1217 (32)507 (41.7)710 (58.3)
Chronic disease
  None2439 (64.2)1000 (41)1439 (59)0.041
  Present 1361 (35.8)521 (38.7)840 (61.7)
Influenza vaccine
  Never3271 (86.1)1334 (40.8)1937 (59.2)0.01
  Once a year or more529 (13.9)187 (35.3)342 (64.7)
Infection with COVID-19
  Not infected2171 (57.1)837 (38.6)1334 (61.4)0.019
  Confirmed cases1629 (42.9)684 (42.0)945 (58.0)
Monthly family income (MAD)
  <2000,1198 (31.5)510 (42.6)688 (57.4)0.043
  2000–4000,197 (5.2)74 (37.6)123 (62.4)
  4000–8000,449 (11.8)178 (39.6)271 (60.4)
  8000–12,000,921 (24.2)364 (39.5)557 (60.5)
  >12,0001035 (27.2)395 (38.5)640 (61.5)
p-value for the Pearson chi-squared proportion test at significance level of (α) 5%.
Table
Table 2. Factors associated with acceptance of COVID-19 vaccine by binary logistic regression.
Table 2. Factors associated with acceptance of COVID-19 vaccine by binary logistic regression.
Total, No (%)nVaccine Acceptance %Odds Ratios95% Confidence Intervalp-Value
Gender
  Men217538.1Ref
  Women162542.61.241.09–1.420.001
Age group
  18–2980642.9ref
  30–44169039.11.191.05–1.670.030
  45–59111438.61.271.13–1.680.023
  >6019044.21.321.18–1.740.047
Marital status
  Others169336.4ref
  Married210742.91.311.16–1.500.000
Educational level
  >Primary school61737ref
  Secondary school100337.60.980.82–1.160.831
  Bachelor’s degree96342.51.180.99–1.400.055
  Master’s and PhD121741.71.210.99–1.480.059
Chronic disease
  None 243941ref
  Present 136138.31.241.09–1.620.004
Influenza vaccine
  Never327140.8ref
  Once a year or more52935.31.070.93–1.230.322
Infection with COVID-19
  No infected217138.6ref
  Confirmed cases1629421.201.05–1.380.007
Monthly income family (MAD)
  <2000,119842.6ref
  2000–4000,19737.60.830.69–1.020.078
  4000–8000,44939.61.050.76–1.4440.774
  8000–12,000,92139.50.960.76–1.210.727
  >12,000103538.30.950.80–1.150.632
Table
Table 3. Descriptive statistics of key variables concerning COVID-19 vaccine acceptance (n = 3800).
Table 3. Descriptive statistics of key variables concerning COVID-19 vaccine acceptance (n = 3800).
ConstructVariable Description (Symbols)Mean (SD)
Perceived susceptibility (Sus)3.90 (1.22)
I am at risk of getting COVID-19 (Suc1)3.88 (1.3)
It is likely that my children will be infected by the Coronavirus (Suc2)3.92 (1.19)
It is possible that the elderly will get COVID-19 in the coming 9 months (Suc3)3.92 (1.18)
Cronbach’s α 0.778
Severity of COVID-19 (Sev)3.28 (1.23)
I think that COVID-19 is a serious threat to human health (Sev1)4.12 (1.05)
I believe that if I catch COVID-19 it will have a serious consequence for my life (Sev2)2.42 (1.21)
I believe that COVID-19 can lead to death of my loves one if they get infected (Sev3)2.97 (1.36)
I’m afraid to catch COVID-19 (Sev4)3.62 (1.32)
Cronbach’s α 0.703
Perceived barriers (Bar)4.53 (0.63)
I have concerns about COVID-19 vaccine long-term side effects (Bar1)4.78 (0.57)
Not enough research done about COVID-19 vaccine (Bar2)4.45 (0.64)
The COVID-19 vaccine causes a person to get COVID-19 (Bar3)4.45 (0.66)
I am not sure if COVID-19 vaccine is effective in preventing the disease (Bar4)4.45 (0.64)
Cronbach’s α 0.701
Perceived benefits (Ben)4.5 (0.82)
COVID-19 vaccine will be effective in preventing Coronavirus (Ben1)4.67 (0.82)
If I get the vaccines, I will be less likely to get COVID-19 (Ben2)4.57 (0.83)
I think COVID-19 vaccine can prevent people from spreading the virus to others (Ben3)4.27 (0.82)
Cronbach’s α 0.665
Self-efficacy (SE) 4.5 (0.84)
I will be able to get the vaccines to prevent contracting COVID-19 (SE1)4.76 (0.66)
It will be easy for me to get the vaccines to protect myself from COVID-19 (SE2)4.28 (0.87)
Getting vaccinated to prevent COVID-19 is convenient (SE3)4.47 (1.00)
Cronbach’s α 0.763
Cues to action (CtA)4.31 (0.89)
I will be more optimistic about COVID-19 vaccine if I know more about it (CtA1)4.05 (0.96)
The type of vaccine that is available would affect my decision (CtA2)4.39 (0.94)
I would be more confident if experts or people I trust would recommend the vaccine (CtA3)4.49 (0.76)
Cronbach’s α 0.668
Intention to receive a COVID-19 vaccine (Int)3.56 (1.28)
I intend to get vaccinated as soon as possible (Int1)3.49 (1.39)
I probably get it but not as soon as possible (Int2)3.84 (1.19)
I get vaccinated if an expert or doctor I trust recommend me COVID-19 vaccines (Int3)2.74 (1.41)
I am currently undecided (Int4)4.18 (1.16)
Cronbach’s α 0.718
Table
Table 4. Results of first-order confirmatory factor analysis.
Table 4. Results of first-order confirmatory factor analysis.
ConstructsMeasurement ItemsStd. Loadingt-ValueReliability and Variability
Perceived susceptibility (Sus)Sus10.28129.579AVE = 0.656; CR = 0.828; ASV = 0.017; MSV = 0.033
Sus20.97793.517
Sus30.963fixed
Severity of COVID-19 (Sev)Sev10.85616.052AVE = 0.502; CR = 0.789; ASV= 0.024; MSV = 0.062
Sev20.3617.342
Sev30.721fixed
Sev40.723.626
Perceived barriers (Bar)Bar10.68129.601AVE = 0.509; CR = 0.794; ASV = 0.025; MSV = 0.098
Bar20.39121.988
Bar30.859fixed
Bar40.82651.166
Perceived benefits (Ben)Ben10.839fixedAVE = 0.524; CR = 0.740; ASV = 0.026; MSV = 0.171
Ben20.88732.896
Ben30.28916.638
Self-efficacy (SE)SE10.82520.785AVE = 0.548; CR = 0.779; ASV = 0.051; MSV = 0.098
SE20.56316.638
SE30.804fixed
Cues to action (CtA)CtA10.61327.509AVE = 0.528; CR = 0.767; ASV = 0.030; MSV = 0.171
CtA20.68831.806
CtA30.858fixed
Intention (Int)Int10.8fixedAVE = 0.504; CR = 0.866; ASV = 0.051; MSV = 0.171
Int20.66626.623
Int30.75920.535
Int40.82823.626
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Berni, I.; Menouni, A.; Filali Zegzouti, Y.; Kestemont, M.-P.; Godderis, L.; El Jaafari, S. Factors Associated with COVID-19 Vaccine Acceptance in Morocco: Applying the Health Belief Model. Vaccines 2022, 10, 784. https://0-doi-org.brum.beds.ac.uk/10.3390/vaccines10050784

AMA Style

Berni I, Menouni A, Filali Zegzouti Y, Kestemont M-P, Godderis L, El Jaafari S. Factors Associated with COVID-19 Vaccine Acceptance in Morocco: Applying the Health Belief Model. Vaccines. 2022; 10(5):784. https://0-doi-org.brum.beds.ac.uk/10.3390/vaccines10050784

Chicago/Turabian Style

Berni, Imane, Aziza Menouni, Younes Filali Zegzouti, Marie-Paule Kestemont, Lode Godderis, and Samir El Jaafari. 2022. "Factors Associated with COVID-19 Vaccine Acceptance in Morocco: Applying the Health Belief Model" Vaccines 10, no. 5: 784. https://0-doi-org.brum.beds.ac.uk/10.3390/vaccines10050784

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop