Next Article in Journal
Primary School Physical Education at the Time of the COVID-19 Pandemic: Could Online Teaching Undermine Teachers’ Self-Efficacy and Work Engagement?
Next Article in Special Issue
Use of MOOCs in Health Care Training: A Descriptive-Exploratory Case Study in the Setting of the COVID-19 Pandemic
Previous Article in Journal
Possibility of a Solution of the Sustainability of Transport and Mobility with the Application of Discrete Computer Simulation—A Case Study
Previous Article in Special Issue
Adult Education: A Sustainable Model for the Reduction of Psychosocial and Educational Risks Caused by COVID-19
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Development and Validation of a Measure to Assess COVID-19 Stress among College Students for Sustainable Education

Department of Psychology, Manipal University Jaipur, Jaipur 303007, India
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(17), 9828; https://0-doi-org.brum.beds.ac.uk/10.3390/su13179828
Submission received: 2 August 2021 / Revised: 25 August 2021 / Accepted: 27 August 2021 / Published: 1 September 2021

Abstract

:
The impact of the COVID-19 pandemic on students’ personal, academic, and social life has been quite stressful. The threat to life from the virus, social isolation, and the need to shift from face-to-face learning to online mode has been challenging. Thus, the purpose of the present study was to develop and validate a self-administered tool to assess the source of COVID-19 pandemic stress among college students. The data was collected online using google forms in two phases. The data collected from 173 participants from phase one was analyzed using exploratory factor analysis (EFA). The second phase data of 216 participants were analyzed to validate the factor structure using confirmatory factor analysis (CFA). The standardized factor loadings, Composite Reliability (CR), and AVE of factors were assessed to determine the convergent validity of the scale. Similarly, discriminant validity and concurrent predictive validity were assessed through the HTMT ratio of correlations and ROC curves, respectively. A succession of Exploratory Factor Analysis yielded a five-factor solution, explaining 73.83 percent variance with 13 indices. CFA via maximum likelihood with bootstrapping indicated a good fit for the five-factor model (SRMR = 0.037, RMSEA = 0.049, CFI = 0.981). The standardized factor loadings, Composite Reliability (CR), and AVE of factors together suggest acceptable convergent validity. Further, the ROC curve results to assess stress indicated an acceptable AUC, with a magnitude of 0.79, p < 0.01, indicating concurrent predictive validity for the five-factor Student COVID-19 Stress Scale (SCSS). HTMT ratio of correlations <0.85 indicated discriminant validity for the factor structure. The CR > 0.70 for the dimensions indicated acceptable reliability. Thus, the SCSS can be an effective instrument to assess the source of pandemic stress among students so that tailor-made timely interventions can be provided to prevent long-term adverse effects.

1. Introduction

The spread of COVID-19 (SARS-CoV-2) worldwide has endangered the lives of individuals and posed a threat to the sustainable growth and development of nations. It has been more evident in the case of developing nations. Higher education institutions, particularly universities and colleges, tend to be the mainstay of any nation’s intellectual capital. In the contemporary world, these institutions are responsible for supplying human capital and have become innovation incubators for budding entrepreneurs.
For achieving sustainable growth and development, health and education have been considered critical indicators [1]. Since the start of the COVID-19 pandemic, university and college teaching has shifted to online platforms. These online platforms did provide an alternative solution to sustain teaching when offline classroom teaching was not feasible. However, for students, in particular, the transformation to online teaching could not have been easy. Access to laptops and internet connections for online learning became a challenge for students belonging to developing countries. The resource scarcity coupled with an uncertain, life-threatening COVID-19 pandemic can create stress in young minds. Social scientists have rightly expressed concerns on the negative impact of the pandemic of COVID-19 on the education system as one of the biggest challenges faced by the educational system in recent times [2]. Under these circumstances, students pursuing higher education had to face multiple challenges that can ultimately affect any nation’s sustainable growth and development.
The lack of physical, experiential learning due to social distancing, repeated lockdowns, and quarantine has created havoc on the students’ mental health, leading to increased stress, anxiety, depression, and pressure to perform with limited resources. The quality of education being imparted with restricted resources has led to disengagement and lowered motivation in the student population. It has mainly created upheaval in the lives of students who could not adapt quickly to the rapidly changing environment. The stress resulting due to COVID-19 requires different coping mechanisms. Thus, the future workforce responsible for the country’s sustainable growth is at risk. A psychologically unhealthy emerging workforce cannot lay the foundation for a sustainable and healthy future. The chronic stress resulting from the pandemic can be managed at an initial stage through screening and reduce the risk of jeopardizing the country’s sustainable development. Thus, there is a need to screen out such a population. An intervention at an early stage can reduce the COVID-19 stress on the students. For any country, a stress-free and mentally fit young population is essential for the growth and development of the nation. The aim of achieving sustainability through sustainable education can only be achieved if the youth can develop the skills of adaptation and flexibility.
Sustainable education is the pathway to attain and sustain socioeconomic, ecological, and holistic wellbeing at an individual and community level. It implies the kind of education that equips the learner to be open and adaptable to the new changing environments of uncertainty and unpredictability prevalent in the current everyday scenario. Education should foster the qualities of flexibility, originality, self-reliance, optimism, and resilience in students. Education inculcates qualities of authenticity, care for community, meaningful engagement, and purposefulness in the learners. Education also empowers people to have sustainable livelihoods. Thus, sustainable education can only lead to sustainable growth and development of the country.
When the COVID-19 pandemic was emerging, the general thought process was that the elderly and the immunocompromised population were more vulnerable to the pandemic. However, as the pandemic situation evolved, it adversely affected the young population also. The emotional trauma due to the prevailing stressful situation adversely affects individuals. Any mental burden that an individual carries affects the performance of the individual, whether at school, college, or the workplace. The synergy between individuals enhances the output of any system, whether it is academics or industry. COVID-19 can affect sustainable development psychologically. Therefore, the young individual who is concerned about getting infected by the virus or cannot adapt to the rapidly changing academic environment during the pandemic cannot be expected to deliver according to his or her potential fully. The industrial output and economic prosperity of nations depend upon the collective performance of the young population. Thus, the adverse impact of stress on the wellbeing of individuals can put severe doubts on maintaining the sustainable growth of nations. It can be particularly more concerning for developing nations where economic and healthcare resources are scarce.
Researchers have reported the prevalence of stress among the normal population during the current COVID-19 pandemic [3,4]. The prevalence rates for stress due to the pandemic in the general populace were 29.6% [5]. However, the students have been experiencing a significantly high level of stress than the other people, as reported by researchers who compared them to health professionals, academicians, mental health professionals, corporate workers [6]. Furthermore, pandemics lead to psychological reactions, including maladaptive behaviors, emotional distress, suicidal thoughts, and defensive responses [7,8,9,10,11]. These findings are corroborated by studies showing that the level of depression, anxiety, and stress was significantly more in young people (21–40 years old) during the pandemic. The plausible reason for this seems to be that this age group is experiencing more financial challenges and adverse consequences due to the pandemic. As they belong to society’s workforce, they must bear the brunt of laying off and closing businesses [12,13]. In addition, some researchers have attributed the higher level of anxiety among youth due to their more access to information through social media, leading to stress [14,15,16]. In other words, the infodemic can be one of the reasons for stress during the times of pandemic in the youth.
Therefore, researchers further attempted to study the influence of the pandemic on the psychological health of college-going students studying in a public university in the United States. The findings revealed an increase in anxiety and stress in 71% of students due to the pandemic. The multiple stressors faced by the students were apprehension and fear about their own and loved ones’ health, poor concentration, disruptive sleep and eating patterns, and concern for academic performance [17,18].
The sudden change to the online mode of education and evaluation created uncertainty and distress in their academic lives. The students aged between 17 and 22 years (college-going) have expressed more negative reviews of online classes as compared to the younger age group of 7–17 years (school-going) [19]. Online classes have been a challenge for students even during normal times [20]. The lack of face-to-face in-person communication, which gave a feeling of connection and enhanced peer-to-peer learning experiences, became a thing of the past, and many suffered from separation anxiety. During the pandemic, measures like social distancing and quarantine to prevent contamination increased isolation and further added to the existing sources of pressure and stress for students. Studies have reported that self-isolation due to COVID-19 resulted in stress in university students [21]. The stress took many forms in the life of the students impacting their motivation, concentration, and zeal to perform to their actual potential.
In the Indian scenario, studies have also reported the students’ vulnerability towards poor mental health due to fear and the feeling of uncertainty about their studies and future careers [22,23,24]. For example, a study on schools, colleges, and universities reported that due to the pandemic, students had issues with sleeping behaviors, daily workout habits, and social communication, which significantly affected their health status. Besides, 51.4% could not constructively utilize their time [19]. Thus, it implies a feeling of dissatisfaction and frustration of not working to their full capacities. Furthermore, the lockdowns during the pandemic resulted in increased anxiety and depression among university undergraduates [25,26], and the isolation guidelines have adversely affected their mental health to a great extent [27]. In addition to these, a study on college students pointed out that academic stress, separation from peers and campus environment, and fear of contagion were the major sources of stress and had adverse effects on college students’ physical and mental health via perceived stress [28]. In a study conducted on university students in Saudi Arabia, it was revealed that rather than the distress of being alone, isolated, or separated from others due to the COVID-19, students were concerned about becoming a burden on their family in case they got infected [29]. Still, others have reported increased stress levels after one year of COVID-19 in university students of a Spanish university [30].
Most of the students throughout the world did not suffer from symptomatic COVID-19, but the uncertainty and constant fear of getting infected became a source of stress [31]. Students also had to face the loss of experiential learning and practical field experience due to being confined to their homes. In this scenario, there is a vital need for developing the tools to assess the psychosocial burden experienced by them. Only after the assessment can the right kind of intervention be given to the individuals [32,33]. The initial instruments used to assess COVID-19 stress were modified scales derived from existing perceived stress assessment measures, e.g., PSS-10-C [34] COVID-19 Peritraumatic Distress Index [35] COVID-19-related psychological distress in healthy public (CORPD) [36]. Then, the COVID-19 stress scale was developed [37,38]. The majority of the instruments available to measure COVID-19 pandemic stress targeted the general adult population. However, the seven-item COVID-19 Student Stress Scale (CSSQ) [38] constructed to measure pandemic stress on Italian students focused on only the three sources of stress based on the transactional stress model of Lazarus and Folkman [39], namely, relationship, isolation, and fear of contagion mainly due to disruption caused by lockdowns.
Nevertheless, since the start of the COVID-19 pandemic, we have seen that stress sources during the current pandemic are multifaceted for students, particularly in developing countries where access to the internet and technological devices are limited. Further, psychophysiological and social changes are equally relevant stressors during a pandemic [40,41]. Therefore, an instrument is needed to assess the psychophysiological and social sources of stressors among the students so that timely intervention can be given to them. Thus, the objective of the current study was to construct and validate a self-report tool to measure the source of COVID-19 pandemic stress among students of higher educational institutions.

2. Materials and Methods

2.1. Sample and Data Collection Procedures

Data were collected from college students from India between June 2020 and January 2021 using online google forms. The data collection was done in two phases. The inclusion criteria in the study were students 18 years and above studying various courses in Indian colleges and universities. The first phase of data collection was completed in August 2020. The second phase of data collection was from October 2020 to January 2021. Incomplete forms were discarded. The final sample comprised 389 participants, 173 participants in the first and 216 in the second phase. The participants were studying in different Universities and colleges spread across the country. The data was collected from both undergraduate and postgraduate students.
A purposive sampling procedure using the snowball technique was used to collect data. The phase 1 data comprised 108 females and 65 males within 18 to 24 years with a mean age of 21.71 (SD = 1.33) years. For a representative sample, students belonging to different academic domains have been included. The sample comprised of Engineering (34.6%), Arts (16.2%), Science (17.9%), Commerce (5.2%), Management (5.8%), Medical (3.5%), Law (8.1 %), Designing (6.9%), Journalism (1.2%), and Pharmacy (.06%). For the second phase, 216 participants with a mean age of 21.95 (SD = 1.42) years comprising of 106 males (49.1%) and 110 females (50.9%) formed the sample. The age range was 18 to 24 years. The sample comprised of Engineering (36.6%), Arts (16.1%), Science (13%), Commerce (4.2%), Medicine (4.2%), Management (11.1%), Law (4.6 %), Designing (4.2%), Journalism (2.3%), Pharmacy (1.4%), and Others (2.3%). Other categories comprised of students pursuing various diploma courses. The data was collected online, using google forms. The online google form was circulated using Email, WhatsApp, and Facebook. Participating in the study was voluntary. The participants were informed that by filling the form, they give consent to participate in the study.

2.2. Tool for Validation

To evaluate the convergent validity of the current scale, the authors’ used the 14 items stress subscale of DASS-42 [42]. DASS-42 is a well-established and standardized screening tool that has been used on normal adults to discriminate between the three negative conditions of stress, depression, and anxiety. The items on the three subscales are scored on a four-point rating scale ranging from 0 to 3, with a higher score indicating a higher negative emotional state of the individual. The 14 items of the stress subscale assess nervous strain, inability to relax, and irritability in an individual. The Cronbach alpha for the scale was 0.89.

2.3. Candidate SCSS Items

The initial items of the Student COVID-19 Stress Scale (SCSS) was constructed after reviewing the existing literature [14,16,17,18,21,27,37,39,43,44,45,46,47,48] and consulting experts on stress. Initially, a pool of forty items was formed. The student COVID stress scale items belonged to health concerns, fear of death, fear of losing loved ones, lifestyle changes, academic distress, fear of lockdowns, somatic symptoms, social isolation, uncertainty, xenophobia, and future concerns. Three items from the initial item pool were culled on a rational basis (see Supplementary Material). Thus, 37 items were retained for the study.
As a pilot study, the thirty-seven items were administered to 15 college students for checking the language and understanding. The items were in simple words, without jargon, so that students, in general, could easily comprehend them. Instructions for the study mentioned, “The questions ask you about your thoughts and feelings during the last month, particularly concerning the prevailing COVID-19 situation. In each case, you have to respond to how often you felt in a certain way. Please answer all of the questions the best that you can. There are no right or wrong answers. Answer 1 for never, 2 for almost never, 3 for sometimes, 4 for fairly often, and 5 for very often, respectively.” The results indicated that the college students could understand and perform the questionnaire successfully. Thus, the thirty-seven items formed the preliminary scale for the phase one study.

2.4. Data Analysis Procedure

The goal was to construct a short, internally consistent, and valid scale. The initial scale consisted of thirty-seven items. Construct validity of the scale was assessed using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Preliminary screening of the thirty-seven items for EFA was done by assessing item-total correlation. Items with a value greater than 0.30 are considered related to the overall scale [49]. The data was also assessed for normality using the Shapiro Wilk test. Shapiro Wilk values with p > 0.05 were considered normal distribution indicators [50]. EFA on the thirty-seven items was analyzed using Principal component analysis with oblique rotation. SPSS version 21 was used for the analysis. The sampling adequacy was assessed by KMO measure and Barlett test of sphericity. The criteria for extraction and interpretation of components were eigenvalues > 1.0, communality > 0.50 for each item, factor loading > 0.50. Items with a factor loading of 0.50 have been considered enough to be “strong” [51]. Further, items loading at more than 0.40 on two factors were not retained [52]. For assessing the internal consistency of the instrument and subfactors, Cronbach alpha (α) was assessed. α > 0.70 have been considered acceptable [53].
CFA was performed using Amos version 26 to validate the factor structure created using EFA. Before performing the CFA, data screening for normality and outliers was performed. Inter-correlations assessed the preliminary association between SCSS total and its factors. Data screening of the data collected in the second phase identified 09 extreme scores. Excluding these scores left a total sample of 216. Kurtosis and skewness scores for all factors and total score fell within −2 and +2. The kurtosis coefficient of 20.423 with a critical ratio of 7.599 signified multivariate non-normality [54]. Multivariate non-normal data can result in standard error biases [55]. Therefore, Maximum Likelihood (ML) estimation with bootstrapping (1000 resamples) was performed to generate accurate estimations of standard errors with accompanying confidence intervals (bias-corrected at the 95% confidence level) and p-values [56,57]. Thus, to assess whether the drawn-out model from the data provides an excellent fit, empirically derived cut-off values by Hu and Bentler [58,59] were used. An excellent model fit is indicated by SRMR ≤ 0.08, RMSEA ≤ 0.06, and CFI ≥ 0.95. A Good fit is indicated by GFI ≥ 0.90. For RMSEA, the 90% confidence interval (CI) was included. Simultaneously, χ2 and the Bollen–Stine bootstrap p [60] were assessed. χ2 is sensitive to sample size and, therefore, not a sufficient standalone criterion as a CFA index [61]. Bollen–Stine measures fit without normal theory limitations [60], and p > 0.05 signifies excellent global fit. Alternate factor structures were also created to assess the suitability and validity of the default model explored through EFA. Additionally, Akaike’s Information Criterion (AIC) was assessed to make model comparisons. Lower values signify superior fit [62].
Composite reliability (CR) was examined to assess the internal consistency of scale following the model specification. CR above 0.70 was considered the criteria for internal consistency [63]. The convergent validity of SCSS was analyzed by calculating standardized factor loadings, CR, and AVE of factors [64,65]. Moreover, convergent validity was also assessed by correlational analyses between the scales scores of SCSS and the Stress scale [42,66]. Further, the Concurrent predictive validity of the scale was assessed using ROC curves. The stress scale was used to identify cases who scored high on the stress index. A score of nineteen or above on the stress scale indicates moderate to severe stress [42]. During a pandemic, it is natural for individuals to be slightly more stressed than during normal circumstances. Therefore, mild stress cases were treated with normal cases. Those who scored nineteen or above on the stress scale were categorized as ‘stressed’ for plotting the ROC curves. The area under the curve (AUC) of the ROC curve represents the SCSS’s ability to differentiate between stressed and those who are not. The possible values of AUC range from 0.5 to 1.0. An AUC > 0.5 is considered acceptable. Further, an AUC value between 0.7 and 0.9 indicates moderate accuracy. AUC above 0.9 indicates high accuracy [67]. The Heterotrait-Monotrait (HTMT) ratio of correlation [68] was assessed to measure the discriminant validity of the factor structure. Values close to 1 indicate a lack of discriminant validity. A value higher than the threshold of 0.85 was considered an indicator of poor discriminant validity [69].

3. Results

3.1. Exploratory Factor Analysis

The preliminary assessment of the item total correlation on the thirty-seven-item scale revealed that all the items were above the required threshold of 0.30. Therefore, the Kaiser-Meyer-Olkin (KMO) test and Bartlett sphericity test were conducted on the thirty-seven-item scale. The KMO value was 0.881, and the Barlett Chi-square approximation was 2857.026, p < 0.001. Thus, the values indicated significant sampling adequacy for performing EFA [70,71].
Exploratory Factor analysis using principal component analysis with oblique rotation was performed on the thirty-seven items to assess the underlying factor structure of SCSS. A succession of factor analyses yielded a five-factor solution with an eigenvalue of more than 1 [70]. All the retained items had factor loadings of 0.70 and above [65]. Cross-loading items (with factor loadings above 0.40) on multiple factors were dropped [52]. All the retained thirteen items had a communality score above 0.50. Thus, the five factors with eigenvalues of more than one together explained 73.83 percent of the total variance indicating acceptable construct validity for the scale. The first component, called social isolation with an eigenvalue of 4.07, had three items. The second and third components, with eigenvalues of 1.77 and 1.41, included three and two items. They were called sleep-somatic problems and contamination fears, respectively. The fourth and fifth components called academic distress and career apprehension, with eigenvalue of 1.29 and 1.04, had three and two items. Table 1 presents the factor loadings, communalities, eigenvalue, variance explained, and Cronbach alpha. The Cronbach alpha for 13 items SCSS was 0.82. Similarly, all the five factors had Cronbach alpha above 0.70, indicating acceptable reliability for the EFA model [53,72].

3.2. Confirmatory Factor Analysis

Table 2 presents the descriptive statistics and intercorrelations for SCSS Total and subscales.
Due to the multivariate non-normal nature of the data, the five-factor structure obtained in EFA was tested using confirmatory factor analysis with the maximum likelihood method (with bootstrapping, 1000 resamples). The correlated five-factor default model derived in the EFA demonstrated a good fit on all the required indices; SRMR = 0.037, RMSEA = 0.049 (CI of 0.041–0.078), CFI = 0.98, Bollen–Stine p = 0.093. Bollen–Stine measures fit without normal theory limitations [60], and p > 0.05 suggests excellent global fit. However, a second-order analysis was conducted to assess if the five factors belonged to a single broader latent factor of Student COVID-19 Stress. The resulting alternate model indicated fit indices of SRMR = 0.063, RMSEA = 0.067 (CI of 0.049–0.084), CFI = 0.96, Bollen–Stine p = 0.007, resulting in poor fit [60]. Another four-factor model was also tested for model fit as the correlational analysis indicated a high correlation between factor two and factor three [66]. Fit indices for the four-factor model indicated SRMR = 0.041, RMSEA = 0.057 (CI of 0–037–0.076), CFI = 0.972; however, Bollen–Stine p = 0.007 indicated poor fit [60]. Further, the lower AIC value of the default five-factor model also indicated superior fit in comparison to other models [62]. Therefore, the five-factor default model was retained as it had indicated it had acceptable construct validity [58,59,60]. Table 3 demonstrates the factor loading for all thirteen items. Thus, all the factor loadings were above 0.50 and significant (p < 0.01). Table 4 presents the fit indices of the alternate stress models.

3.3. Assessment of Reliability

Based on the results of the factor analysis, five scales were constructed to form the SCSS. Table 5 presents composite reliability coefficients for each scale. Moreover, the CR values for all the factors were >0.7 (i.e., Social Isolation CR = 0.81; Sleep and somatic Problems CR = 0.71; Contamination fear CR: 0.82; Academic distress CR: 0.82; Career Apprehensions CR: 0.91). These values indicated acceptable reliability of the five-factor model [63]. Similarly, the Cronbach alpha for the SCSS was also 0.89, indicating good to excellent reliability [73].

3.4. Assessment of Validity

The standardized factor loadings of SCSS items were all >0.5 (see Figure 1) and statistically significant (p < 0.001). The values of AVE of all factors, i.e., Social Isolation = 0.65; Sleep and Somatic Problems = 0.50; Contamination Fear = 0.68; Academic Distress: 0.68 and career Apprehensions = 0.83 were in acceptable limits [64,74]. Therefore, the standardized factor loadings, CR, and AVE of factors together suggest that the SCSS has convergent validity [63].
Further, Table 5 demonstrates the correlation between the Stress scale and the SCSS total and sub-dimensions. The low to moderate correlation with the Stress scale further collaborated convergent validity for SCSS [66]. Regarding the concurrent predictive validity of the SCSS, the ROC curve indicated that SCSS could differentiate those stressed from those who are not stressed. Figure 2 presents the ROC curve and AUC for the 13-item SCSS. The AUC for the measure was acceptable [67] at a value of 0.779, p < 0.001, 95% CI [0.708, 0.850].
For assessing the discriminant validity of the factor structure, the Heterotrait-Monotrait (HTMT) ratio of correlation has been assessed. HTMT ratio of correlation values <0.85, establish discriminant validity for the constructs [69]. Table 6 demonstrates that all the HTMT values were <0.85, establishing the discriminant validity for the five-factor model.

4. Discussion

The purpose of the present study was to construct, validate, and evaluate the psychometric properties of a brief scale to assess the source of COVID-19 pandemic stress on students pursuing higher education in universities and colleges of India. The Exploratory factor analysis derived a five-factor solution with 13 items. The factor structure was validated using Confirmatory Factor Analysis. The analysis indicated adequate construct validity for the Student COVID-19 Stress scale (SCSS). The first factor, labeled social isolation, consisted of items covering an individual’s inability to socialize, travel, and interact with friends and family. Measures like frequent lockdowns, mandatory quarantines, and social distancing have restricted students’ movement, leading to increased social isolation among them. Research studies have indicated that social isolation has increased stress among students [20,21].
The second factor, sleep-somatic problems, comprises items related to sleep disturbances, somatic symptoms, and nightmares due to the pandemic environment. Often psychological stress is manifested through sleep disturbances and psychosomatic symptoms. The findings are similar to earlier research, which indicated that students reported sleep disturbances due to stress [18]. Sleep disturbance can have a negative impact on student’s daily activities and impact their performance. The third factor, labeled contamination fear, is related to perceived stress due to the fear of getting infected and the stress produced by the constant barrage of COVID-19 related news on various platforms. Fear of contagion has been considered a major source of stress that negatively affects college students’ physical and mental health [23,24,28]. Due to this, students had to restrict their movements. These restrictions contributed to increased isolation as they became hesitant to meet their friends and relatives. In addition, the constant news reports about the adverse impact of COVID-19 around the world on various platforms also might have resulted in increased anxiety. The curiosity coupled with an inability to restrict information flow may create a psychological burden in students. This psychological burden often manifests in the form of stress.
The fourth component, labeled academic distress, consisted of items related to online assignment submissions, deadlines, and excess workload in the current academic environment. Most teaching has transferred to online modes. Research has demonstrated that transformation from face-to-face teaching to an online classroom has not been easy for students. Further, the uncertainty about studies and career have contributed to enhanced stress. The adaptation process seems quite stressful for the students [19]. The fifth component, labeled academic distress, comprises items related to the perceived impact of the pandemic on the Career prospects and future of students. The uncertainty and fear during the pandemic have cast doubt in the minds of students. In addition, due to the restrictions imposed on various academic and economic activities, students feel concerned about their studies and prospects getting adversely impacted and limited [23].
The standardized factor loadings, CR, and AVE of factors taken together demonstrate that the SCSS (See Appendix A.1) has adequate convergent validity [63]. Similarly, the correlation between the Stress Scale and the SCSS (and its dimensions) validated the existence of convergent validity [64,74]. Furthermore, the concurrent predictive validity of the scale assessed through ROC curves demonstrated that SCSS could successfully discriminate between stressed and those who are not. Thus, indicating concurrent predictive validity of the scale [67]. In addition, the HTMT ratio of correlation was assessed to assess the discriminant validity of the factor structure. As a result, the HTMT values indicate the discriminant validity of the factor structure [69]. Similarly, composite reliability values demonstrated the internal consistency of SCSS [53]. Thus, the psychometric attributes indicate SCSS as a valid and reliable instrument to assess COVID-19 pandemic stress among college students.

5. Limitations

The scale was administered online, thereby reducing the participation of those without the internet. Secondly, as self-report instruments were used, social desirability bias affecting the findings could not be ruled out. Cronbach alpha and composite reliability were assessed, but due to the urgency of the need for a scale like SCSS, test-retest reliability of the instrument could not be assessed. Further, India is a diverse nation, and data were collected from different parts of the country. Thereby, the potential of social and cultural variables affecting the construct cannot be omitted. Lastly, a relatively larger sample comprising data from different nations would be needed to confirm the findings.

6. Conclusions and Future Implications

Student COVID-19 stress scale (SCSS) is a short, easy to administer, valid, and reliable instrument to assess COVID-19 pandemic stress among college students. The instrument can be a helpful tool to prepare and administer tailor-made solutions to students to prevent adverse long-term psychological consequences arising due to pandemic stress. The five factors of the scale, namely, social isolation, sleep-somatic problems, contamination fear, academic distress, and career apprehension, will enable us to better understand and explore the sources of stress due to the pandemic among college students. The mutating coronavirus has created an environment of uncertainty and continues to be a source of stress and anxiety for the global community. The chances of the pandemic becoming an endemic seem quite high. This concern was recently highlighted in a survey where 89% of the researchers (immunologists) expressed that the current COVID-19 pandemic can become an endemic [75]. Thus, it seems the intensity of the pandemic may reduce [76], but it may be difficult to eradicate it. However, it has been noticed that when the COVID-19 cases go down, educational institutions reopen for face-to-face teaching. The same had happened in India, where face-to-face teaching was reinitiated a few months ago as there was a decline in COVID-19 cases. However, just after a month, the lockdown was imposed again due to a surge in positive cases. Thus, students had to return to online teaching again. The second wave of COVID-19 reignited the fear in all. Thus, even today, the fear of contagion may have slightly reduced, but it has not entirely disappeared. The threat of a surge in cases due to mutated strains like Delta, Delta plus, and Kappa is still prevalent. Students are still apprehensive due to the threat of COVID-19. Despite the development of vaccines to prevent COVID-19, their availability in developing countries has been a matter of concern. Thus, due to the prevailing circumstances, the necessity of a scale to assess sources of COVID-19 stress among the students holds significant value.
Thus, focus on students’ mental health is of prime importance during times of pandemic, as stress can create long-term adverse effects on students’ minds, which can adversely alter their current and future performance. Moreover, as the pandemic is constantly evolving, instruments like SCSS can efficiently help screen students to deliver timely psychological interventions to the needful, particularly in developing nations. Thus, for achieving sustainable growth and development, timely interventions through instruments like SCSS in higher educational institutions will be productive for building sustainable education by nurturing healthy individuals who can contribute to national growth. Moreover, scales like SCSS can also help guide the development of psychological instruments to deal with future pandemics. Currently, the scale has been developed for assessing COVID-19 stress in college students. However, it can also be modified and adapted to assess stress occurring due to any pandemic of similar nature in the future.

Supplementary Materials

The following are available online at https://0-www-mdpi-com.brum.beds.ac.uk/article/10.3390/su13179828/s1, Figure S1: Scree plot output indicating that the data have five factors, Figure S2. Model diagram for the second-order analysis, Figure S3. Model diagram for the four-factor model, Table S1. Descriptive Statistic and Inter-Item Correlation Matrix, Table S2. Item-Total Statistics, Table S3: Total Variance Explained, Table S4. Descriptive Statistics and Bivariate Correlations between the five factors, Table S5. Test of Normality: Shapiro-Wilk, Table S6. CFA: Assessment of normality, Table S7. Descriptive Statistics and Inter-Item Correlations for the five-factor model, Table S8. Bootstrapped Standard Errors, Table S9. Squared multiple correlations, Table S10. T-test comparison between genders, Table S11: Bivariate Correlation between the Stress Scale and SCSS, Table S12. Area Under the Curve: SCSS, Table S13. Sensitivity and Specificity of the Scale, Table S14. Student COVID 19 Stress Scale (Initial Items).

Author Contributions

Conceptualization, V.V.K. and B.C.; methodology, V.V.K. and B.C.; formal analysis, V.V.K. and G.T.; writing—original draft preparation, V.V.K. and B.C.; writing—review and editing, V.V.K. and G.T.; supervision, V.V.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data available upon request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Appendix A.1. Student COVID-19 Stress Scale (SCSS)

The questions ask you about your thoughts and feelings during the last month, particularly concerning the prevailing COVID-19 situation. In each case, you have to respond to how often you felt in a certain way. Please answer all of the questions the best that you can. There are no right or wrong answers. Answer 1, for never, 2 for almost never, 3 for sometimes, 4 for fairly often, and 5 for very often, respectively.
Table A1. Student COVID-19 Stress Scale (SCSS).
Table A1. Student COVID-19 Stress Scale (SCSS).
ItemsNever
1
Almost Never
2
Sometimes
3
Fairly Often
4
Very Often
5
SI1How often have you felt irritated for not being able to go out with your friends or family?
SI2How often have you felt frustrated for not being able to meet friends?
SI3How often have you become irritated because of social distancing maintained due to COVID-19?
SSP1How often have you felt sleep disturbances?
SSP2How often have you complained about having headaches, body pain, and feeling of faintness or dizziness?
SSP3How often you got disturbing nightmares or dreams?
CF1How often have you got nervous about getting infected while thinking of going out?
CF2How often have you got worried after getting news and updates regarding the pandemic (COVID-19)?
AD1How often have you felt nervous about not being able to meet the deadline for submissions?
AD2How often have you felt stressed out with online task submissions?
AD3How often have you felt stressed due to the excess load of task completion?
CA1How often have you been upset while thinking about your future life?
CA2How often have you been worried about the impact of the pandemic on your career prospects?

References

  1. Wong, C. Sustainable Development Indicators. In Encyclopedia of Quality of Life and Well-Being Research; Michalos, A.C., Ed.; Springer: Dordrecht, The Netherlands, 2014. [Google Scholar]
  2. Daniel, S.J. Education and the COVID 19 pandemic. Prospects 2020, 49, 91–96. [Google Scholar] [CrossRef] [Green Version]
  3. Li, X.; Lyu, H. Epidemic Risk Perception, Perceived Stress, and Mental Health during COVID 19 Pandemic: A Moderated Mediating Model. Front. Psychol. 2021, 11, 563741. [Google Scholar] [CrossRef]
  4. Uehara, M.; Fujii, M.; Kobayashi, K. A Model of Stress Change under the First COVID 19 Pandemic among the General Public in Japanese Major Cities and Rural Areas. Sustainability 2021, 13, 1207. [Google Scholar] [CrossRef]
  5. Salari, N.; Hosseinian-Far, A.; Jalali, R.; Vaisi-Raygani, A.; Rasoulpoor, S.; Mohammadi, M.; Rasoulpoor, S.; Khaledi-Paveh, B. Prevalence of stress, anxiety, depression among the general population during the COVID 19 pandemic: A systematic review and meta-analysis. Global. Health 2020, 16, 57. [Google Scholar] [CrossRef] [PubMed]
  6. Rehman, U.; Shahnawaz, M.G.; Khan, N.H.; Kharshiing, K.D.; Khursheed, M.; Gupta, K.; Kashyap, D.; Uniyal, R. Depression, anxiety and stress among indians in times of COVID 19 lockdown. Community Ment. Health J. 2021, 57, 42–48. [Google Scholar] [CrossRef] [PubMed]
  7. Taylor, S. The Psychology of Pandemics: Preparing for the Next Global Outbreak of Infectious Disease; Cambridge Scholars Publishing: Newcastle upon Tyne, UK, 2019. [Google Scholar]
  8. Cullen, W.; Gulati, G.; Kelly, B.D. Mental health in the COVID 19 pandemic. QJM 2020, 113, 311–312. [Google Scholar] [CrossRef]
  9. Wang, C.; Pan, R.; Wan, X.; Tan, Y.; Xu, L.; Ho, C.S.; Ho, R.C. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID 19) epidemic among the general population in China. Int. J. Environ. Res. Public Health 2020, 17, 1729. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Gao, W.; Ping, S.; Liu, X. Gender differences in depression, anxiety, and stress among college Students: A Longitudinal Study from China. J. Affect. Disord. 2020, 263, 292–300. [Google Scholar] [CrossRef]
  11. Huang, Y.; Zhao, N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID 19 outbreak in China: A web-based cross-sectional survey. Psychiatry Res. 2020, 288, 112954. [Google Scholar] [CrossRef]
  12. Moghanibashi-Mansourieh, A. Assessing the anxiety level of Iranian general population during COVID 19 outbreak. Asian J. Psychiatry 2020, 51, 102076. [Google Scholar] [CrossRef]
  13. Ahmed, M.Z.; Ahmed, O.; Aibao, Z.; Hanbin, S.; Siyu, L.; Ahmad, A. Epidemic of COVID 19 in China and associated psychological problems. Asian J. Psychiatry 2020, 51, 102092. [Google Scholar] [CrossRef]
  14. Campisi, J.; May, J.; Burch, K.; Larson, K.; Doscher, J.; Doherty, S.; Isaacson, K.; Sebring, K.; Gahan, A. Anxiety-inducing Facebook behavior is associated with higher rates of upper respiratory infection in college-aged users. Comput. Hum. Behav. 2017, 76, 211–217. [Google Scholar] [CrossRef]
  15. Cheng, C.; Jun, H.; Liang, B. Psychological health diathesis assessment system: A nationwide survey of resilient trait scale for Chinese adults. Stud. Psychol. Behav. 2014, 12, 735–742. [Google Scholar]
  16. Siegel, D.M.; Burke, T.A.; Hamilton, J.L.; Piccirillo, M.L.; Scharff, A.; Alloy, L.B. Social anxiety and interpersonal stress generation: The moderating role of interpersonal distress. Anxiety Stress Coping 2018, 31, 526–538. [Google Scholar] [CrossRef] [PubMed]
  17. Hurst, C.S.; Baranik, L.E.; Daniel, F. College student stressors: A review of the qualitative research. Work Stress 2013, 29, 275–285. [Google Scholar] [CrossRef]
  18. Son, C.; Hegde, S.; Smith, A.; Wang, X.; Sasangohar, F. Effects of COVID 19 on college students’ mental health in the United States: Interview survey study. J. Med. Internet Res. 2020, 22, e21279. [Google Scholar] [CrossRef]
  19. Chaturvedi, K.; Vishwakarma, D.K.; Singh, N. COVID 19 and its impact on education, social life and mental health of students: A survey. Child. Youth Serv. Rev. 2021, 121. [Google Scholar] [CrossRef] [PubMed]
  20. Adams, A.; Randall, S.; Traustadóttir, T. A tale of two sections: An experiment to compare the effectiveness of a hybrid versus a traditional lecture format in introductory microbiology. CBE Life Sci. Educ. 2015, 14, ar6. [Google Scholar] [CrossRef] [Green Version]
  21. Wang, C.; Zhao, H. The impact of COVID 19 on anxiety in Chinese university students. Front. Psychol. 2020, 11, 1168. [Google Scholar] [CrossRef]
  22. Chhetri, B.; Goyal, L.M.; Mittal, M.; Battineni, G. Estimating the prevalence of stress among Indian students during the COVID 19 pandemic: A cross-sectional study from India. J. Taibah Univ. Med. Sci. 2021, 16, 260–267. [Google Scholar] [CrossRef] [PubMed]
  23. Kapasia, N.; Paul, P.; Roy, A.; Saha, J.; Zaveri, A.; Mallick, R.; Barman, B.; Das, P.; Chouhan, P. Impact of lockdown on learning status of undergraduate and postgraduate students during COVID 19 pandemic in West Bengal, India. Child. Youth Serv. Rev. 2020, 116, 105194. [Google Scholar] [CrossRef] [PubMed]
  24. Sahu, P. Closure of universities due to coronavirus disease 2019 (COVID 19): Impact on education and mental health of students and academic staff. Cureus 2020, 12, e7541. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Mazza, C.; Ricci, E.; Biondi, S.; Colasanti, M.; Ferracuti, S.; Napoli, C.; Roma, P. A nationwide survey of psychological distress among Italian people during the COVID 19 pandemic: Immediate psychological responses and associated factors. Int. J. Environ. Res. Public Health 2020, 17, 3165. [Google Scholar] [CrossRef] [PubMed]
  26. Santini, Z.I.; Jose, P.E.; Cornwell, E.Y.; Koyanagi, A.; Nielsen, L.; Hinrichsen, C.; Meilstrup, C.; Madsen, K.R.; Koushede, V. Social disconnectedness, perceived isolation, and symptoms of depression and anxiety among older Americans (NSHAP): A longitudinal mediation analysis. Lancet Public Health 2020, 5, e62–e70. [Google Scholar] [CrossRef] [Green Version]
  27. Chen, B.; Sun, J.; Feng, Y. How have COVID 19 isolation policies affected young people’s mental health? Evidence from Chinese college students. Front. Psychol. 2020, 11, 1529. [Google Scholar] [CrossRef] [PubMed]
  28. Yang, C.; Chen, A.; Chen, Y. College students’ stress and health in the COVID 19, pandemic: The role of academic workload, separation from school, and fears of contagion. PLoS ONE 2021, 16, e0246676. [Google Scholar]
  29. Alghamdi, A.A. Impact of the COVID 19 pandemic on the social and educational aspects of Saudi university students’ lives. PLoS ONE 2021, 16, e0250026. [Google Scholar] [CrossRef]
  30. Marcén-Román, Y.; Gasch-Gallen, A.; de la Martín, V.; Mota, I.I.; Calatayud, E.; Gómez-Soria, I.; Rodríguez-Roca, B. Stress perceived by university health sciences students, 1 Year after COVID 19 Pandemic. Int. J. Environ. Res. Public Health 2021, 18, 5233. [Google Scholar] [CrossRef]
  31. Campo-Arias, A.; Pedrozo-Cortés, M.J.; Pedrozo-Pupo, J.C. Pandemic-Related Perceived Stress Scale of COVID 19: An exploration of online psychometric performance [Escala de estrés percibido relacionado con la pandemia de COVID 19: Una exploración del desempeño psicométrico en línea]. Rev. Colomb. Psiquiatr. 2020, 49, 229–230. [Google Scholar] [CrossRef] [PubMed]
  32. Cortez, P.A.; Joseph, S.J.; Das, N.; Bhandari, S.S.; Shoib, S. Tools to measure the psychological impact of the COVID 19 pandemic: What do we have in the platter? Asian J. Psychiatry 2020, 53, 102371. [Google Scholar] [CrossRef]
  33. Das, N.; Narnoli, S.; Kaur, A.; Sarkar, S. Pandemic, panic, and psychiatrists—What should be done before, during, and after COVID 19? Asian J. Psychiatry 2020, 53, 102206. [Google Scholar] [CrossRef]
  34. Pedrozo-Pupo, J.C.; Pedrozo-Cort´es, M.J.; Campo-Arias, A. Perceived stress associated with COVID 19 epidemic in Colombia: An online survey. Cad. Saúde Pública 2020, 36, e00090520. [Google Scholar] [CrossRef]
  35. Qiu, J.; Shen, B.; Zhao, M.; Wang, Z.; Xie, B.; Xu, Y. A nationwide survey of psychological distress among Chinese people in the COVID 19 epidemic: Implications and policy recommendations. Gen. Psychiatry 2020, 33, e100213. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Feng, L.S.; Dong, Z.J.; Yan, R.Y.; Wu, X.Q.; Zhang, L.; Ma, J.; Zeng, Y. Psychological distress in the shadow of the COVID 19 pandemic: Preliminary development of an assessment scale. Psychiatry Res. 2020, 291, 113202. [Google Scholar] [CrossRef]
  37. Taylor, S.; Landry, C.A.; Paluszek, M.M.; Fergus, T.A.; McKay, D.; Asmundson, G.J.G. Development and initial validation of the COVID Stress Scales. J. Anxiety Disord. 2020, 72, 102232. [Google Scholar] [CrossRef] [PubMed]
  38. Zurlo, M.C.; Cattaneo, D.; Volta, M.F.; Vallone, F. COVID 19 student stress questionnaire: Development and validation of a questionnaire to evaluate students’ stressors related to the coronavirus pandemic lockdown. Front. Psychol. 2020, 11, 576758. [Google Scholar] [CrossRef]
  39. Lazarus, R.S.; Folkman, S. Stress, Appraisal and Coping; Springer: New York, NY, USA, 1984. [Google Scholar]
  40. Batais, M.A.; Temsah, M.H.; AlGhofili, H.; AlRuwayshid, N.; Alsohime, F.; Almigbal, T.H.; Al-Rabiaah, A.; Al-Eyadhy, A.A.; Mujammami, M.H.; Halwani, R.; et al. The coronavirus disease of 2019 pandemic-associated stress among medical students in Middle East Respiratory Syndrome-CoV endemic area: An observational study. Medicine 2021, 100, e23690. [Google Scholar] [CrossRef]
  41. Al-Rabiaah, A.; Temsah, M.H.; Al-Eyadhy, A.A.; Hasan, G.M.; Al-Zamil, F.; Al-Subaie, S.; Somily, A.M. Middle East Respiratory Syndrome-Corona Virus (MERS-CoV) associated stress among medical students at a university teaching hospital in Saudi Arabia. J. Infect. Public Health 2020, 13, 687–691. [Google Scholar] [CrossRef]
  42. Lovibond, P.F.; Lovibond, S.H. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther. 1995, 33, 335–343. [Google Scholar] [CrossRef]
  43. Robertson, E.; Hershenfield, K.; Grace, S.L.; Stewart, D.E. The psychosocial effects of being quarantined following exposure to SARS: A qualitative study of Toronto health care workers. Can. J. Psychiatry 2004, 49, 403–407. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Wong, T.W.; Yau, J.K.; Chan, C.L.; Kwong, R.S.; Ho, S.M.; Lau, C.C.; Lau, F.L.; Lit, C.H. The psychological impact of severe acute respiratory syndrome outbreak on healthcare workers in emergency departments and how they cope. Eur. J. Emerg. Med. 2005, 12, 13–18. [Google Scholar] [CrossRef] [PubMed]
  45. Barker, E.T.; Howard, A.L.; Villemaire-Krajden, R.; Galambos, N.L. The rise and fall of depressive symptoms and academic stress in two samples of university students. J. Youth Adolesc. 2018, 47, 1252–1266. [Google Scholar] [CrossRef] [PubMed]
  46. Karaman, M.A.; Watson, J.C. Examining associations among achievement motivation, locus of control, academic stress, and life satisfaction: A comparison of US and international undergraduate students. Pers. Individ. Differ. 2017, 111, 106–110. [Google Scholar] [CrossRef]
  47. Coiro, M.J.; Bettis, A.H.; Compas, B.E. College students coping with interpersonal stress: Examining a control-based model of coping. J. Am. Coll. Health 2017, 65, 177–186. [Google Scholar] [CrossRef] [PubMed]
  48. Metzger, I.W.; Cooper, S.M.; Ritchwood, T.D.; Onyeuku, C.; Griffin, C.B. Profiles of African American college students’ alcohol use and sexual behaviors: Associations with stress, racial discrimination, and social support. J. Sex Res. 2017, 54, 374–385. [Google Scholar] [CrossRef] [Green Version]
  49. Cristobal, E.; Flavián, C.; Guinaliu, M. Perceived E-Service Quality (PeSQ) measurement validation and effects on consumer satisfaction and web site loyalty. Int. J. Qual. Serv. Sci. 2007, 17, 317–340. [Google Scholar] [CrossRef]
  50. Shapiro, S.S.; Wilk, M.B. An analysis of variance test for normality (Complete Samples). Biometrika 1965, 52, 591–611. [Google Scholar] [CrossRef]
  51. Costello, A.B.; Osborne, J.W. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract. Assess. Res. Eval. 2005, 10, 7. [Google Scholar] [CrossRef]
  52. Schonrock-Adema, J.; Heijne-Penninga, M.; Van Hell, E.A.; Cohen-Schotanus, J. Necessary steps in factor analysis: Enhancing validation studies of educational instruments. Med. Teach. 2009, 31, e226–e232. [Google Scholar] [CrossRef] [Green Version]
  53. Santos, J.R.A. Cronbach’s alpha: A tool for assessing the reliability of scales. J. Ext. 1999, 37, 1–5. [Google Scholar]
  54. Mardia, K.V. Measures of multivariate skewness and kurtosis with applications. Biometrika 1970, 57, 519–530. [Google Scholar] [CrossRef]
  55. Bentler, P.M.; Wu, E.J.C. EQS 6.1 for Windows User’s Guide; Multivariate Software: Encino, CA, USA, 2005. [Google Scholar]
  56. Byrne, B.M. Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming; Routledge: New York, NY, USA, 2010. [Google Scholar]
  57. Nevitt, J.; Hancock, G.R. Performance of bootstrapping approaches to model test statistics and parameter standard error estimation in structural equation modeling. Struct. Equ. Model. 2001, 8, 353–377. [Google Scholar] [CrossRef]
  58. Hu, L.; Bentler, P.M. Fit indices in covariance structure modeling: Sensitivity to under parameterized model misspecification. Psychol. Methods 1998, 3, 424–453. [Google Scholar] [CrossRef]
  59. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ. Model. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  60. Bollen, K.A.; Stine, R.A. Bootstrapping goodness-of-fit measures in structural equation models. Sociol. Methods Res. 1992, 21, 205–229. [Google Scholar] [CrossRef]
  61. Byrne, B.M. Structural Equation Modeling with EQS and EQS/Windows: Basic Concepts, Applications, and Programming; Sage: Thousand Oaks, CA, USA, 1994. [Google Scholar]
  62. Burnham, K.P.; David, R.A. Model Selection and Multimodel Inference: A Practical Information-Theoretical Approach, 2nd ed.; Springer: New York, NY, USA, 2002. [Google Scholar]
  63. Hair, J.F.; Hult, G.T.M.; Ringle, C.M.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), 2nd ed.; Sage Publications: Thousand Oaks, CA, USA, 2016. [Google Scholar]
  64. Fornell, C.; Larcker, D.F. Structural equation models with unobservable variables and measurement error: Algebra and statistics. J. Mark. Res. 1981, 18, 382–388. [Google Scholar] [CrossRef]
  65. Hair, J.F., Jr.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis, 7th ed.; Pearson Education: Edinburgh, UK, 2010. [Google Scholar]
  66. Hinkle, D.E.; Wiersma, W.; Jurs, S.G. Applied Statistics for the Behavioral Sciences, 5th ed.; Houghton Mifflin: Boston, MA, USA, 2003. [Google Scholar]
  67. Fischer, J.E.; Bachmann, L.M.; Jaeschke, R. A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Med. 2003, 29, 1043–1051. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Henseler, J.; Ringle, C.; Sinkovics, R.R. The use of partial least squares path modeling in international marketing. J. Acad. Market. Sci. 2009, 20, 227–319. [Google Scholar]
  69. Kline, R.B. Principles and Practice of Structural Equation Modeling, 3rd ed.; The Guilford Press: New York, NY, USA, 2011. [Google Scholar]
  70. Kaiser, H.F. The varimax criterion for analytic rotation in factor analysis. Psychometrika 1958, 23, 187–200. [Google Scholar] [CrossRef]
  71. Bartlett, M.S. A note on multiplying factors for various chi-squared approximations. J. R. Stat. Soc. 1954, 16, 296–298. [Google Scholar]
  72. DeVellis, R.F. Scale Development: Theory and Applications, 2nd ed.; Sage Publications: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  73. Tavakol, M.; Dennick, R. Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011, 27, 53–55. [Google Scholar] [CrossRef] [PubMed]
  74. Diamantopoulos, A.; Siguaw, J.A. Introducing LISREL: A Guide for the Uninitiated; Sage Publications: London, UK, 2000. [Google Scholar]
  75. Phillips, N. The Coronavirus is Here to Stay—Here’s What That Means. Available online: https://0-www-nature-com.brum.beds.ac.uk/articles/d41586-021-00396-2 (accessed on 19 August 2021).
  76. Torjesen, I. Covid-19 Will Become Endemic but with Decreased Potency over Time, Scientists Believe. Available online: https://0-www-bmj-com.brum.beds.ac.uk/content/372/bmj.n494 (accessed on 19 August 2021).
Figure 1. Model diagram of the five-factor model of the Student COVID Stress Scale.
Figure 1. Model diagram of the five-factor model of the Student COVID Stress Scale.
Sustainability 13 09828 g001
Figure 2. ROC curve for the 13 item SCSS, 0.779, p < 0.001, 95% CI [0.708, 0.850], SE = 0.036.
Figure 2. ROC curve for the 13 item SCSS, 0.779, p < 0.001, 95% CI [0.708, 0.850], SE = 0.036.
Sustainability 13 09828 g002
Table 1. Results of exploratory factor analysis (EFA) on the 13-item Student COVID-19 Stress Scale.
Table 1. Results of exploratory factor analysis (EFA) on the 13-item Student COVID-19 Stress Scale.
Factor/ItemsαF1F2F3F4F5h2
F1: Social Isolation0.83
SI1 0.902−0.0650.0300.007−0.0490.834
SI2 0.900−0.039−0.0800.051−0.1080.795
SI3 0.7660.1460.069−0.0950.1520.690
F2: Sleep and Somatic Problems0.74
SSP1 −0.0440.8540.0460.015−0.0490.746
SSP2 0.0240.796−0.0270.080−0.0910.636
SSP3 0.0340.753−0.004−0.1230.0790.623
F3: Contamination Fear0.76
CF1 −0.0230.0340.9430.0840.0280.843
CF2 0.024−0.0330.812−0.116−0.0850.768
F4: Academic Distress0.75
AD1 −0.069−0.0620.030−0.863−0.0240.711
AD2 0.048−0.014−0.099−0.861−0.0340.736
AD3 0.0740.1620.131−0.6340.0030.595
F5: Career Apprehensions0.72
CA1 −0.0770.144−0.073−0.049−0.9030.856
CA2 0.168−0.0770.188−0.034−0.7540.764
Eigenvalues 4.0791.8431.3381.2721.064
% of variance 31.37614.18010.2949.7888.183
Note: Total variance explained = 73.82%. Values in bold indicate major loadings. h2 is item communality.
Table 2. Descriptive statistics and intercorrelations for SCSS Total and subscales.
Table 2. Descriptive statistics and intercorrelations for SCSS Total and subscales.
Factors/ScaleαMSkewnessKurtosisRangeF1F2F3F4F5
Social Isolation (3)0.839.310.156−0.26512
Sleep-Somatic problems (3)0.756.870.325−0.591100.46
Contamination fear (2)0.815.630.296−0.15080.490.65
Academic Distress (3)0.869.43−0.099−0.019120.360.290.25
Career Apprehensions (2)0.907.06−0.209−0.43580.490.440.500.45
SCSS-130.8938.30 0.780.770.750.660.75
Note: All correlations are significant at p < 0.01; Parenthesis contains the number of items.
Table 3. Results of the CFA for the 13 items Student COVID-19 Stress Scale.
Table 3. Results of the CFA for the 13 items Student COVID-19 Stress Scale.
Unstandardized
95% CI
Standardized
95% CI
ItemsEstimateLowerUpperpEstimateLowerUpperp
SI11.0001.0001.000-0.8810.8190.9330.002
SI21.050.9291.1740.0030.8930.8310.9360.003
SI30.6430.5080.790.0020.610.5040.6940.003
SSP11.0001.0001.000-0.7160.6020.7980.002
SSP20.7770.5930.9860.0030.5580.4230.6690.002
SSP31.1770.9791.4810.0020.8150.7440.8760.003
CF11.0001.0001.000-0.8460.7760.9080.002
CF21.0010.8271.1650.0030.8040.7170.8630.004
AD11.0001.0001.000-0.6990.6010.7920.002
AD21.2461.0471.4950.0030.920.8630.9690.002
AD31.1260.9521.3470.0020.840.7640.8980.003
CA11.0001.0001.000-0.8850.8080.9460.003
CA21.0720.9281.2670.0020.9320.8620.9820.004
Table 4. Fit Indices for Student COVID-19 Stress Models.
Table 4. Fit Indices for Student COVID-19 Stress Models.
Modelχ2dfBollen-Stine pSRMRRMSEA (90% CI)CFIGFIAIC
5 factor83.054550.0930.0370.049 (0.041–0.083)0.9810.95155.054
Second Order117.34600.0070.06310.067 (0.049–0.084)0.9620.92179.34
4 factor100.263590.0260.04110.057 (0.037–0.076)0.9720.93164.263
Note ** p < 0.01.
Table 5. Test of Convergent Validity: Average Variance Extracted and Correlations between the Stress scale and the SCSS total and sub-dimensions.
Table 5. Test of Convergent Validity: Average Variance Extracted and Correlations between the Stress scale and the SCSS total and sub-dimensions.
Factors/ScaleCRAVEStress Scale
Social Isolation0.840.650.429
Sleep-Somatic Problems0.750.500.547
Contamination fear0.810.680.467
Academic Distress0.860.680.374
Career Apprehensions0.910.830.465
SCSS-13 0.610
All correlations are significant at p < 0.01.
Table 6. Test of Discriminant Validity: Heterotrait-Monotrait ratio of correlations (HTMT).
Table 6. Test of Discriminant Validity: Heterotrait-Monotrait ratio of correlations (HTMT).
S NoFactors12345
1Social Isolation (SI)-
2Sleep and Somatic Problems (SSP)0.583-
3Contamination fear (CF)0.6030.830-
4Academic Distress (AD)0.4270.3670.305-
5Career Apprehensions (CA)0.5590.5310.5790.514-
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kumar, V.V.; Chatterjee, B.; Tankha, G. The Development and Validation of a Measure to Assess COVID-19 Stress among College Students for Sustainable Education. Sustainability 2021, 13, 9828. https://0-doi-org.brum.beds.ac.uk/10.3390/su13179828

AMA Style

Kumar VV, Chatterjee B, Tankha G. The Development and Validation of a Measure to Assess COVID-19 Stress among College Students for Sustainable Education. Sustainability. 2021; 13(17):9828. https://0-doi-org.brum.beds.ac.uk/10.3390/su13179828

Chicago/Turabian Style

Kumar, V. Vineeth, Bhagyasree Chatterjee, and Geetika Tankha. 2021. "The Development and Validation of a Measure to Assess COVID-19 Stress among College Students for Sustainable Education" Sustainability 13, no. 17: 9828. https://0-doi-org.brum.beds.ac.uk/10.3390/su13179828

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop