Next Article in Journal
Selected Respiratory Protective Devices: Respirators and Significance of Some Markings
Next Article in Special Issue
The COVID-19 Outbreak and Psychological Distress in Healthcare Workers: The Role of Personality Traits, Attachment Styles, and Sociodemographic Factors
Previous Article in Journal
Applied Big Data Analysis to Build Customer Product Recommendation Model
Previous Article in Special Issue
Teaching during the Pandemic: A Comparison in Psychological Wellbeing among Smart Working Professions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Effects of the COVID-19 Pandemic on Classrooms: A Case Study on Foreigners in South Korea Using Applied Machine Learning

1
College of Informatics, Korea University, Seoul 02841, Korea
2
Department of Computing Science, University of Alberta, Edmonton, AB T6G 2R3, Canada
*
Authors to whom correspondence should be addressed.
Sustainability 2021, 13(9), 4986; https://0-doi-org.brum.beds.ac.uk/10.3390/su13094986
Submission received: 18 March 2021 / Revised: 21 April 2021 / Accepted: 26 April 2021 / Published: 29 April 2021
(This article belongs to the Special Issue Working during the COVID-19 Global Pandemic)

Abstract

:
In this study, we qualitatively and quantitatively examine the effects of COVID-19 on classrooms, students, and educators. Using a new Twitter dataset specific to South Korea during the pandemic, we sample the sentiment and strain on students and educators using applied machine learning techniques in order to identify various topical pain points emerging during the pandemic. Our contributions include a novel and open source geo-fenced dataset on student and educator opinion within South Korea that we are making available to other researchers as well. We also identify trends in sentiment and polarity over the pandemic timeline, as well as key drivers behind the sentiments. Moreover, we provide a comparative analysis of two widely used pre-trained sentiment analysis approaches with TextBlob and VADER using statistical significance tests. Ultimately, we analyze how public opinion shifted on the pandemic in terms of positive sentiments about accessing course materials, online support communities, access to classes, and creativity, to negative sentiments about mental fatigue, job loss, student concerns, and overwhelmed institutions. We also initiate initial discussions about the concept of actionable sentiment analysis by overlapping polarity with the concept of trigger management to assist users in coping with negative emotions. We hope that insights from this preliminary study can promote further utilization of social media datasets to evaluate government messaging, population sentiment, and multi-dimensional analysis of pandemics.

1. Introduction

The COVID-19 pandemic has affected several industries worldwide and especially the educational sector as most countries temporarily closed their institutions. With the educational institutes closed, learners and educators have had to find ways to continue learning. Some countries were fortunate enough to have the proper infrastructures in place to move the classrooms online from the start. One of these countries is South Korea, as even before the pandemic, South Korea had been developing K-MOOCs and had launched their 5G internet [1]. Due to this, 99.7% of South Korean households have access to the Internet, and 99.9% of South Korean teenagers use the Internet for education purposes [2]. Therefore, the transition to online learning due to the pandemic was relatively smooth.
However, online classes are still a challenge for both learners and educators worldwide and South Korea is no exception. There is a small niche that has been more adversely affected by online classes. With the schools closed, embassies were declining visa offers to students, most English teachers were not paid while their schools were closed, and schools were not hiring new teachers [3]. This led to people expressing their emotions on social media.
As these issues affect both educators and students, this research will investigate the emotional flow of educators and students in South Korea using Machine Learning (ML) on Twitter posts. By applying sentimental analysis to conversations happening on social media during the COVID-19 pandemic, it is possible to gauge the emotional wellness of key stakeholders in the educational sector.
To the best of our knowledge, this is the first study focusing on the impact of COVID-19 on South Korea’s educational sector using social media datasets. The social media sampling methodology has been used frequently in research. Hence, this study provides a valuable data analysis of the sentiments during this period by looking into the emotions and the transfer between positive and negative polarity over time. However, one drawback of this strategy is the presence of noise and uncertainty about sample size significance. Nevertheless, this method has demonstrated useful results at much lower costs than traditional qualitative and quantitative survey methods [4,5].

2. Literature Review

This paper focuses on the impact of the Covid 19 pandemic in South Korea, specifically in the educational sector. As schools were opened and closed repeatedly while the pandemic lingered on, students and educators had to make various types of changes to accommodate government mandates. This is not the first time that we have used twitter data to analyze and predict the people’s reaction during a disease outbreak [6,7].

2.1. Covid 19 Public Sentiments in Korea

Sentiment analysis of information was also used during pandemic worldwide [8,9] specifically in countries such as Australia [10] and South Korea [11]. Understandably, around March 2020, the overall sentiment was mostly positive due to trust in governments [12] and fear was the major negative sentiment [13]. However, as the pandemic dragged on, countries began to defer as worries about their economic growth [14].
In the case of South Korea, the top topics were the US Economy and Sports [15]. With the elections occurring in the US, this is not surprising considering Korea’s close ties with the United States. With regards to the pandemic, the public were more interested in how it will affect the economy and their regular lives. Most of the news given to the public was very informative about covid cases, vaccine development, the pandemic impact, and economic relief. In addition, the government was very transparent to the state of breaching privacy on the cases that the public had complete trust in the government’s ability to overcome the pandemic. However, Koreans are very active people and as they host sporting events like the Olympics and World Cup, the Korean public interest in sports also increases [16]. Thus, one of the top negative sentiment topics was either related to sports or a physical activity.

2.2. Sentiment Analysis of the Education Sector in a Pandemic

The main goal for every worker is job satisfaction and for educational institutions, this is correlated with student satisfaction [17]. By mid-2020, roughly 70 countries had incorporated some form of online education system to make up for the academic year disrupted by the pandemic [18]. With meeting students in person being impossible, feedbacks were gotten online. As it is physically impossible for humans to manually process all the data online, many have turned to machine learning techniques to extract information. One of those techniques is sentiment analysis. Sentiment analysis is the process of extracting sentiment polarities to gain insights that fulfill a need. In this regard, a need was created in the education sector, and this led to an increase in sentiment-analysis-based research in the education sector [19,20].
Voluntarily, there is no difference between face-to-face classes and online classes [21]. However, with a government mandate on the overall lifestyle, there is bound to be some negative feedback. Despite the negative sentiment triggers like worry of future insecurity among students and internal digital divide within countries [22,23], countries like Ecuador [24] and Malaysia showed relatively high positive sentiment levels among students. Upon further analysis, we found that the only common factor among the countries was the pandemic. Every country had different factors which influenced their sentiments in the education sector at different levels and at different intervals.

3. Methodology

The overarching methodology of this research involves data collection and sentiment analysis. This is followed by quantitative analysis of emotional polarity and associated topics using applied ML for sentiment analysis using two unsupervised and pre-trained libraries, namely TextBlob and Valence Aware Dictionary for sEntiment Reasoning (VADER), as well as qualitative analysis of anecdotal tweets in order to understand and provide insights on the overall emotions being expressed by South Korean English speaking tweeters.

3.1. Research Questions

We cover the following three main research questions in this research, with emphasis on insights to be gained from analysis of social media chatter. These questions are relevant to researchers and practitioners in digital epidemiology, public health, and the educational sector.
  • What are the major themes and topics driving strong sentiment around education and learning during the COVID-19 pandemic?
  • How can the key markers that are contributing to positive sentiment during the COVID-19 pandemic be amplified?
  • What actions can be taken on the drivers and triggers of negative sentiment during the pandemic in order to mitigate and reduce their recurrence and impact?

3.2. Data Collection

In order to analyze the sentiments of foreign students and educators, a sampling of the population was taken from Twitter’s publicly available tweets. After data collection and filtering, the collected tweets were analyzed for expressions of emotions. An outline of the steps from data collection to analysis is presented in the flowchart in Figure 1. Data was collected in two phases: keywords and users. We have provided our dataset as open source to other researchers via GitHub: https://github.com/DS-Titi/SKCovid-Edu.

3.2.1. Phase I: Keywords Search

In the keywords phase, termed Phase I, a set of search keywords were curated in order to find related tweets. The list of keywords is outlined in Figure 2. These keywords were selected in order to identify tweets that would involve conversations related to classrooms and education. Within Phase I, all collected tweets were also geo-fenced to ensure they were tweets being made by users from within South Korea. Date filters were also applied from beginning of January to end of October 2020, covering the pandemic’s timeline and waves experienced in South Korea.

3.2.2. Phase II: User Profiles Search

The users’ phase, termed Phase II, was necessary to enhance and expand the number of tweets collected from Phase I. In Phase II, unique users from the tweets collected in phase one were enumerated. All tweets made by these users were then collected from their public Twitter profiles.
The process for Phase II is illustrated in Figure 3. The intuition behind collecting all tweets from the users’ profiles was that it would be likely these users were in South Korea and were having other conversations prior to the pandemic that could serve as early indicators and predictors useful for digital epidemiology. Date filters were re-applied in the second phase to retrieve only relevant tweets that were made within the pandemic timeline.
For both Phase I and Phase II, only tweets made in English were collected in order to target foreigners living in South Korea. This underlying assumption is made from anecdotal evidence of observed tweets, where a large majority of English tweets made within South Korea were from foreigners, and the locals predominantly tweeted in Korean exclusively. Overall, given the entire population K of South Korea, our methodology samples a subset E of the population that expressed opinions using Twitter in the year 2020 from within South Korea using English, hence our final dataset d can be represented as { d | d E , E K } . Further details on geo-positioning users and tweets are provided in the proceeding section.

3.3. Platform for Retrieval of Tweets

Tweets were gathered using the Grebe open source social data aggregation platform [25]. Grebe allows indexing of real-time tweets with a specified geo-fence, as well as direct querying of the Twitter API for retrieval of tweets with applied filters. Grebe uses Point-In-Polygon (PIP) method [26] which utilizes the winding number to ensure that a tweet’s reported location is within a specified bounding area, such as shown in Figure 4. The winding number is the sum of values for each angle formed by the polygon given for each of the vertices of the edges and depending on the direction, where vi is the ith vertex of the polygon, and θ are polar coordinates of vi. If the winding number is zero, the coordinates are considered outside of the polygon, otherwise it is inside the polygon and Grebe uses the coordinates. Mathematically, we can define the winding number as follows.
W = 1 2 π i = 0 n 1 θ V i V i + 1
Grebe currently has over 28 million indexed public tweets from Canada, and has been leveraged by other researchers for text mining in social media for identifying city-wide events [27], investigating how public, organizations and health care professionals in Alberta, Canada express wellness in relation to children [28], and for exploring the effects of urban design on mental health [29]. For our research, both phases of data collection were implemented with Grebe, including all applied filters. Hence, Grebe was repurposed to collect tweets with South Korea as the geo-filter and the additional filter of tweet language set as English in order to target foreigners tweeting from within the country. The geo-filtering parameters used in Grebe are illustrated in Figure 4. Essentially, the centers and radii of the bounding circles around the overlays on the area of South Korea are used as inputs to Grebe for filtering tweets based on reported longitudes and latitudes of the tweets when available.

3.4. ML Techniques and Sentiment Analysis

To analyze the sentiment expressed in the collected tweets, we leveraged Machine Learning (ML). In general, ML techniques can be categorized into three main groups: supervised, unsupervised, and semi-supervised learning. With supervised learning, the ML algorithms require prior knowledge to guide decisions. This would typically include a collection of tweets with an assigned emotion label. On the other hand, unsupervised ML methods do not rely on training data or curated labels, but on group threats and non-threats based on general patterns within observations. One benefit of unsupervised ML is that historical data is not needed for training. Semi-supervised ML is useful when the training data is insufficient for supervised ML, but the unsupervised alternative may not give the best results. In this scenario, a small curated data set can be used to make temporary inferences in conjunction with unsupervised ML approaches. Mathematically, we can explain ML methods using the general equation Y = f ( x ) , where Y is the dependent variable, x is the independent variable, and f is a function. For supervised learning, the task is to determine the function f, given the class labels Y and the features x. With unsupervised learning, the task is to determine the function f, given only the observed values of x. With semi-supervised learning, a limited number of values for Y and x are available.
To circumscribe user locations within the geo-fenced areas, the longitude and latitude of the tweet were retrieved from the Twitter API. Twitter allows users to enable location coordinates when tweeting. If the location coordinates are disabled, then Twitter uses the self-reported location in the user’s profile to estimate the coordinates. These coordinates were then passed through the Point-In-Polygon (PIP) methodology described earlier to determine if the coordinates were circumscribed within the specified geo-fence circles or not.
For this research work, we applied two unsupervised and pre-trained libraries, namely TextBlob and VADER. TextBlob uses heuristics with a dictionary-based sentiment polarity lexicon in conjunction with a Naïve Bayes classifier [30]. With this approach, training data is not needed to analyze the sentiments of our tweets dataset. We can mathematically define the classifier using the associated probabilities between the prior observed lexicon entries as features and the sentiment labels as class labels, where X is a feature vector defined as X = { x 1 , x 2 , . . x n } and y is the class label, as follows.
P ( X | y ) = i = 1 n P ( x i | y )
Although TextBlob’s utilization of Naïve Bayes uses all features, it treats all features individually and does not consider the relationship between the features. Thus, using the Naïve Bayes classifier and polarity lexicon, certain words are already labeled with an associated probability of being positive or negative [30,31]. Hence, after stop words removal, the overall sentiment of any given tweet is determined by averaging the probabilities per keyword. After normalizing the averages, this approach returns the polarity as a range between −1 and +1. Very positive statements will be given a score of 1 and very negative statements will be assigned −1.
VADER is a rule-based sentiment analyzer which utilizes pre-labeled lexical features in addition to the semantic orientation of a given text to compute the probability that the sentence is expressing positive, negative, or neutral emotions [32]. In addition, VADER provides a compound score by summing valence scores of all words in the lexicon, and normalized as a range between −1 and +1, equivalent to TextBlob’s polarity score.
Both these sentiment analysis approaches with TextBlob and VADER can be summarized via the Plate notation [33] in Figure 5. In the Results Section, we also investigate the research question of whether both these methods provide similar results or not. For this, the null hypothesis is that there is no significant statistical difference between the polarity results of TextBlob and VADER.

4. Results

Over 31 k tweets were collected for Phase I. By augmenting with additional user profiles from Phase II, around 900 k tweets in total were available. Finally, by applying the COVID-19 pandemic’s timeframe till the end of October 2020, in addition to filtering English-only tweets, the final dataset size was over 16 k tweets for analysis. A summary of the trends in polarity observed in the dataset is shown in Figure 6, demonstrating majority neutral and leaning positive tweets on average.
The trend of polarity over time observed from within our collected dataset is summarized in Figure 7, which also provides further insight into the variance in sentiments via error bars. The results are congruent with Figure 6 and show that polarity was more positive-leaning overall. This can be observed in both the TextBlob and VADER pre-trained sentiment analyzers.
While the results look similar at higher aggregated levels, deeper analysis and drill-down shows that TextBlob and VADER show some variance on reported polarity. The weekly trends presented in Figure 8 demonstrate some discrepancy among both models’ results. Hence, in order to rigorously analyze these discrepancies, we perform statistical significance testing using McNemar’s test statistic, stated as follows, where Χ2 is the test statistic, based on a chi-squared distribution with 1 degree of freedom, and the values for b and c are retrieved from a contingency table, which is a summarized tabulation of outcomes from two tests [34].
Χ 2 = ( b c ) 2 b + c
To tabulate the contingency table, we used the daily mean polarity scores retrieved from TextBlob and VADER. To translate the scores to positive or negative labels, two separate thresholds were used: for TextBlob, values below 0.01 were marked as negative, while those above the threshold were marked as positive, while residuals were marked as neutral. Similarly, for VADER, the threshold was set at 0.05. While these thresholds are arbitrary, our experiments with other threshold values did not change our upcoming conclusion and results on this statistical comparison. The null hypothesis is that there is no significant statistical difference between the polarity results of TextBlob and VADER, with α at 5% being customary, in order to compare with the p-value.
The contingency table for our data is provided in Table 1, giving the test statistic as 23.106. Hence, the p-value from chi-square with 1 degree of freedom is 0.00000153. Thus, we reject the null hypothesis. Further research and evaluation is needed in order to determine which library gives more accurate results, involving comparison with expert-annotated emotion labels, as it is beyond the scope of this research on the impact of COVID-19 on classrooms. While both these pre-trained libraries have been used in research on social media sentiment, one may be more favorable to use due to the underlying lexicons utilized.

5. Discussion

Within the COVID-19 tweets, the words associated with each polarity are obvious. However, the kind of words used was different over time. We further break down this analysis based on the timeframes of the pandemic waves in South Korea. The date range to analyze was determined by the day the term was officially announced on the Yonhap English newspaper. Table 2 shows the major sentiment features from the whole dataset. At the first wave around week 9, sentiment started to decline but was upholstered by positive opinion about the South Korean government’s initial response, as shown in Figure 8. When the second wave hit around week 34, sentiment was only slightly on the rise, most likely driven by local travel. In the subsequent sections that follow, we provide deeper insights into the negative and positive sentiments as well as the key topics and keywords from the analyzed tweets that are driving these sentiments. We provide anecdotal tweets from our dataset to support and demonstrate these insights shown in Twitter-themed boxes and quoted verbatim (including any residual spelling or grammatical errors by the original tweeters).

5.1. Tweets about the Education Sector with Negative Sentiment

As expected, there were also a significant number of negative emotions expressed due to growing frustrations during the pandemic’s timeline. Key drivers we identified included government response, overwhelmed institutions, student concerns, job loss, and mental fatigue.

5.1.1. Government Response

In South Korea, the semester starts on March 1st, and the first wave of the pandemic started a day before the first day of school. Therefore, it is no surprise that most of the negative tweets in the beginning came from the educational sector. Even educational institutions did not know how they would proceed and had 24 h to come up with a solution. The government called for a 2-week temporary closure, which most institutions complied with, and led to some negative tweets against the incompetency of the educational institutes, complaints from people returning from vacations to quarantines and Shincheon-ji, the group responsible for the first wave. However, as the days progressed, there were a lot of positive tweets, “amazing” being the top positive word in the dataset.
While 58% of the positive tweets received were during the first wave, 71% of the negative tweets were during the second wave. This was a global reaction as tensions began to rise because most people had expected the pandemic to be over by summer vacation. However, there was added tension in South Korea as the government was trying various strategies to control the spread of the virus. Some of these regulations included no more than 10 people indoors, and temporary closure of schools in Level 2. During exam period, students had to be broken into groups because no more than 10 people were allowed indoors. This meant teachers had to make multiple exams for the same class and students had to wait their turn to take their exams in a social-distance controlled classroom. The standards for Level 2 have changed over the course of the pandemic but in the beginning, it meant whenever there were more than 100 active cases in a city. This meant schools could suddenly close and change back to online classes. The continual changing schedules meant both students and teachers had to be flexible and match the curriculum accordingly.

5.1.2. Overwhelmed Institutions

South Korea might have an enhanced digital literacy, but they also had some problems switching to online teaching due to unforeseen problems. Institutions had to help both teachers and students to understand the programs implemented but they had insufficient time to implement them, so in the beginning, there was a lot of trial and error. For younger students, a lot of the burden fell on the parents and since South Korea did not have a shutdown, working parents expected the school to handle the situation. In South Korea, education is very important and parents pay close attention on their children’s studies. Due to this, parents also expected the same level of teaching as offline classrooms and held the institutes responsible in ensuring that the quality of education remained the same. This resulted in a lot of pressure on the teachers venting on Twitter about parents. These vents increased after the second wave, which indicates probably a drop in grades or parent’s thinking that their children were not getting proper education based on the previous semester.

5.1.3. Students’ Concerns

This was a common trend around the world filled with mockery and sarcasm but taking Korean educational culture into account, this was not classified as just sarcastic but genuine concern from students in South Korea about their future. Most of the negative tweets ranged from the inability to not understanding a class to how any failures during this online classrooms phase would affect their whole future. Some teachers expressed concern upon seeing these tweets, and there might be a possible solution to this in the future given how much importance the society puts on education.

5.1.4. Job Loss

Most of the government teachers did not have to worry about job losses. However, in South Korea, there is a third educational institution called Academy, which provides after-school education and is mostly privately owned. Most of the government’s closures meant losses to the Academies. The closures led to some of them not being able to afford to keep their institutions running, and most of them had to lay off a lot of teachers. South Korean teachers were still covered by stipends and still could afford to support themselves. However, in these institutions, there were a lot of foreign teachers. This meant inability to afford to live in South Korea and often loss of visa status. This led to a lot of negative tweets whenever there was a new cluster, especially misbehavior, from a fellow foreigner.

5.1.5. Mental Fatigue

By the end of the second wave, there was a noted tone of resignation in the tweets. It can be inferred people expected the pandemic to be over and were planning for their summer vacation. However, with the second wave entering full force in the middle of summer vacation, that was not possible. The second semester started early September, while the second wave was still riding and there were no positive tweets within our sample for the first 3 weeks of the semester.

5.2. Tweets with Positive Sentiments about the Education Sector

In line with the major insights from other sectors, the education industry also had a similar view of the positives during the pandemic. The key driving topics we identified were accessing course materials, online communities, support groups, access to classes, and creativity

5.2.1. Accessing Course Materials

With people at home and the classrooms moving online, teachers had to adapt their materials to fit the new format. However, in South Korea, with more than 90% being computer literate, adaptation was not the problem. Students easily transitioned to online classes. Teachers, especially those teaching younger students, were able to get more creative with their materials. Most of the tweets showed the inclusion of “games” and “DIY activities” into the curriculum. Although ages were not specified, it is likely that the tweets were made by teachers teaching younger grades. Inferred from the overall positive tone of tweets containing the keywords “online” and “zoom”, it is observable that older students preferred online classes to no classes with readings.

5.2.2. Access to Classrooms

Both students and teachers became more lax with time and location as classrooms could be held anywhere at any time. Both sides seemed to be understanding of the arrangements as long as it didn’t disrupt the class. This can be considered a negative effect but with a positive sentiment.

5.2.3. Creativity

The education industry was deeply affected by the sudden intensity of COVID-19 and had to quickly adapt to the situation. This required a lot of creativity from teachers and management. Struggling with creating a new form of normality, students and teachers had to come up with new ways to leverage technology in an interactive and natural way, understand the curriculum, evaluate students without being physically located together, collaborate, and thrive in an online academic year.

5.2.4. Online Communities

With everyone at home and faced with a new education platform, a lot of teachers went online for answers. Twitter played a big part as one of the biggest online communities, producing a platform for teachers from around the world to find and share resources and ideas. 82% of the tweets were from teachers seeking opinions from other teachers around the world. With the use of hashtags, teachers could be very specific about their inquiries and the information they were trying to share. Among the provinces in South Korea, Gyeonggi Province was ahead and set a precinct for most teachers in South Korea. However, as time moved on, basic information was not enough to resolve all the problems encountered by the online platforms and there was a lack of resources to troubleshoot problems.
Most teachers had correspondences and shared information with teachers from the United States, followed by Japan, Philippines, and Taiwan.

5.2.5. Support Groups for Trigger Management

Although serving a similar purpose to online communities, support groups are categorized to be more open to interpretations. ‘Triggers’ are negative sentiment tweets that evoke a strong, often negative, emotional reaction. Twitter is full of strangers and people are more inclined to be less vulnerable to strangers than to people they know. Extracting based on the frequency of replies (@username), the most common triggers include but not limited to those shown in Table 3.
Due to Twitter’s ability to allow people to connect without commitment, support can be seen in different ways. The most popular form is through hashtags and their effectiveness has been seen through popular hashtags like #BlackLivesMatter and #MeToo [35]. Most of the posts about these hashtags were spread through bottom-up process driven by individuals to share information [36], helping people to manage their triggers. In the context of our South Korean dataset, we identified hashtags like #ProfLife, #SelfQuarantine, and #NewNormal, which also created a support network in Korea for people who were stuck at home struggling with coming to terms with the effects of the pandemic.
The support networks we observed used three types of engagements in social media: consuming content via reading posts about other people’s experiences, contributing content by replying and engaging in conversation, and retweeting others’ opinions [37]. We observed trigger management techniques being utilized by these support networks based on their engagement types, and provide the following preliminary grouping of trigger management techniques for social media: active, consolidatory, and deflective. Firstly, active trigger management involved physical activity and was mostly driven by content creators, such as #HomeTraining exercises, DIY crafts, cooking classes, walks, among others. Secondly, consolidatory trigger management was employed by people through use of words of encouragement or offered advice and assistance. Thirdly, deflective trigger management was demonstrated through retweets of Netflix recommendations, memes about Korea, and similar content. Sentiment analysis has typically been used to analyze emotions, but there are interesting research possibilities and questions regarding actionable sentiment analysis. Emotions being expressed before posting a tweet could be used to identify potential triggers and mitigate their effects by use of the appropriate trigger management strategy, ultimately to enable users to cope with negative emotions.

6. Conclusions

In this study, we have examined the effects of COVID-19 on students and educators by analyzing and summarizing data collected from Twitter using machine learning with sentiment analysis to analyze emotions and also identified key topics via qualitative analysis. Ultimately, we highlighted and discussed negative sentiments and triggers about mental fatigue, job loss, student concerns, overwhelmed institutions, and government responses. We also looked into positive sentiments around accessing course materials, online support communities, access to classes, and creativity. In our analysis, we saw how online support communities were able to mitigate some of these negative triggers. However, the underline observation was the direct relationship between the government decisions and the sentiments perceived.
Our contributions include a novel geo-fenced dataset on student and educator opinions within South Korea and an analytical overview of student and educator sentiment during the COVID-19 pandemic. We also discussed the notion of actionable sentiment analysis where the polarity of a post could be overlapped with trigger management strategies in order to enable users to ultimately cope with negative emotions. We also provided statistical significance tests using our datasets to compare TextBlob and VADER, two widely used pre-trained sentiment analysis libraries. To generate further research and discussion, our dataset is available as open source on GitHub for other researchers with the hope that this preliminary study can motivate further studies into utilizing machine learning to investigate the effects of the pandemic on the educational sector and how South Korea can serve as a model in the case of a future pandemic.

Author Contributions

Conceptualization, I.D.A.; methodology, I.D.A. and H.S.; validation, I.D.A., H.L.; formal analysis, I.D.A. and H.L.; investigation, H.L.; resources, I.D.A. and H.L.; data curation, I.D.A. and H.L.; writing—original draft preparation, I.D.A. and H.S.; writing—review and editing, H.S. and H.L.; visualization, I.D.A. and H.S.; supervision, H.L.; project administration, H.L.; funding acquisition, H.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-0-01405) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation). Additionally, it was also supported by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. United Nations (UN). 2020 United Nations E-Government Survey. 2020. Available online: www.un.org/development/desa/publications/publication/2020-united-nations-e-government-survey (accessed on 21 January 2021).
  2. Ministry of Education, Republic of Korea. Responding to COVID-19: Online Classes in Korea. 2020. Available online: www.moe.go.kr/boardCnts/fileDown.do?m=0303&s=english&fileSeq=0da904a1ab25a9ae9d7273b7433120e5 (accessed on 21 January 2021).
  3. Korea Immigration. Immigration and Foreign Policy Statistics Report. Available online: www.immigration.go.kr/immigration/1569/subview.do (accessed on 21 January 2021).
  4. Ghosh, S.; Zafar, M.B.; Bhattacharya, P.; Sharma, N.; Ganguly, N.; Gummadi, K. On Sampling the Wisdom of Crowds: Random vs. Expert Sampling of the Twitter Stream. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management (CIKM), San Francisco, CA, USA, 28 October 2013. [Google Scholar]
  5. Gerlitz, C.; Rieder, B. Mining One Percent of Twitter: Collections, Baselines, Sampling. M/C J. 2013, 16. [Google Scholar] [CrossRef]
  6. Dredze, M.; Paul, M.J.; Bergsma, S.; Tran, H. Carmen: A Twitter Geolocation System with Applications to Public Health. In Proceedings of the AAAI Workshop on Expanding the Boundaries of Health Informatics Using AI (HIAI), Bellevue, WA, USA, 14–15 July 2013; Volume 23. [Google Scholar]
  7. Lee, K.; Agrawal, A.; Choudhary, A. Real-time Digital Flu Surveillance using Twitter data. In Proceedings of the 2nd Workshop on Data Mining for Medicine and Healthcare, Austin, TX, USA, 4 May 2013. [Google Scholar]
  8. Yin, H.; Yang, S.; Li, J. Detecting Topic and Sentiment Dynamics Due to COVID-19 Pandemic Using Social Media. In Proceedings of the Transactions on Petri Nets and Other Models of Concurrency XV, Foshan, China, 12–14 November 2020; Springer Science and Business Media LLC: Berlin, Germany, 2020; pp. 610–623. [Google Scholar]
  9. Hung, M.; Lauren, E.; Hon, E.S.; Birmingham, W.C.; Xu, J.; Su, S.; Hon, S.D.; Park, J.; Dang, P.; Lipsky, M.S. Social Network Analysis of COVID-19 Sentiments: Application of Artificial Intelligence. J. Med. Internet Res. 2020, 22, e22590. [Google Scholar] [CrossRef] [PubMed]
  10. Zhou, J.; Yang, S.; Xiao, C.; Chen, F. Examination of Community Sentiment Dynamics due to COVID-19 Pandemic: A Case Study from Australia. arXiv 2020, arXiv:2006.12185. [Google Scholar]
  11. Park, H.W.; Park, S.; Chong, M. Conversations and Medical News Frames on Twitter: Infodemiological Study on COVID-19 in South Korea. J. Med. Internet Res. 2020, 22, e18897. [Google Scholar] [CrossRef]
  12. Kumar, A.; Khan, S.U.; Kalra, A. COVID-19 pandemic: A sentiment analysis: A short review of the emotional effects produced by social media posts during this global crisis. Eur. Heart J. 2020, 41, 3782–3783. [Google Scholar] [CrossRef]
  13. Samuel, J.; Rahman, G.G.M.N.; Ali, M.M.; Esawi, E.; Samuel, Y. COVID-19 Public Sentiment Insights and Machine Learning for Tweets Classification. Information 2020, 11, 314. [Google Scholar] [CrossRef]
  14. Indonesia’s Economic Growth to Slow in 2020 on COVID-19 Impact, but Gradual Recovery Expected in 2021. Available online: https://www.adb.org/news/indonesia-s-economic-growth-slow-2020-covid-19-impact-gradual-recovery-expected-2021 (accessed on 21 January 2021).
  15. Piyush, G.; Okamura, K. Investigating COVID-19 News Across Four Nations: A Topic Modeling and Sentiment Analysis Approach. IEEE Access 2021, 9, 36645–36656. [Google Scholar]
  16. Byun, J.; Leopkey, B. Exploring Issues within Post-Olympic Games Legacy Governance: The Case of the 2018 PyeongChang Winter Olympic Games. Sustainability 2020, 12, 3585. [Google Scholar] [CrossRef]
  17. Mostafa, L. Egyptian Student Sentiment Analysis Using Word2vec During the Coronavirus (Covid-19) Pandemic. Adv. Intell. Syst. Comput. 2020, 195–203. [Google Scholar] [CrossRef]
  18. World Bank. How Countries Are Using Edtech (Including Online Learning, Radio, Television, Texting) to Support Access to Remote Learning during the COVID-19 Pandemic. Available online: https://www.worldbank.org/en/topic/edutech/brief/how-countries-are-usingedtech-to-support-remote-learning-during-the-covid-19-pandemic (accessed on 20 April 2021).
  19. Tseng, C.-W.; Chou, J.J.; Tsai, Y.C. Text Mining Analysis of Teaching Evaluation Questionnaires for the Selection of Outstanding Teaching Faculty Members. IEEE Access 2018, 6, 72870–72879. [Google Scholar] [CrossRef]
  20. Hew, K.F.; Hu, X.; Qiao, C.; Tang, Y. What Predicts Student Satisfaction with MOOCs: A Gradient Boosting Trees Supervised Machine Learning and Sentiment Analysis Approach. Comput. Educ. 2020, 145, 103724. [Google Scholar] [CrossRef]
  21. Mullen, J.S.; Sullivan, J.M. Student-perceived effectiveness of online content delivery modes. In Proceedings of the 2015 IEEE Frontiers in Education Conference (FIE), El Paso, TX, USA, 21–24 October 2015. [Google Scholar]
  22. Xiao, C.; Li, Y. Analysis on the influence of the epidemic on the education in China. In Proceedings of the International Conference on Big Data and Informatization Education (ICBDIE), Hangzhou, China, 2–4 April 2020. [Google Scholar]
  23. Khattar, A.; Jain, P.R.; Quadri, S.M.K. Effects of the disastrous pandemic covid 19 on learning styles, activities and mental health of young indian students—A machine learning approach. In Proceedings of the 4th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 13–15 May 2020. [Google Scholar]
  24. Pazmiño, R.; Badillo, F.; González, M.C.; García-Peñalvo, F.J. Ecuadorian Higher Education in COVID-19: A Sentiment Analysis. In Proceedings of the Eighth International Conference on Technological Ecosystems for Enhancing Multicul-Turality (TEEM’20), Salamanca, Spain, 21–23 October 2020; Association for Computing Machinery: New York, NY, USA, 2020. [Google Scholar]
  25. Samuel, H.; Noori, B.; Farazi, S.; Zaiane, O. Context Prediction in the Social Web using Applied Machine Learning: A Study of Canadian Tweeters. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence (WI), Santiago, Chile, 3–6 December 2018. [Google Scholar]
  26. Samuel, H.; Zaiane, O.; Martz, P. Supporting Digital Epidemiology in Alberta via Twitter Tracking. In Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Orland, FL, USA, 16–19 February 2017. [Google Scholar]
  27. Esha. Harnessing Tweets to get the Pulse of a City. Master’s Thesis, Department of Computing Science, University of Alberta, Edmonton, AB, Canada, 2020. [Google Scholar]
  28. Martz, P. An Observational Study of Social Media Technology Conversations: Exploring how Members of the Alberta Public, Organizations and Health Care Professionals Express Wellness, in Relation to Children. Master’s Thesis, University of Alberta, Public Health Sciences, Edmonton, AB, Canada, 2015. [Google Scholar]
  29. Betkowski, B. Researchers Tap Twitter to Look at How Urban Design Affects Mental Health. 2019. Available online: https://www.folio.ca/researchers-tap-twitter-to-look-at-how-urban-design-affects-mental-health/ (accessed on 21 January 2021).
  30. Loria, S. Textblob Documentation. 2018. Available online: https://textblob.readthedocs.io/en/dev/ (accessed on 22 January 2021).
  31. Kaur, C.; Sharma, A. Twitter Sentiment Analysis on Coronavirus using Textblob. In Proceedings of the 28th International Conference on Computers in Education, Roorkee, India, 25–26 March 2021. [Google Scholar]
  32. Hutto, C.; Gilbert, E. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. In Proceedings of the International AAAI Conference on Web and Social Media, Ann Arbor, MI, USA, 1–4 June 2014. [Google Scholar]
  33. Asriadie, M.S.; Mubarok, M.S. Adiwijaya Classifying emotion in Twitter using Bayesian network. J. Phys. Conf. Ser. 2018, 971, 012041. [Google Scholar] [CrossRef]
  34. McNemar, Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 1947, 12, 153–157. [Google Scholar] [CrossRef] [PubMed]
  35. Yang, G. Narrative Agency in Hashtag Activism: The Case of #BlackLivesMatter. Media Commun. 2016, 4, 13–17. [Google Scholar] [CrossRef] [Green Version]
  36. Wang, R.; Liu, W.; Gao, S. Hashtags and Information Virality in Networked Social Movement: Examining Hashtag Co-Occurrence Patterns. Online Inf. Rev. 2016, 40, 850–866. [Google Scholar] [CrossRef]
  37. Tsai, W.-H.S.; Men, L.R. Motivations and Antecedents of Consumer Engagement with Brand Pages on Social Networking Sites. J. Interact. Advert. 2013, 13, 76–87. [Google Scholar] [CrossRef]
Figure 1. Data Collection Steps Overview.
Figure 1. Data Collection Steps Overview.
Sustainability 13 04986 g001
Figure 2. Phase I’s Search Keywords.
Figure 2. Phase I’s Search Keywords.
Sustainability 13 04986 g002
Figure 3. Phase II of User Profiles Search.
Figure 3. Phase II of User Profiles Search.
Sustainability 13 04986 g003
Figure 4. Geo-Fencing of South Korean Tweets using Grebe [25].
Figure 4. Geo-Fencing of South Korean Tweets using Grebe [25].
Sustainability 13 04986 g004
Figure 5. Plate Notation for Sentiment Analysis (Ti = Tweet, w = Keywords, Pi = Polarity).
Figure 5. Plate Notation for Sentiment Analysis (Ti = Tweet, w = Keywords, Pi = Polarity).
Sustainability 13 04986 g005
Figure 6. Sentiment Polarity during the COVID-19 Pandemic for TextBlob and VADER.
Figure 6. Sentiment Polarity during the COVID-19 Pandemic for TextBlob and VADER.
Sustainability 13 04986 g006
Figure 7. Monthly Polarity Trends and Variance over Time for TextBlob and VADER.
Figure 7. Monthly Polarity Trends and Variance over Time for TextBlob and VADER.
Sustainability 13 04986 g007
Figure 8. Weekly Polarity Trends for TextBlob and VADER.
Figure 8. Weekly Polarity Trends for TextBlob and VADER.
Sustainability 13 04986 g008
Table 1. Contingency Table for McNemar’s Test using Daily Polarity and Thresholds for assigning Positive and Negative labels.
Table 1. Contingency Table for McNemar’s Test using Daily Polarity and Thresholds for assigning Positive and Negative labels.
VADER PositiveVADER Negative
TextBlob Positive43972193
TextBlob Negative18867716
Table 2. Major Topics Driving Sentiment.
Table 2. Major Topics Driving Sentiment.
WaveStart DatePositive SentimentNegative Sentiment
1st29/02/2020Government
Response
Activities
Quarantine
Shincheonji
2nd16/08/2020Online communities
Throwbacks
Local travel
Cancelled plans
New regulations
Mental Fatigue
Table 3. Common Triggers in Social Media Chatter and Affected Cohort.
Table 3. Common Triggers in Social Media Chatter and Affected Cohort.
TriggersStudentsTeachers
Complaints about insufficient materials for online classesXX
Fear/Irritation whenever there were confirmed cases in schoolsXX
Wearing masks all the time in the classroomXX
Phone calls/Constant supervision from parents X
Inability to create teacher–student and student–student rapports X
Irritation at regular fever checks at school entrancesX
Inability to get help with class materials (students)X
Physically going to schoolX
Inability to fully utilize the school facilitiesX
Fear to eat in school cafeterias/meeting people during breaksXX
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Aiyanyo, I.D.; Samuel, H.; Lim, H. Effects of the COVID-19 Pandemic on Classrooms: A Case Study on Foreigners in South Korea Using Applied Machine Learning. Sustainability 2021, 13, 4986. https://0-doi-org.brum.beds.ac.uk/10.3390/su13094986

AMA Style

Aiyanyo ID, Samuel H, Lim H. Effects of the COVID-19 Pandemic on Classrooms: A Case Study on Foreigners in South Korea Using Applied Machine Learning. Sustainability. 2021; 13(9):4986. https://0-doi-org.brum.beds.ac.uk/10.3390/su13094986

Chicago/Turabian Style

Aiyanyo, Imatitikua D., Hamman Samuel, and Heuiseok Lim. 2021. "Effects of the COVID-19 Pandemic on Classrooms: A Case Study on Foreigners in South Korea Using Applied Machine Learning" Sustainability 13, no. 9: 4986. https://0-doi-org.brum.beds.ac.uk/10.3390/su13094986

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop