Next Article in Journal
The Learning of E-Sustainability Competences: A Comparative Study between Future Early Childhood and Primary School Teachers
Next Article in Special Issue
Measuring the Post-Impact of Programming MOOCs: Development and Validation of an Instrument
Previous Article in Journal
Lifestyle Sports and Physical Education Teachers’ Professional Development in the United Kingdom: A Qualitative Survey Analysis
Previous Article in Special Issue
Telework Experience of Pedagogues during the COVID-19 Pandemic: Strong Learning Seniors and Relaxed Leaders?
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Dropout Time and Learners’ Performance in Computer Programming MOOCs

Institute of Computer Science, University of Tartu, 51009 Tartu, Estonia
*
Author to whom correspondence should be addressed.
Submission received: 18 September 2021 / Revised: 9 October 2021 / Accepted: 11 October 2021 / Published: 14 October 2021
(This article belongs to the Special Issue Innovative Technologies in Distance Learning Education)

Abstract

:
One of the problems regarding MOOCs (Massive Open Online Courses) is the high dropout rate. Although dropout periods have been studied, there is still a lack of understanding of how dropout differs for MOOCs with different levels of difficulty. A quantitative study was conducted to determine the periods with the highest dropouts in computer programming MOOCs and the performance of the dropouts on the course before dropping out. Four occurrences of three MOOCs, with different durations, difficulty of the topic, and the degree of supportive methods, were included. The results showed that dropout was highest at the beginning of all studied courses. Learners also dropped out before the project. In the easier and shorter courses, most dropouts were successful until they quit the course. In longer and more difficult courses, learners mainly dropped out in the week they started due to experiencing problems with the course activities. It is suggested to recommend that learners take courses at a level that suits them if their current course is too easy or difficult and encourage learners to use course resources for help. It would be a good idea to provide learners with example topics to assist them in starting with a project.

1. Introduction

MOOCs (Massive Open Online Courses) have the potential to provide access to high-quality education beyond social and geographical restrictions [1,2], allowing learners to learn at their own pace [3,4], from anywhere [3], at any time [4] and free of charge [5]. MOOCs also create opportunities for professional development [6] and participation in top university courses [4]. It has been said that MOOCs can provide wider access to higher education while reducing the cost [7]. MOOCs’ popularity is reflected in the number of participants and the number of universities providing the courses [8].
Learning in MOOCs often takes place autonomously, away from other learners and organizers [5]. The low completion rate is a significant challenge for MOOC organizers [4,9]. Most of the learners do not complete the MOOCs [3]. Previous studies have shown dropout rates to be even around 90% [7,10,11,12]. It is important to study dropouts, as only the learners who persist with the course have the opportunity to gain the prescribed educational experience [2].
In order to provide help to learners when they need it the most, and keeping in mind that course completion or dropout is often contingent on learners’ performance in the course, this study seeks to identify the periods when learners are most likely to drop out, as well as an individual’s performance before dropping out. Research concerning dropout rates has focused on the successive offerings of the same course [12] or different organisations of the same course [13] and has been based on one [9,14] or multiple different courses with different durations [11,15], and demographics [11]. It has also been studied how course length predicts completion rates [15] and how a learners’ background data and organisation of evaluation, among other aspects, are related to dropouts in courses with different durations [11]. However, the duration is only one aspect regarding how the courses differ. Luik et al. [16] (p. 3690) defined the term “level of difficulty” for the MOOCs on the same subject, which “refers to, besides the difficulty of covered topics, the variety of supportive teaching methods and different course durations”. To the authors’ best knowledge, there is no research on dropout rates on courses that, in addition to the different duration of the course, also vary in offered supportive teaching methods and difficulty of covered topics. This study deepens the knowledge on learners’ dropout time and their performance before dropping out, in regard to courses of varied difficulty and duration.

2. Literature Review

In this section, an overview is given about the different definitions of dropout in the context of MOOCs, dropout time, and the reason why learners quit the course, as well as the specifics of the learning program.

2.1. Definition of Dropout

Dropping out is frequently considered as the non-completion of a course, with an indication of the timeframe of when the dropping out took place [17]. However, there is no single definition of dropout in the context of MOOC [17,18], and different researchers define dropout based on the design of the course and used environments. This makes it challenging to compare dropout rates in different studies [17]. For example, when trying to predict a dropout, it is important to consider the learner’s activity on the course. However, activity is difficult to define. It may be that the learner is taking part in any activity, such as either just watching videos [19] or only completing mandatory activities [13]. Chen et al. [20] and Fei and Yeung [18] have gathered definitions from different authors and divided them into three groups. The first definition concerns learners’ engagement in the last week of the course, i.e., whether the learners perform relevant course activities. This approach provides an overview of whether the learner stays to the end of the course or not. Determining dropouts based on the second definition gives researchers information about a learner’s last week of engagement and reviews if the current week is the last week when a learner had any defined activities. Dropouts can be identified in both cases at the end of the course [18] and the course organisers are not able to intervene during the course in the learners’ activities based on this information. The third definition is related to the ongoing state and checks if learners have activities in the next week. For example, if a learner has activities in the first and third learning weeks, he/she is considered to be a dropout in the first and third weeks, as there is no activity in the second and fourth weeks. The learner is not considered to be a dropout in the second week, if he/she has performed activities in the next, third week. This definition enables this study to define whether the learner has dropped out or not with a time lag of one week, and the course organisers can provide timely feedback to learners during the course [18]. Additionally, in the case of a self-based MOOC, an inactivity period has been chosen to define a learner who has dropped out. If a learner is inactive during that period, he/she is considered to have dropped out, as it is unlikely that the learner returns to the course and can complete it [3].
Some researchers have found that dropouts from MOOCs should be measured against learners’ personal goals and not against completing all the course’s mandatory activities and earning the certificate [21,22]. It is proposed to consider learners as successful if they do as they intended, or do more, and as dropouts, if they do less than they intended or quit during the MOOC’s runtime [21]. This is because learners’ individual goals may not coincide with the course organizers’ goals [21,22]. For example, learners may simply want to use course materials later and not in relation to the MOOC [22], or just familiarise themselves with the course without intending to complete it [1]. The downside of this approach is that learners must always be asked for information about their goals in addition to learning analytics. However, it is almost impossible to obtain such data from all learners, as not all learners may answer the questionnaires. This kind of emphasis on learners’ perspective in dropout definition would also impact the dropout rates. According to the example of Henderikx et al. [21], success rates were 6.5% and 5.6% when course completion was taken into account, and 59% and 70% when measured from the learners’ perspective.

2.2. Dropout Time and Reasons

Previous studies are contradictory about the periods of MOOCs when the dropout rate is the highest. There are a group of learners who do not interact with the course after registration and abandon the course before it starts [1,7,12]. It has been found that learners who start with the course activities earlier in a learning week are more likely to stay until the end of the course [23]. It is concluded that the number of learners who completed learning activities decreased each week [8,10]. Multiple MOOCs lasting 6–10 weeks have shown that the dropout rate was high before or at the mid-point of the course [2,4] or after the second quartile, usually during the second or third week or module [5]. Other studies have found that dropouts in the case of 4–11-week MOOCs is most problematic at the start of the course [1,11], especially in the first week [11]. Quick stabilisation of the number of active learners occurs in the case of 4–7-week courses [1]. During longer courses, stabilisation takes place from the sixth week [9]. When learners do not have a weekly schedule and can finish the modules at their own pace, most learners drop out during the first module. The dropout rate decreases in the second module and stabilises afterward [14].
At the beginning of the course, learners may not know enough about the curriculum [23] and have different expectations for the course content [24]. They may discover that the course is not suitable for them [13,23,24], for example, learners realise that the course requires too much time to complete [13]. Learners may choose to only engage with the learning materials and never perform any mandatory activities [12]. They may also be simply curious about the course, without any motivation or interest to complete the course [7]. During the last weeks of the course, learners are also at risk of dropping out [11]. At the end of the course, learners may choose to leave the course because of the final course activities, such as the final exam [23]. On the other hand, it has been shown that some learners become more active during the final days of the course to try to obtain the course certificate [12].
Previous studies have pointed out that some of the reasons for learners dropping out are related to the organization of the course. Although there have been learners who claim that dropout is not highly impacted by the actions of course organizers [24], some learners have had issues with the structure of the course, lack of a tutorial to guide learners [25], monitoring mechanism [4] and feedback [1,4,6]. Sometimes, a course can be too easy or too difficult [24]. The difficulty level of assessments that is consistent with the learners’ level of knowledge and skills helps learners finish the course [1] as learners may drop out if the content is too hard [24]. Learners’ learning skills impact success during the course [24,26]. Related to the course, learners have also criticized the quality of the learning materials [25], such as unconsolidated materials [6]. Social aspects of MOOCs, such as the lack of communication between [4] and support from [1] learners and course organizers, affect learners’ retention on course. Although MOOCs provide the possibility for communication, many learners do not take advantage of it [26].
Personal reasons, such as sickness [24], a bad experience with the topic, online format of the course [27], family responsibilities [10,24,25], and employment situation [10,27] can also have a negative impact on the completion of a course. Dropouts have more conflicts at home and work as they attend the course [28]. In other cases, working has also increased the odds that the course will be completed [25] as learners use the working time for the course activities [28]. From learners’ characteristics, lower education levels decrease the chances of success in the MOOC [25,27]. Learners’ self-efficacy, perceived effectiveness [4], and motivation [4,24] positively impact course completion. It has been found that not all motivation-related factors differ between dropouts and completers [29]. Learners who use the knowledge gained from the MOOC at work or in everyday life [25] or believe that the course content will be useful in their future careers are less likely to drop out. Ineffective time management [24] and finding time for MOOC activities [6,24,27] have a negative impact on the MOOC completion, as learners who drop out are more likely to experience problems finding time for the MOOC compared to completers. Even weather can influence the time available for the course [28].
Some suggestions have been made on how to reduce dropping out. For example, design the course according to the needs of learners from different backgrounds [26] and with different motivations [29], improve the course activities [4], add weekly update videos [24], and add courses to university accredited programs [2]. At the beginning of the course, a test could be conducted among learners, which would give feedback to the course organizers and learners on how well learners would handle the course materials [8]. Adding content supplemented by multimedia, gameplay, and improving social interaction can increase the number of completed learners [1].

2.3. Learning to Program

Programming is a difficult subject [30] and learning to program is especially problematic for students who lack previous experience [31]. In general, learners have different prior knowledge, and it is not easy to design a course that would be interesting and challenging for all the participants [30]. Even learners with previous programming experience do not feel assured with self-directed programming activities [32].
Learners struggle with understanding code [32,33], learning to code [30,32], and debugging [30,34]. The ability to read code is the basis for writing code [33]. Learners and instructors have found error handling, recursion, and pointers to be challenging to learn. Learners also have pointed out problems with syntax errors and variables. According to teachers’ opinions, learners have issues with operators, conditional statements, loops, arrays, and functions [35]. Understanding abstract concepts [30], logical reasoning, problem-solving skills [36], and knowledge of mathematics [36,37] are needed to be successful in learning to program. Reading theory, coding, and learning in groups positively impact and improve programming skills [38].
The basis for success in a course is to succeed at the beginning of the course [33] as this is the period when learners tend to drop out from regular programming courses [34] and programming MOOCs [13]. If learners have problems at the start of learning to program, they are less motivated and may have to quit the course [37]. Collaboration helps to improve motivation [33]. Another common dropout period is at the project phase. According to learners’ opinions, the reasons for failing to finish the project are that the course did not prepare them enough to complete the project or they had issues with time management [34].

2.4. Aim and Research Questions

To provide support to learners, it is necessary to know in which period of MOOCs learners are most likely to drop out and how they perform before dropping out. Previous studies have not shown consistent results. First, this may be due to a different definition of dropout, which researchers define according to their research criteria [17]. In addition, MOOCs differ in the level of difficulty, subject, and organisation. The level of difficulty is used in the same sense in the current study as defined by Luik et al. [16]. Previously described aspects may make it difficult to compare dropout data between different studies.
This study aims to find out the periods of different computer programming (CP) MOOCs in which the dropout is the highest, and how the dropouts performed in the course before dropping out. The basis of the dropout definition in the current study is the second definition of dropout by Chen et al. [20] and Fei and Yeung [18] that relies on the course completion and last week of engagement. This approach provides a more detailed view of the learner’s progress compared to relying only on the completion or non-completion of the course. Depending on the environments used to conduct the courses, engagement is based on the mandatory activities. A learner is considered to have dropped out if he/she did not complete all mandatory course activities, and the dropout week is defined as the last week when he/she submitted at least one of the mandatory activities. When a learner skipped a learning week and later returned to the course, the last week of activity is considered to be the dropout week. The dropouts were identified after the course.
The following research questions are addressed in this paper:
  • What are the periods of CP MOOCs with different levels of difficulty in which dropout is the highest?
  • How did the dropouts perform during the dropout week and the weeks before in terms of successful completion of weekly activities in CP MOOCs with different levels of difficulty?

3. Methodology

3.1. Context of Study

The Institute of Computer Science at the University of Tartu has organised three CP MOOCs. These courses have different levels of difficulty [16] but share the adopted teaching methodologies and are organised by the same group of people. The first course is a four-week course called “About Programming” (in Estonian, “Programmeerimisest maalähedaselt”, hereafter AP) and the second an eight-week course “Introduction to Programming 1” (in Estonian, “Programmeerimise alused”, hereafter IP1). Both are designed for learners with little or no previous programming experience. The third course lasts for eight weeks and is called “Introduction to Programming 2” (in Estonian, “Programmeerimise alused 2”, hereafter IP2). It is for learners with previous programming knowledge. Respectively, the courses had over 11,800, 7100 and 2700 registrants and more than 7300 (around 62%), 3900 (around 55%) and 800 (around 33%) successful completions. The completers receive 1 ECTS for the “About Programming” and 3 ECTS for the other courses.
To complete the AP and IP1 courses, learners had to perform mandatory programming exercises and take a quiz every week. In the case of IP2, the first six weeks were similar to the other courses, but a project was assigned in the last two weeks of the course. To support learners, a forum [16] and troubleshooters [39] for programming exercises were available and, in addition, the learners in “About Programming” and IP1 could contact a helpdesk 24/7. These courses have been described in more detail by Luik et al. [16]. The topics of the “About Programming” course include variables, data types, conditional statements, strings, loops, functions, and regular expression. The materials of IP1 contain similar topics as “About Programming”, but they are covered in more detail. In IP2, learners are introduced to two-dimensional arrays, nested loops, data structures, data analysis, recursion.

3.2. Data Collection and Sample

The courses relevant for this study were “About Programming” and “Introduction to programming” and ran in the autumn of 2018 (this “About Programming” hereafter referred to as AP18), and “About Programming” and “Introduction to programming 2” from the spring of 2019 (this “About Programming” hereafter referred to as AP19). The sample of this study consists of learners who started the courses. In the current case, it means that they submitted at least one mandatory exercise or quiz in Moodle. The number of these learners and their background data is presented in Table 1. Learners’ personal data came from the Study Information System of the university through which they signed up for the course. The data about learners’ activities during the course were gathered from Moodle, which was the main learning environment.
Students were asked to mark their location on a map. It showed that there were learners not only from Estonia but also from Africa (e.g., Cabo Verde), Asia (e.g., Afghanistan, Qatar), Australia, Europe (e.g., Serbia, Spain, United Kingdom), North America (e.g., Canada, USA), and South America (e.g., Brazil, Columbia).

3.3. Data Analysis

Quantitative research was used. In the case of the first research question, descriptive statistics were used to find the weekly number of dropouts. McNemar’s test was used to compare the number of dropouts in different weeks to find the week with the highest dropout rate. To compare the number of dropouts between different courses during the period of the highest dropout, the Z-test was used. For the second research question, the number of learners whose performance in terms of successful completion of weekly activities differed during the dropout week and the weeks before was found using descriptive statistics. The largest group of dropouts each week based on their performance was determined with the Z-test.

4. Results

4.1. Last Week of Activities

Figure 1 describes the learners’ last week in both of the “About Programming” courses. In AP18 and AP19, most learners dropped out in weeks one and two. In AP18, 83 learners (8%) submitted their last mandatory activity in the first week and 95 (9.1%) in the second week. In AP19, 92 (10.9%) and 74 (8.8%) learners dropped out in the first two weeks of the course, respectively. For both courses, there was no statistically significant difference between these two weeks when the percentages of dropped-out learners were compared (p > 0.05). Weeks one and two were both statistically significantly different from weeks three and four (AP18: chi-squared between 9.42 and 17.99, AP19: chi-squared between 5.12 and 29.27, p < 0.05).
Figure 2 describes the learners’ last week in the “Introduction to Programming” courses. In the case of IP1, a comparison between weeks shows that more learners performed their last activities in the course in weeks one and two compared to weeks six to eight (chi-squared between 6.97 and 46.04, p < 0.05). A statistically significant difference was also found between weeks two and three (chi-squared = 4.76, p < 0.05), five and six (chi-squared = 4.14, p < 0.05) and weeks six and seven (chi-squared = 5.32, p < 0.05).
In IP2, 95 (15.7%) and 86 (14.2%) learners dropped out in the first two weeks of the course, respectively. Weeks one and two had more dropped out learners compared to weeks three to five and seven (chi-squared between 16.07 and 81.82, p < 0.05). In week six, 68 (11.2%) learners submitted their last mandatory activities. Week six had a statistically significant difference with weeks five (chi-squared = 14.89, p < 0.05) and seven (chi-squared = 55.13, p < 0.05). The dropouts of weeks seven and eight are grouped under week seven, as learners worked on the project in both of these weeks. In these weeks, the least learners, 4 (0.7%) dropped out from the course (chi-squared between 17.46 and 81.19, p < 0.05). The result of the Z-test showed that more students dropped out of the PA2 course in the first two weeks compared to the “About programming” courses (chi-squared = 86.82, p < 0.05) and PA1 course (chi-squared = 63.15, p < 0.05). There was no statistically significant difference between the “About programming” courses and PA1 (p > 0.05).

4.2. Completing the Week of the Dropout and Previous Weeks

The level of completion of course activities in different weeks by dropped out learners is presented in Table 2. The column “Completed current and all previous weeks” includes learners who had successfully completed the activities of the dropout week and of all previous weeks. Learners who completed the dropout week but failed to complete the activities in some of the previous weeks, are included in the “Completed current week, but not previous weeks” column. The column “Completed previous weeks, but not current week” shows learners who had completed their activities in all previous weeks but failed to complete them in the dropout week, while the column “Did not complete current and at least one of the previous weeks” is for learners who did not complete the activities of the dropout week and at least one, but not all, of the previous weeks. Learners who failed to complete the activities in any of the previous weeks and also in the dropout week are presented in the “All weeks not completed” column.
In this study, the dropout week was defined as the week when the dropouts submitted their last mandatory activity. In AP18 and AP19, the dropout group of the first week included more learners who completed all the mandatory activities than those who did not. More than half of the second week’s dropouts had completed the first and second weeks successfully but did not continue in the third week. In the third week, the largest group of dropouts had completed previous weeks but failed to complete the third week.
In the IP1 course, almost three-fourths of the learners whose last week was week two completed the previous week but failed to complete the dropout week, and the same was true for over half of the learners who quit after week three. The same pattern occurred also in week five where over one-thirds of the dropouts had finished all previous weeks successfully, but not week five. In IP2, over two-thirds of the dropped-out learners in week one were not able to complete the week. In the second and third weeks, the largest group of dropouts included learners who had completed the previous weeks but failed to complete the dropout week. More than half of those who dropped out in the sixth week had completed six weeks successfully but did not continue the course in the following weeks.

5. Discussion

The first research question aimed to find out the periods of CP MOOCs with different levels of difficulty at which dropout is the highest. Regardless of the level of difficulty of the course, learners dropped out more at the beginning of the course in the case of all courses. Dropout stabilised in the second half of AP18 and AP19 and the middle of IP1 and IP2. Another high dropout period in IP2 was the sixth week. Previous studies have also found the highest dropout period to be at the beginning of the course [1,11,13,14] followed by stabilisation of the weekly dropout rate [1]. However, when comparing these results, the different approaches to dropout have to be taken into account. The criteria used by Ihantola et al. [13] for determining dropped-out learners were quite similar to those in the current study. In other studies, however, learners who enrolled but did not start the course could be included [1], learners who learned at their own pace could be excluded [11], the completion of mandatory activities could be used as the basis of calculating the last learning week [1,14], or learners could unsubscribe from the course after enrolling [14]. These aspects may affect the number of dropouts in a given week. Other criteria to be considered when comparing courses in different studies are the activities and methodologies used in the course. For example, in some courses learners do not have direct contact with course organisers [1], but in AP and IP1 courses in the current study, learners were able to contact organisers via helpdesk. There are also different ways to use the same activities, such as quiz, for which the number of attempts can be limited to a certain number of times [1,14] or be unlimited such as in the courses in the current study.
The second research question aimed to determine how the dropouts performed in the dropout week and the weeks before in terms of successful completion of weekly activities. In the case of AP18 and AP19, a similar pattern occurred. At the beginning of the course, where the dropout was highest, most dropped-out learners completed the mandatory activities of the dropout week and the previous weeks. They did not submit any exercises or quizzes in the following weeks. This shows that these learners did not drop out due to problems with the course content. Ihantola et al. [13] also noticed learners with similar behaviour in their programming MOOCs and suggested that learners may have found the future workload to be too high. As “About Programming” was a short introductory course in programming and required about 6 h of work per week, it is rather possible that learners may not have found this course challenging and did not see how they could use the knowledge gained from the course for their career. Previous studies have also found that learners drop out if the course is too easy [24] or they cannot use the learned skills in their work or everyday life [25].
In the third week, most of the dropped-out learners could not successfully finish the mandatory activities. One reason might be related to the loop topic covered this week. Based on teachers’ experiences, it is one of the topics that learners have problems with [35]. Interestingly, regarding week four, over one-third of the learners who dropped out this week were active in their last week, i.e., submitted exercises and quizzes, although they had not completed any previous weeks. They could be the learners that Henderikx et al. [21] and Stracke [22] described as learners who have set individual goals for the course, and these may not include completing the course.
In the case of longer and more difficult courses, IP1 and IP2, the results show that most learners who drop out at the beginning of the course are not able to complete the activities of the week in which they drop out. It may indicate that the courses were too demanding for those learners. Eriksson et al. [24] have also found that difficult content affects dropout. Although IP1 was an introductory course, it has been pointed out that programming is hard to learn without previous experience [31], and even learners with prior knowledge of programming can have problems [32]. In IP2, learners may have overestimated their previous experience with programming when registering for the course. The balance between learners’ knowledge and skills and the course’s difficulty level helps learners be successful [1]. Overestimating their previous experience may also be one of the reasons why during the first two weeks more learners dropped out of the IP2 course compared to the “About programming” and IP1 courses. In addition, in the IP2 course there was no helpdesk. Learners may not have found the help they needed from the other provided teaching methods, such as forum and troubleshooters, and had to drop out. As both of the “Introduction to Programming” courses covered programming more thoroughly and required more work with course materials and more programming exercises to be solved compared to the “About Programming” courses, learners’ poor time management [24] and learning skills may have had an effect on their ability to complete all activities. In addition, learners may need more support than the current MOOC environment can offer. De Freitas et al. [1] have also found that support from other learners and course organizers effects learners’ success in a course.
Over half of the learners who dropped out from IP2 in week six had completed all the required activities before the project but decided not to tackle the project. Lu et al. [23] have also found that learners can drop out of a course because of the final course activities. One of the reasons for this might be that learners feel that the course did not prepare them for the project [34]. On the other hand, it may indicate that learners gained the knowledge they needed from the course in the first six weeks, as these were the so-called typical learning weeks. Learners might feel that they do not need to participate in the project as it does not add any new knowledge. This also shows that receiving a certificate might not be necessary for these learners. Another reason not to start the project may have been the fact that learners had to think of the idea of the project themselves. On the other hand, all learners who submitted their project completed it successfully. This was even true for learners who had not completed some of the mandatory activities in previous weeks. These learners might have found the unfinished topics unnecessary or perhaps hoped to complete the rest of the activities required for a certificate after the project but were unable to do so.
In all of the courses, there were dropouts in each week who had completed all the mandatory activities by the end of the dropout week. This indicates that learners may have also had personal reasons to quit the course. This can include, for example, work [10,27] or family [10,24,25] responsibilities that prevented them from focusing on the course activities. Learners could suddenly become ill [24]. In addition, in the spring course, learners had difficulties finding time for the course because of the good weather [28].

6. Conclusions

Four occurrences of three different CP MOOCs with different levels of difficulty were included in this study. Regardless of the differences between the courses in the level of the difficulty, the findings show that dropout was highest at the beginning of all the studied courses and then stabilised. Therefore, we suggest that the beginning of the course needs more attention such as more attractive tasks and materials to maintain learners. The main difference in dropout periods between courses was related to the course activities. In IP2, learners also dropped out before the project at the end of the course. To help learners start with the project, we suggest giving them example topics for the project if they do not have a good idea of their own. This would help learners understand the scope that is acceptable to do as a project, but they would still have to figure out how to do it. The fact that there were learners who finished the project even if they had not completed previous weekly activities or were active in the last week of the course without completing any of the preceding weeks shows that not all learners are interested in a course certificate.
Learners who dropped out of the shorter and easier courses, AP18 and AP19, in the highest dropout periods, were mainly learners who had succeeded in the course until the end of the dropout week. To motivate learners to stay in the course, we suggest providing more complex additional activities to challenge those who would find the introductory courses too easy otherwise. If possible, learners could be encouraged to take more challenging courses. In the current case, learners for whom the “About Programming” course was too easy could have been transferred to IP1.
In longer and more difficult courses, learners mostly dropped out when they faced problems with the course activities. Therefore, we recommend motivating learners to make more use of different course tools, such as a forum, to find help and support. Simpler courses may be recommended to learners. For example, with the current courses, if learners had issues with IP1, they could have taken “About Programming” instead.
The current study results can be useful for MOOC organisers. For all courses, dropout was higher at the beginning of the course, but looking at learners’ activities before dropping out, our results indicate that learners in courses with different levels of difficulty and different learning weeks may not need similar support. On easier courses, the difficulties learners face is not so much related to course content as to other factors, such as illness. In this case, the course should be flexible enough to adapt to learners’ needs. In addition, learners may participate in easier courses out of curiosity and simply come to explore the course. When they found the course not suitable for them or their curiosity was satisfied, they dropped out of the course. In this case, learners do not need help from the course organisers, and course organisers do not have to pay so much attention to the number of dropouts as learners met their personal goals. In courses with a higher level of difficulty, learners have more problems with successfully performing the course activities. Therefore, support should focus on making the topic more understandable for learners, for example, adding more examples to learning materials and providing different options for support for learners, especially at the beginning of the MOOC. If the course includes a project, learners need more help starting the project than for successful completion. It may also be considered to make the project voluntary. Learners may not want to start with the project as it does not give them any new knowledge, only implementing the previously gained knowledge.
There are some limitations related to this study. The number of participants is smaller than usual for MOOCs but still considerable in the Estonian context. In addition, only four occurrences of three different MOOCs were addressed. As this study is based on CP MOOCs, the results may not be generalisable to MOOCs on other topics.
This study provides other researchers with an initial overview of dropping out of courses with different levels of difficulty. In the future, it will be possible to expand the research, including MOOCs with other topics, and increase the number and heterogeneity of participants. In addition, as the main dropout period was at the beginning of all studied courses, then learners’ difficulties at the beginning of the course and learning skills should be studied to provide the needed support. As there were learners who continued the course activities even if they could no longer obtain the certificate due to the previously uncompleted activities, then in the future, it would be interesting to examine dropouts in terms of a learners’ personal goals.

Author Contributions

Conceptualization, M.R., M.L. and P.L.; data curation, M.R., M.L. and P.L.; formal analysis, M.R.; investigation, M.R., M.L. and P.L.; methodology, M.R., M.L. and P.L.; resources, M.R.; writing of the original draft, M.R.; writing—review & editing, M.R., M.L. and P.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study has obtained approval of the Research Ethics Committee of the University of Tartu (341/T-2, 2021-2026).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. de Freitas, S.I.; Morgan, J.; Gibson, D. Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. Br. J. Educ. Technol. 2015, 46, 455–471. [Google Scholar] [CrossRef] [Green Version]
  2. Hone, K.S.; El Said, G.R. Exploring the factors affecting MOOC retention: A survey study. Comput. Educ. 2016, 98, 157–168. [Google Scholar] [CrossRef] [Green Version]
  3. Moreno-Marcos, P.M.; Munoz-Merino, P.J.; Maldonado-Mahauad, J.; Perez-Sanagustin, M.; Alario-Hoyos, C.; Kloos, C.D. Temporal analysis for dropout prediction using self-regulated learning strategies in self-paced MOOCs. Comput. Educ. 2020, 145, 1–15. [Google Scholar] [CrossRef]
  4. Sujatha, R.; Kavitha, D. Learner Retention in MOOC Environment: Analyzing the Role of Motivation, Self-Efficacy and Perceived Effectiveness. Int. J. Educ. Dev. Using Inf. Commun. Technol. 2018, 14, 62–74. [Google Scholar]
  5. Gregori, E.B.; Zhang, J.; Galván-Fernández, C.; de Asís Fernández-Navarro, F. Learner support in MOOCs: Identifying variables linked to completion. Comput. Educ. 2018, 122, 153–168. [Google Scholar] [CrossRef]
  6. Liu, M.; Kang, J.; McKelroy, E. Examining learners’ perspective of taking a MOOC: Reasons, excitement, and perception of usefulness. Educ. Media Int. 2015, 52, 129–146. [Google Scholar] [CrossRef]
  7. Perna, L.W.; Ruby, A.; Boruch, R.F.; Wang, N.; Scull, J.; Ahmad, S.; Evans, C. Moving Through MOOCs: Understanding the Progression of Users in Massive Open Online Courses. Educ. Res. 2014, 43, 421–432. [Google Scholar] [CrossRef]
  8. Coffrin, C.; de Barba, P.; Corrin, L.; Kennedy, G. Visualizing patterns of student engagement and performance in MOOCs. In Proceedings of the LAK 2014, Indianapolis, IN, USA, 24–28 March 2014; Association for Computing Machinery: New York, NY, USA, 2014; pp. 83–92. [Google Scholar]
  9. Mourdi, Y.; Sadgal, M.; El Kabtane, H.; Fathi, W.B. A machine learning-based methodology to predict learners’ dropout, success or failure in MOOCs. Int. J. Web Inf. Syst. 2019, 15, 489–509. [Google Scholar] [CrossRef]
  10. Gutl, C.; Rizzardini, R.H.; Chang, V.; Morales, M. Attrition in MOOC: Lessons learned from drop-out students. In International Workshop on Learning Technology for Education in Cloud; Springer: Cham, Switzerland, 2014; pp. 37–48. [Google Scholar]
  11. Labrador, M.M.; Vargas, G.R.G.; Alvarado, J.; Caicedo, M. Survival and Risk Analysis in MOOCs. Turk. Online J. Distance Educ. 2019, 20, 149–159. [Google Scholar] [CrossRef]
  12. Vitiello, M.; Walk, S.; Helic, D.; Chang, V.; Guetl, C. User behavioral patterns and early dropouts detection: Improved users profiling through analysis of successive offering of MOOC. J. Univers. Comput. Sci. 2018, 24, 1131–1150. [Google Scholar]
  13. Ihantola, P.; Fronza, I.; Mikkonen, T.; Noponen, M.; Hellas, A. Deadlines and MOOCs: How Do Students Behave in MOOCs with and without Deadlines. In Proceedings of the 2020 IEEE Frontiers in Education Conference (FIE), Uppsala, Sweden, 21–24 October 2020; pp. 1–9. [Google Scholar]
  14. Ricart, S.; Villar-Navascués, R.A.; Gil-Guirado, S.; Hernández-Hernández, M.; Rico-Amorós, A.M.; Olcina-Cantos, J. Could MOOC-Takers’ Behavior Discuss the Meaning of Success-Dropout Rate? Players, Auditors, and Spectators in a Geographical Analysis Course about Natural Risks. Sustainability 2020, 12, 4878. [Google Scholar] [CrossRef]
  15. Jordan, K. Initial trends in enrolment and completion of Massive Open Online Courses. Int. Rev. Res. Open Distance Learn. 2014, 15, 134–160. [Google Scholar] [CrossRef] [Green Version]
  16. Luik, P.; Feklistova, L.; Lepp, M.; Tõnisson, E.; Suviste, R.; Gaiduk, M.; Säde, M.; Palts, T. Participants and completers in programming MOOCs. Educ. Inf. Technol. 2019, 24, 3689–3706. [Google Scholar] [CrossRef]
  17. Goopio, J.; Cheung, C. The MOOC dropout phenomenon and retention strategies. J. Teach. Travel Tour. 2020, 21, 177–197. [Google Scholar] [CrossRef]
  18. Fei, M.; Yeung, D.-Y. Temporal Models for Predicting Student Dropout in Massive Open Online Courses. In Proceedings of the 2015 IEEE International Conference on Data Mining Workshop (ICDMW), Atlantic City, NJ, USA, 14–17 November 2015; IEEE Computer Society: Washington, DC, USA, 2015; pp. 256–263. [Google Scholar] [CrossRef]
  19. Gitinabard, N.; Khoshnevisan, F.; Lynch, C.F.; Wang, E.Y. Your Actions or Your Associates? Predicting Certification and Dropout in MOOCs with Behavioral and Social Features. arXiv 2018, arXiv:1809.00052. [Google Scholar]
  20. Chen, J.; Feng, J.; Sun, X.; Wu, N.; Yang, Z.; Chen, S. MOOC Dropout Prediction Using a Hybrid Algorithm Based on Decision Tree and Extreme Learning Machine. Math. Probl. Eng. 2019, 2019, 1–11. [Google Scholar] [CrossRef]
  21. Henderikx, M.A.; Kreijns, K.; Kalz, M. Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Educ. 2017, 38, 353–368. [Google Scholar] [CrossRef] [Green Version]
  22. Stracke, C.M. Why we need High Drop-out Rates in MOOCs: New Evaluation and Personalization Strategies for the Quality of Open Education. In Proceedings of the 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), Timisoara, Romania, 3–7 July 2017; IEEE Computer Society Publications: Los Alamitos, CA, USA, 2017; pp. 13–15. [Google Scholar]
  23. Lu, X.; Wang, S.; Huang, J.; Chen, W.; Yan, Z. What decides the dropout in MOOCs? In Proceedings of the Database Systems for Advanced Applications: DASFAA 2017 International Workshops: BDMS, BDQM, SeCoP, and DMMOOC, Suzhou, China, 27–30 March 2017; Bao, Z., Trajcevski, G., Chang, L., Hua, W., Eds.; Springer: Cham, Switzerland, 2017; pp. 316–327. [Google Scholar]
  24. Eriksson, T.; Adawi, T.; Stöhr, C. “Time Is the Bottleneck”: A Qualitative Study Exploring Why Learners Drop out of MOOCs. J. Comput. High. Educ. 2017, 29, 133–146. [Google Scholar] [CrossRef] [Green Version]
  25. Gomez-Zermeno, M.G.; Aleman de La Garza, L. Research Analysis on MOOC Course Dropout and Retention Rates. Turk. Online J. Distance Educ. 2016, 17, 3–14. [Google Scholar] [CrossRef] [Green Version]
  26. Park, Y.; Jung, I.; Reeves, T.C. Learning from MOOCs: A Qualitative Case Study from the Learners’ Perspectives. Educ. Media Int. 2015, 52, 72–87. [Google Scholar] [CrossRef]
  27. Shapiro, H.B.; Lee, C.H.; Wyman Roth, N.E.; Li, K.; Çetinkaya-Rundel, M.; Canelas, D.A. Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Comput. Educ. 2017, 110, 35–50. [Google Scholar] [CrossRef]
  28. Rõõm, M.; Luik, P.; Lepp, M. Learners’ use of time in MOOCs about programming. In Proceedings of the EDULEARN20 Proceedings: 12th International Conference on Education and New Learning Technologies, Palma de Mallorca, Spina, 6–7 July 2020; Chova, L.G., Martínez, A.L., Torres, I.C., Eds.; IATED: Valencia, Spain, 2020; pp. 4380–4387. [Google Scholar] [CrossRef]
  29. Luik, P.; Lepp, M.; Palts, T.; Säde, M.; Suviste, R.; Tõnisson, E.; Gaiduk, M. Completion of Programming MOOC or Dropping out: Are There Any Differences in Motivation? In Proceedings of the 17th European Conference on e-Learning ECEL 2018, Athens, Greece, 1–2 November 2018. [Google Scholar]
  30. Lahtinen, E.; Ala–Mutka, K.; Järvinen, H.-M. A Study of the Difficulties of Novice Programmers. In Proceedings of the ITiCSE ‘05 Proceedings of the 10th annual SIGCSE conference on Innovation and Technology in Computer Science Education, Caparica, Portugal, 27–29 June 2005; Association for Computing Machinery: New York, NY, USA, 2005; pp. 14–18. [Google Scholar]
  31. Xia, B.S. An In-Depth Analysis of Teaching Themes and the Quality of Teaching in Higher Education: Evidence from the Programming Education Environments. Int. J. Teach. Learn. High. Educ. 2017, 29, 245–254. [Google Scholar]
  32. Wyeld, T.; Nakayama, M. Visualising the Code-in-Action Helps Students Learn Programming Skills. In Proceedings of the 2018 22nd International Conference Information Visualisation (IV), Fisciano, Italy, 10–13 July 2018; IEEE Computer Society: Los Alamitos, CA, USA, 2018; pp. 182–187. [Google Scholar]
  33. Luxton-Reilly, A.; Simon; Albluwi, I.; Becker, B.A.; Giannakos, M.; Kumar, A.N.; Ott, L.; Paterson, J.; Scott, M.J.; Sheard, J.; et al. Introductory programming: A systematic literature review. In Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education, Larnaca, Cyprus, 2–4 July 2018; Association for Computing Machinery: New York, NY, USA, 2018; pp. 55–106. [Google Scholar] [CrossRef] [Green Version]
  34. Kinnunen, P.; Malmi, L. Why students drop out CS1 course? In Proceedings of the 2nd International Workshop on Computing Education Research, Canterbury, UK, 9–10 September 2018; Anderson, R., Fincher, S.A., Guzdial, M., Eds.; Association for Computing Machinery: New York, NY, USA, 2018; pp. 97–108. [Google Scholar]
  35. Bosse, Y.; Gerosa, M.A. Why is programming so difficult to learn?: Patterns of Difficulties Related to Programming Learning Mid-Stage. ACM SIGSOFT Softw. Eng. Notes 2017, 41, 1–6. [Google Scholar] [CrossRef]
  36. Gomes, A.; Mendes, A. A teacher’s view about introductory programming teaching and learning: Difficulties, strategies and motivations. In Proceedings of the 2014 IEEE Frontiers in Education Conference (FIE), Madrid, Spain, 22–25 October 2014; IEEE Computer Society: Los Alamitos, CA, USA, 2014; pp. 1–8. [Google Scholar] [CrossRef]
  37. Dolgopolovas, V.; Jevsikova, T.; Dagiene, V. From Android games to coding in C– An approach to motivate novice engineering students to learn programming: A case study. Comput. Appl. Eng. Educ. 2018, 26, 75–90. [Google Scholar] [CrossRef]
  38. Tshering, P.; Lhamo, D.; Yu, L.; Berglund, A. How Do First Year Students Learn C Programming in Bhutan? In Proceedings of the 2017 International Conference on Learning and Teaching in Computing and Engineering (LaTICE), Hong Kong, China, 20–23 April 2017; IEEE Computer Society: Los Alamitos, CA, USA, 2017; pp. 25–29. [Google Scholar]
  39. Lepp, M.; Palts, T.; Luik, P.; Papli, K.; Suviste, R.; Säde, M.; Hollo, K.; Vaherpuu, V.; Tõnisson, E. Troubleshooters for Tasks of Introductory Programming MOOCs. Int. Rev. Res. Open Distance Learn. 2018, 19, 56–75. [Google Scholar] [CrossRef]
Figure 1. Last week of activities in the “About Programming” courses.
Figure 1. Last week of activities in the “About Programming” courses.
Education 11 00643 g001
Figure 2. Last week of activities in the “Introduction to Programming” courses.
Figure 2. Last week of activities in the “Introduction to Programming” courses.
Education 11 00643 g002
Table 1. Learners’ background data.
Table 1. Learners’ background data.
CourseAP18AP19IP1IP2
Enrollers127810281552745
Completers773 (60.5%)596 (58.0%)828 (53.4%)253 (34.0%)
N (learners)10428411375605
Gender (female)570 (54.8%)459 (54.6%)697 (50.6%)282 (46.6%)
Age range10–7010–7210–7411–74
Average age32.50 (SD = 10.86)33.25 (SD = 11.05)32.69 (SD = 10.29)33.65 (SD = 10.15)
Table 2. Completion of different course weeks by dropped-out learners.
Table 2. Completion of different course weeks by dropped-out learners.
CourseWeekNumber of Drop-OutsCompleted Current and All Previous WeeksCompleted Current Week, but Not Previous WeeksCompleted Previous Weeks, but Not Current WeekDid Not Complete Current and at Least One of the Previous WeeksAll Weeks Not Completed
AP1818356 (67.5%) *xxx27 (32.5%)
29555 (57.9%) *7 (7.4%)23 (24.2%)x10 (10.5%)
3444 (9.1%)0 (0.0%)29 (65.9%) *6 (13.6%)5 (11.4%)
447x7 (14.9%)8 (17.1%)16 (34.0%)16 (34.0%)
AP1919255 (59.8%) *xxx37 (40.2%)
27447 (63.4%) *1 (1.4%)17 (23.0%)x9 (12.2%)
34811 (22.9%)1 (2.1%)23 (47.9%) *7 (14.6%)6 (12.5%)
431x6 (19.4%)3 (9.7%)11 (35.5%)11 (35.5%)
IP119042 (46.7%)xxx48 (53.3%)
211024 (21.8%)0 (0.0%)81 (73.6%) *x5 (4.6%)
37921 (26.6%)1 (1.3%)44 (55.7%) *11 (13.9%)2 (2.5%)
46625 (37.9%) *6 (9.1%)24 (36.4%) *10 (15.1%)1 (1.5%)
58222 (26.8%)2 (4.4%)37 (45.1%) *20 (24.4%)1 (1.2%)
65720 (35.1%)1 (1.8%)21 (36.8%)14 (24.5%)1 (1.8%)
7347 (20.6%)0 (0.0%)15 (44.1%)12 (35.3%)0 (0.0%)
829x0 (0.0%)13 (44.8%) *13 (44.8%) *3 (10.4%)
IP219531 (32.6%)xxx64 (67.4%) *
2868 (9.3%)0 (0.0%)72 (83.7%) *x6 (7.0%)
3407 (17.5%)2 (5.0%)20 (50.0%) *9 (22.5%)2 (5.0%)
42913 (44.8%)0 (0.0%)7 (24.1%)8 (27.7%)1 (3.4%)
5307 (23.3%)2 (6.7%)14 (46.7%)6 (20.0%)1 (3.3%)
66838 (55.9%) *0 (0.0%)17 (25.0%)12 (17.6%)1 (1.5%)
74x4 (100%)0 (0.0%)0 (0.0%)0 (0.0%)
*—largest group(s) of dropped-out learners in the respective week based on the Z-test (p < 0.05).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rõõm, M.; Lepp, M.; Luik, P. Dropout Time and Learners’ Performance in Computer Programming MOOCs. Educ. Sci. 2021, 11, 643. https://0-doi-org.brum.beds.ac.uk/10.3390/educsci11100643

AMA Style

Rõõm M, Lepp M, Luik P. Dropout Time and Learners’ Performance in Computer Programming MOOCs. Education Sciences. 2021; 11(10):643. https://0-doi-org.brum.beds.ac.uk/10.3390/educsci11100643

Chicago/Turabian Style

Rõõm, Marili, Marina Lepp, and Piret Luik. 2021. "Dropout Time and Learners’ Performance in Computer Programming MOOCs" Education Sciences 11, no. 10: 643. https://0-doi-org.brum.beds.ac.uk/10.3390/educsci11100643

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop