Design, Development and Use of a Digital Badges System in Higher Education

Abstract

Badges are non fungible tokens used to document or certify achievements in academics or in any other domain. Besides their utility as proof of achievement, badges are of interest in higher education as pure incentives to push the students and learners in general toward pursuing well-defined goals and skills. In this respect, badges can be naturally incorporated into digital learning platforms also as part of informal learning activities, and stimulate participation, merit, and visibility for the students. However, the level of engagement of learners into these learning activities depends crucially on how the system of badges and rewards has been designed and on how it is applied, if it is to be used effectively. In this paper, we report on the design and implementation of a smart badges system, and an example of its use inside a gamification methodology for informal learning in a master level course, and discuss the benefits and the observed gains in learning performance that can be obtained.

1. Introduction

Digital badges are electronic symbols used to document performance [1]. Recently, they have gained interest in education as a tool for promoting engagement, motivation, participation and achievement [2]. Open badges are intended to provide additional information via metadata, in order to verify issuer details, evaluation criteria and evidence.

The use of badges as credentials implies the importance or standardization, in order to provide credibility for skill acquisition [3]. In 2012, the Mozilla Foundation introduced the Open Badges Infrastructure [4] as a valuable technology for educational settings. Since then, other sites emerged with similar goals and approaches, such as Credly [5], BadgeOS [6], Accredible [7], etc, and smart badges systems have been implemented in several educational institutions around the world. Although different badges environments may accommodate varying degrees of interoperability, the private and open source developers were and are aware of the need for portability of credentials and the expectations that users want to accumulate and display badges.

Since the middle of the last decade, several studies have emerged on how badges can best be purposed [8,9] and on their value and efficacy in educational settings. Related to higher education (see Section 2), its future may benefit from open badges used as mobile credential systems. They are a way of recognizing informal learning that occurs outside the traditional classrooms. Moreover, they have the potential to strengthen traditional programs to support competency-based programs and to link earners to employers and professional organizations.

Nevertheless, more evidence is needed to support the value and contribution of these systems in high educational levels. In this work we describe the features of the Badges module of our custom made Social Learning Environment (SLE) SocialWire [10] and an example of its use in a gamified edition of a master level course on computer networks.

The rest of the paper is organized as follows. Section 2 summarizes some recent related work. Related modules in SocialWire are presented in Section 3. Section 4 describes in detail the badges module and Appendix A its integration with Open Badges. The example of use is explained in Section 5. Finally, some concluding remarks are included in Section 6.

2. Related Work

Following, we present a representative review of works related to the use of digital badges on higher education. Other studies that consider other educational context can be found in [2].

The purpose of the study [11] is to investigate if achievements based on badges can be used to infer the behavior of students of a course on data structures and algorithms, even when the badges have not impact on the grading. Statistically significant differences in students’ behavior are observed with some badges types, while other types do not seem to have such an effect. In [12], the authors design and develop two educational games with and without a range of reward features, and examine learning outcomes among participants in university laboratories. Although both games improve learning, analyses suggest that the quantity of rewards do not have an impact on learning outcomes. Nevertheless, those who feel more rewarded have more favorable views of the gameplay experience. The system described in [13] allows the participation of the administrators, teachers, learners and other members of the university community in the creation, attribution and support of badges. Results of a study of use in two schools reveal that earning badges contribute for users to feel more engaged with specific tasks or activities in the platform. The study [14] examines college undergraduates enrolled in first-year writing courses, where badges represent essential course outcomes. Participants are categorized as having either high or low expectancy-values, and intrinsic motivation to earn badges is measured repeatedly during the 16-week semester. Findings suggest that incorporating digital badges as an assessment model could benefit learners who have high expectations for learning and place value on learning tasks, but it could also disenfranchise students with low expectations. In [15], the authors conduct an experimental study to investigate the effectiveness of gamification of an online course on computer graphics. Badges in the gamified course represent achievements related to certain accomplishments. Each badge is rewarded when a certain set of activities is completed. The positive results obtained encouraged authors to study the impact of the use of gamification in other subjects. The article [16] reports the effects of game mechanics on students’ cognitive and behavioral engagement through an experiment conducted in a design questionnaires course. Participants who collect badges get some points that are displayed in the course leaderboard. Results show that students enrolled in the gamified version of the course are more motivated to perform more difficult tasks and to participate actively in discussion forums. The research addressed in [17] examines the impact of different learning styles and personality traits on students’ perceptions of engagement and overall performance in a gamified business course. Students receive badges for reaching various milestones, including making a specific number of trades, or making a specific number of comments. Findings suggest that students who are oriented towards active or global learning as well as extroverted students have a positive impression of gamification. The work [18] describes the application of gamification in an operations research/management science course, where it is possible to observe an increase of participation in class, better results and a good assessment of the course made by the students. In this case, badges are given to students in many different situations, such as a particularly noticeable participation in the classroom, an extraordinary performance in one activity, and so on. The experiment addressed in [19] describes the gamification methodology employed in a course about teaching principles and methods. In the study, six kinds of badges are used. Students acquire some of the badges in class activities while others are gained through distance learning. Badges are of particular importance as an element of prestige, despite the fact that they are not taken into consideration in assessment and evaluation. The results show positive sentimental attitudes towards the lessons and a moderate effect on achievements. Although there is not difference between the final grades of the gamified and the control groups, teacher candidates regard gamified activities positively.

More recently, the article [20] presents the gamification process, iterations made into the game elements and their features and students’ perceptions in a gamified teacher education course. Related to badges, several of the participants value the feedback about their performance and progress, and for some of them their supportive messages help them to boost their confidence. The study [21] investigates college students’ experiences of a gamified informatics course. Badges are used as social capital and minor rewards. Results show positive trends with respect to students’ perceptions of gamification’s impact on their learning, achievements and engagement in the course material. In [22], authors present a customizable platform aimed at facilitating the process of gamifying academic courses and enabling contextual studies on educational gamification. The platform offers by default a chain of badges given for mastery of skills. In addition, the instructor is able to define their own badges together with rules for receiving them. A user-friendly graphical interface enables easy defining of a badge with the rules for awarding it. The authors of [23] explore the effects of gamification on students’ engagement in online discussions in an undergraduate level course. Badges are the basic units of the gamification system and several other features are built upon the badge system. Two main categories of badges exist: system (automatic) badges and instructor (manual) badges. Automatic badges are assigned by the system based on the quantity of student activities, and manual badges are assigned by the instructor considering the quality of discussion posts. Conclusions and interviews with students and teachers suggest a positive effect of the game-related features of the platform. The work [24] studies the usefulness of gamification to motivate students to carry out quality out-of-class activities in a master level course on statistics. Results obtained reveal higher quality in the work of the students enrolled in the gamified version of the course. The work [25] analyses the effect of using gamification elements in a course related to software development where badges reward the successful completion of tasks. The study confirms that students’ grades and motivation increase as a result of applying gamification to their learning process. In the experiment described in [26], university students are randomly assigned to three different conditions: no badges, badges visible to peers and badges only visible to students themselves. Contrarily to expectations, the last one is evaluated more positively than the second one. The study of the effects of using gamification elements in courses that make use of a wiki environment on the participation rates as well as on student academic success is addressed in [27]. In the gamified experimental group, badges are obtained at the end of each activity rewarding the participation. Authors conclude that wiki activities positively contribute to student academic success, while gamification increases student participation. In [28] authors measure the impact of the gamification methodology used in a university course on the evolution of motivational levels of students. Related to badges, teachers do not announce in advance which ones could be collected in order to avoid that students adscribe their behaviour to the promised rewards. The findings illustrate the significance of the individual nature of motivational processes, the importance of sensitive longitudinal motivation measurements and the relevance of the implemented game elements’ design characteristics. The work addressed in [29] indicates, from a cohort of undergraduate business students, that course performance is significantly higher among those students who participate in the proposed gamified system than in those who engage with the traditional delivery. To give students a sense of competence, badges and leaderboards are used to reward them for achievement. Gold badges are designed for higher-level learning tasks involving evaluation and analysis; silver badges are designed for intermediate-level tasks associated with feedback and application; and bronze badges are developed for comprehension and understanding types of tasks. The article [30] describes an advanced learning environment that detects and responds to computer programming students’ emotions by using machine learning techniques, and incorporates motivation strategies by using gamification elements, such as points, badges and leaderboards. In [31], authors develop a scale to measure the factors that may affect the gamification process via kahoot in a pre-service teachers undergraduate course. Related to badges, gold, silver and bronze medals reward different levels of achievement and are translated and added to the students’ grades at the end of the course. Conclusions of the study suggest that the achievement criteria in the gamification process need to be regularly improved and changed. Moreover, some collaborative and social mechanics must complement the gamification process in order to increase the participation level of less sociable users. The work [32] presents a qualitative multi-case study that examines digital badge programs used at three higher education institutions, at the course level, the department level, and the university level. The main factors found in this study to facilitate diffusion and adoption of badges are compatibility of the badge program with the institution’s values and needs, observability of the value of badges both internally and externally, and relative advantage of badges grounded by a clear purpose communicated to stakeholders. Finally, the article [33] explores the effects of badges on the intrinsic motivation levels of introductory programming students in higher education. Findings are inconclusive as the quantitative results suggest that badges do not increase intrinsic motivation scores. Conversely, badge survey and qualitative data indicate that badges are received positively. Authors conclude that further research is necessary to investigate whether gamification, through the use of digital badges, can foster positive intrinsic motivational results.

3. Gamification in SocialWire

SocialWire [10,34] is the name of a project that started at the middle of the last decade due to the lack at that time of full functionality in the Learning Management Systems (LMS) for embedding social learning and gamification features. The goal was to create a SLE which would integrate a set of classical learning tools with new modules to implement these features.

In this work we focus on the Badges module. To understand correctly the features of this module, it is convenient to describe first some of other modules that coexist in SocialWire related to gamification.

• Questions and answers: this module allows students and teachers to post questions or to submit answers. Valid questions and answers can get a certain number of points, depending on the usefulness, correctness, completeness, difficulty, etc. Different parameters such as the visibility of the answers of other students can be defined. Correct answers are clearly marked, in order to avoid misunderstanding.
• Gamepoints: this module defines a new entity called Gamepoint, that is a numeric value gained by a student after the completion of a number of actions: finishing tasks, taking quizzes, upload questions or answers posed by classmates, etc. Gamepoints are accumulated and registered under the student’s profile.
• Activitypoints: as with Gamepoints, this module defines another entity called Activitypoint similar to a Gamepoint, but applied over a different set of actions to be rewarded. Students can get Activitypoints by reading or writing blog posts, by sending or receiving comments, by giving or receiving likes, by creating new files, etc. Activitypoints are accumulated into an individual counter for every student.
• Tasks. This module allows the creation of assignments to be completed by the members of a group, setting some parameters that apply to the submission: response type, visibility of the answers issued by other students, whether the task is graded or not (and the grade system if it is), and so on.
• Tests. For creating quizzes or test exams. As in Tasks, the possibility of particularization is wide: time/date of start, maximum allowed time, graded/not graded, visibility of correct answers, number of attempts, etc.
• Contests. This is a module conceived for competitions. That is, the students submit their answers, and later each one votes the answers by the other classmates. With the ballots, a ranking is formed to decide the winner(s) of the contest. Some of the fields that ought to be defined when creating a Contest are the type of responses, permissions for multiple or single answers by a student, author’s visibility of the answer before closing the voting, limit on votes per answer, limit on votes per user, etc.

4. Badges Module

In this Section, we describe the design and implementation of the Badges module of SocialWire. Our purpose is to highlight its power and versatility to integrate with other modules of the platform and offer course designers the possibility of implementing a wide range of game mechanics that can be useful in different educational environments.

4.1. Properties

Within a specific group (or course) in the social network there exists a section called Group badges giving access to a view with all the existing options for the badges (creation, edition, list of leaders, etc.), and to a list with all the badges awarded to the members of the group, along with their icons and some attributes:

• Name. Just the name of the badge.
• Description. A short description of the purpose or object for that badge. This is optional if the badge has group visibility, and mandatory whenever that badge is public, since in the latter case the badge will be a proof of knowledge, a proof of completion for the course, etc.
• Icon. The image(s) associated to the badge, to identify it. The icon can be chosen from a predefined set or from the local filesystem of the user creating the badge (Figure 1).
• Text. A short text associated to the icon, usually the name of the badge, or any other alternative motto linked to the badge.
• Surprise. If the option is enabled, the badge will not be shown normally in the list of badges for the group, but it will be given to the users blindly, namely, the users will not know the conditions set for gaining the badge until the moment they receive it. This option is disabled by default, and it will not be possible to modify it in manual badges.
• Global visibility. When enabled, the rest of the members in the group can see which group members have the badge or not. If disabled, a member can only see his/her own information on badges.
• Privacy. There are two options, group (only the users in a group can track the badge) or public (everyone can track the accumulated badges in any group, regardless the membership). In the latter case, since users can monitor the progress of members in foreign groups, they can learn about the completion of the course, the achieved grades, the fulfillment of some skills, etc.
  A public badge allows the users to get a certificate of that badge, making its possession a permanent and definitive fact. That is, even if the group to which the user belonged if eliminated, the badge persists and is included in his/her profile. This does not happen in badges with group privacy, they are associated to a concrete group and, in case the group is deleted, so is the badge.
• Type. Depending on the form to obtain it, the badge can be manual, automatic, or by contest. We explain next the features for each type.

  -

  Automatic badges. As the name suggests, these badges will be automatically awarded to students who comply with a number of requirements listed below.

  *

  Merits. The student must satisfy a set of requirements in the tasks and/or quizzes (Figure 2), such as:

  ·

  To pass that task or quiz (for those which are graded, not the ones that only offer Gamepoints).

  ·

  To reach a threshold in the grade, both for the tasks and quizzes graded and for those bond to Gamepoints. The mixture of tasks and quizzes of either type that contribute to the threshold is designed by the creator.

  ·

  To answer a task or quiz, no matter which grade is attained with the answer. The intent is giving to the teachers the chance to reward not only the excellent answers, but also the effort and dedication of the students.

  These requirements for quizzes and tasks can also be combined as the designer wishes to, thus incorporating for instance the possibility of requiring a minimum amount of Gamepoints in a task and take two quizzes with A grade in both, or instead like passing a Task, simply.

  *

  Required Gamepoints. This is the minimum Gamepoints a student has to accumulate in order to get the badge.

  *

  Required Activitypoints. Same as above, the designer can set a minimum value of Activitypoints to get the badge.

  *

  Required badges. This is to define a sequence of previous badges that must be won before aiming at the new one.

  -

  Manual badges. Either the administrators of the learning platform or the tenants of each group will be entitles to award or withdraw under their own criterion this kind of badge to the members of a group. So, manual badges are discretional, no prior requirement is necessary to win one, in contrast to the automatic or by contest badges.

  *

  Gamepoints. Manual badges can convey a certain amount of Gamepoints which will be (de)accumulated to the total Gamepoints score of the user in case the badge is won.

  -

  Badges by contest. A badge by contest will be awarded to the students shortlisted in the first positions of a given contest (Figure 3). The number of winners is set at the moment of creation of the contest.

  *

  Selection of contests. A badge of this sort, differently from the other two types, can only be associated to a single contest. And a contest can only be associated to a badge, too. Because of this, at the moment of showing the available contests, the list will include exclusively those contests not yet linked to a badge.

  *

  Text and color. The creator of the contest can select a text and a color personalized, therefore providing in this way a distinct and exclusive badge to each winner. Furthermore, a student can gain more than one badge in a contest if his/her participations during the challenge are worth of placing him/her at the winners’ ranking positions more than once. Ties are not broken, all the participants will receive the badge equally.

• Labels. Badges can display descriptive labels, so that users can easily follow the topics of their interest.

Once the badge has been created, two final options exist for its state: enabled or disabled. Enabled badges are those openly shown to all the members of a group in the list of available badges (unless the surprise flag has been activated too), students and teachers; a disabled badge is shown only to instructors or administrators of the platform. Even though some students could already fulfill the requirements to have the badge, they will not be awarded until the badge is enabled.

4.2. Views

Within the section Group badges, previously mentioned, a number of possibilities will be given for the development and the viewing of the plugin, for which a varied set of features exist.

-

Creation and edition of badges. These views are used so that the user can create or modify badges, setting their desired parameters as summarized in previous paragraphs. Accordingly, it is possible to produce many types of badges, from the simplest ones with only a few parameters and requirements, up to the more complex end, just adapting to the user’s preferences.

-

Leaders table. This is an ordered list of users according to the badges they possess (total number of badges and their icons). Moreover, looking at a better user experience, an extra functionality for ordering the list by user name, and by number of badges per user as well, has been included (Figure 4). The list is configurable, with the administrator defining how many students are shown in a screen page.

-

View of all badges. A list with all the badges in a group along with a short description of their most important properties (icon, labels, visibility, and type), the creator’s identity and the creation date (Figure 5).

-

Full view of the badge. A view that shows the set of properties of the badge in detail. In addition, it includes the options to enable/disable the badge, to commit/revoke a badge to a student in case it is a manual badge, and a list of the holders of the badge (Figure 6). Had the badge been a public badge, in this view we will be given the option of creating or deleting a certificate for all the users who have the badge. The reception of such certificate is a proof that the certificate is not tied to a group in the social network, but it is instead a unique and own distinction for the student which will continue in his/her profile even after the group in which it was awarded or the person who created it ceases to exist in the platform.

-

View of badges per user. For each user, there exists a view in his/her profile listing the badges won, public and private. The path to this view is through the option “My badges” presented in the user profile, or through the button for quick access to the personal badges displayed in the view of badges for all the group members. Here, one can see the icon, group, and description of the badge, provided the latter exists (Figure 7). In the group badges, it should be highlighted that, if the user accessing the view is not a member of that group, she/he will not be included in the table of private badges neither. Finally, next to the table of public badges, a student is given the option to generate a badge that can be linked to his/her account in Mozilla Open Badges (see Appendix A).

In case a badge has to be removed, the option is available either in the private view of the badge or in the view of all the group badges. It is important to recall the removing a badge implies the removal of the Gamepoints assigned to the badge when this is manual.

Since different types of users exist in the social network, each one with individual characteristics for accessing and viewing the contents, their view of the social network will be obviously different. Thus, users with all the permissions (administrators, instructors, others) will be able to see and edit any type of content, spanning from the values of the fields for the badge to the addition of new badges, the modification of the existing, etc. Normal users (typically the students) will only be able to visit the views wherein the change of badges is not possible, or to badges with private visibility. So, their ability to do changes in the global state of the social network is restricted.

5. Application

As explained in [34], SocialWire implements an SLE that integrates classical learning tools with new modules that allow the implementation of social learning, gamification and a system for social meritocracy. Since the middle of last decade, it has been used as the e-learning platform of several undergraduate and graduate courses at the University of Vigo. For example, it has been used for six years in the Computer Networks course of the second year of the degree in Telecommunication Technologies Engineering, and for four years in the Networking Technologies course of the first year of the master in Telecommunication Engineering.

In this example, we focus on the 2017–2018 edition of the Networking Technologies course, when the latest version of Socialwire with full functionality was available. Although a complete description and analysis of the behavior and performance of the students when this SLE is used in conjunction with traditional teaching is described in [35], following we present a summary for completeness of this work with an example of use of the digital badges systems.

This course has a weekly schedule that last 14 weeks. Classroom activities consist of lectures and laboratory sessions. To encourage self-learning before class and collaborative work, different online activities allows students to gain points (collaborative questions and answers, tasks previous to the laboratory sessions, tests to practice before the midterm exams, etc.). The resulting rankings can be made public to the group. The score of the part of online activities is given in merit points (Gamepoints). The use of the virtual classroom is also rewarded by the automatic scoring of different actions carried out in the platform related to the normal activity unfolded along the term. Maximum values of these experience points (Activitypoints) are controlled by the teachers.

Two modalities of assessment are possible: final examination covering all the material or continuous assessment. In continuous assessment, the final exam represents the 50% and the other 50% is divided into a 40% from two midterm exams (each one a 20%), and a 10% from the merit points obtained by accomplishment of the online activities.

Related to the results obtained in the edition of the course analyzed, of the 32 students that followed the continuous assessment only 5 finally failed the course. The 4 students not engaged in continuous assessment finally dropped out in the course.

Badges

The possibility of obtaining experience points was active along the whole term. Students could get points for many types of activity: posting questions or answers, viewing questions of files uploaded by teachers or classmates, viewing blog posts submitted by teachers, sharing or viewing bookmarks, posting forum threads or replies, viewing forum posts, receiving forum replies from classmates or teachers, adding comments, receiving comments from classmates or teachers, adding likes, receiving likes of classmates or teachers, etc.

All these events were registered, and were used to build a social graph. In this graph nodes represent teachers (label 0) or students (green if they passed the subject at the first (light) or second (dark) exam date and grey if they dropped off the course or failed the subject) and they are connected by an edge if one has interacted (answered, viewed, replied, commented or liked) with a resource of the other. The weight of the edge is related to the points obtained for this interaction. Figure 8 depicts this graph considering a simplified version where the width of each edge is proportional to the sum of the weights of all the edges between the underlying pair of nodes.

The ranking of experience was organized in levels of 50 points. When students level up they obtain badges (cups of different colors) that offer the possibility of choosing a reward: extra merit points (gift badge), extra time (clock badge) or help (book or notes) in the final exam. This motivates students to take part in the optional activities that are rewarded with Activitypoints, and generates continuous monitoring of the subject and a better exploitation of the social learning methodology. All the students that chose the continuous assessment earned some experience points, although with differences in the level of engagement. While four students got more than 100 points, reaching at least the second level cup badge with the possibility of choosing at least two rewards, other eight students did not get the 50 points needed for reaching the first level cup badge and obtaining a reward. Five of of these students were not able to pass.

In addition to automatic badges related to experience points, other badges recognize other types of quality participation of the students along the course: manual badges for good answer in class (microphone badge), manual badges for bug finders (bug badge), automatic badges for good accomplishment of tasks and tests that free students of extra work of reinforcement (joker badge), etc. Showing them publicly on the platform is a way of rewarding attention in class or when reviewing the material of the course, of rewarding interest in contributing to the correct functioning of the software of the platform, of giving visibility to good performers as good references for his/her classmates, etc.

In Figure 9 we show the badges obtained per student.

6. Discussion

Badges can be naturally incorporated into digital learning platforms as part of formal or informal learning activities, and stimulate participation, merit, and visibility for the students. However, the level of engagement of learners into these learning activity depends crucially on how the system of badges and rewards has been designed and on how it is applied, if it is to be used effectively. In this sense, a smart badges system integrated into a SLE should offer the power and versatility necessary to implement a wide range of game mechanics that can be useful in different educational environments, taking advantage of all the academic potential of the SLE.

In this paper, we have reported on the design and implementation of a complete badges system and we have described one example of its use as part of the gamification methodology for informal learning in a master level course. A complete analysis of the behavior and performance of the students of this course was conducted and presented in a previous work, where we found evidence that motivational game mechanisms and reinforcement feedback from teachers or peers turned out to be of key importance to draw the attention of the students and keep them attached to the activities, so that their perception of utility and progress does not decayed as the semester advanced.

At this point it is important to note that almost all the students enrolled each year in the master in Telecommunication Engineering are well motivated students, with successful trajectories in the degree. But their computer networking background is diverse. In the edition of the Networking Technologies course described, only 12 of the enrolled students held an undergraduate degree with specialization subjects related to computer networks. For this reason, the proposed social learning methodology and the game mechanisms employed where enough and useful to offset the differences. Other cohorts of students are different, more heterogeneous in capacity of work, motivation, etc. An example are the cohorts enrolled each year in the Computer Networks course of the degree in Telecommunication Technologies, where we have also used SocialWire as SLE for many years, adapting and enhancing the social learning methodologies and the meritocracy system according to the observations and results we were obtaining. SLE in general and gamification systems in particular must be powerful enough to deal with the diversity of learning contexts.

Other simple, direct consequence of our work is to provide evidence that valuation of the work and performance of the students along informal learning tasks does not imply the deployment of complex assessment stages, or elaborated rubrics. Instead, the opposite can be more effective: the public acknowledgement of merits and a public ranking suffice to send the proper signals to the participants in the SLE. Moreover, the second important conclusion from our work is that this open challenge-reward-badge cycle also sends proper signals to the instructors, since the results can be systematically used to predict early success or failure in the process of knowledge acquisition.