Serious Game as Support for the Development of Computational Thinking for Children with Hearing Impairment

Abstract

Computational thinking (CT) has been a topic of interest in research, not only in the field of computer science, but also in education, since it allows the development of a set of competencies in the child related to problem-solving and decision-making. However, few studies on CT are focused on children with disabilities. Developing computational thinking skills for children with hearing problems is a challenge, even more so when their language skills are limited. Following a methodology for conception of serious games for children with hearing impairment called MECONESIS (Acronym in Spanish, MEtodología para CONcepción de juEgos Serios para nIños con discapacidad auditiva), we designed the serious game Perdi-Dogs for children between 7 and 11 years old with hearing impairment. We considered a set of aspects, such as challenges/learning, control, rules, feedback, interaction, rewards, surprise, communication/language, and fantasy. Perdi-Dogs involves both a physical and a digital interface, specifically a physical board together with digital interaction, by means of a QR (Quick Response) code and vibrotactile feedback system. Perdi-Dogs simulates a real environment, using physical elements able to interact simultaneously with technology. Evaluation was carried out with a group of seven children between 7 and 11 years old from the Institute for Deaf and Blind Children (Colombia). The results showed a high motivation to play for all of the children involved in the experiment.

1. Introduction

Computational thinking (CT) was introduced in 1980 by Papert [1], who developed the concept of constructionism. Based on Piaget’s constructivism, constructionism emphasizes the agency of young people as active participants in the construction of their own learning. CT was popularized in 2006 by Wing [2]. CT has been a subject of interest in studies from several fields in recent years, since it can help children develop skills related to problem-solving [3] and decision-making [4,5]. Therefore, CT is a fundamental skill for everyone, not just those in the field of computer science.

In 2017, Doleck et al. [6] defined CT as an umbrella term related to cognitive skills involved in computational tasks, such as abstraction, algorithmic thinking, cooperativity, creativity, critical thinking, decomposition, reasoning, and problem-solving [7,8,9]. Logo was the first programming language created for children, and it was integrated with a turtle robot. It was based on constructionist learning, which is when learners construct mental models to understand the world around them [10]. Tangible objects are a way to connect active learning with the real world. CT activities are related to a set of steps that must take a certain order for problem-solving.

CT skills can be integrated into class activity, with or without technology. The integration of a physical object can be supported by constructionism theory, where the child can engage in active learning. In addition, the process of knowledge acquisition can be enhanced through serious play [11] to motivate the child during learning, especially if their interaction involves other children, allowing children to try new things, learn from their mistakes, and gain experience in complete safety.

When designing a serious game for children, it is important to consider those elements of the game that meet their needs. A balance is required between meeting these needs and the mechanics of the game. When this balance is not achieved, one of two different situations may occur: (1) if the challenges are too complex for the skills of the individual, a state of anxiety is created due to the game being too difficult, or (2) if the game is too simple, the skills of the player are too far above the challenges of the game, the child may easily become bored and lose motivation. In addition, the interaction with physical objects attracts the interest of the children and can motivate them; however, the interaction digital with physical objects have not yet been widely adopted in learning for children. Social interactions with other children offer a rich source of information for children’s learning not only in language but also in terms of behavioral, cognitive and social dynamics [12].

CT through a serious game can be a helpful alternative in the teaching–learning for children, especially for children with special needs, such as hearing impairment. Designing a serious game requires an adequate methodology, which involves the participation of different experts in the field in order to define objectives applied to the context of use. However, most of the existing methodologies are oriented to a user without special needs. Cano et al. [13] proposed a methodology to design serious games for children with hearing problems, which is used in this research.

Children with hearing impairment have a visual–spatial–perceptual deficit [14], in which the acquisition of reading, writing and social development skills is delayed. They have low scores in evaluations such as problem-solving, logical thinking and reasoning [15,16]. A study by Deveci et al. [17] identified that algorithm teaching for deaf/hard-of-hearing students has a significant effect on problem-solving skills. Children with hearing impairment who have a cochlear implant encode and process information through the cochlear implant. Therefore, they have problems related to information processing and related areas such as perception, learning, memory, attention, and language processing [18]. For children with hearing impairment, it is difficult to solve problems relating to language skills, and they have difficulties correctly assimilating the received information. After receiving a cochlear implant, these children have significant problems with emotional control, initiation, working memory, planning, and organization [19].

Most CT researchers focus on children without hearing impairment. However, a child with a hearing impairment has specific problems, such as problem-solving, reasoning, communication, socializing and planning. Therefore, CT can be an alternative to help in the development of problem-solving, reasoning, communication, socializing and planning to be achieved by working with a serious game through tangible objects. The research question for our study was the following: how can computational thinking skills be developed through a tangible serious game for children with hearing impairment? We developed Perdi-Dogs, a serious game for children with hearing impairment, which aims to help to develop CT. We used MECONESIS (Acronym in Spanish, MEtodología para CONcepción de juEgos Serios para nIños con discapacidad auditiva), a methodology proposed in 2016 by Cano et al. [13].

2. Background

2.1. Computational Thinking

The term CT was introduced in 1980 by Seymour Papert, who developed a game called Logo that featured both physical and digital interactions. The distinctive feature of Logo was that, in order to generate graphics with the movement of a turtle (physical interaction), the user had to establish the movements by means of instructions. In developing this idea, Papert was inspired by Piaget’s theory of constructivism [20], in which interactions with the environment and working other children cause a child’s individual intelligence to develop. Papert adds something new to Piaget’s theory, i.e., play and affection as important elements in the learning process [21].

Some studies have linked CT to critical thinking [22], further defining it as a new method of solving a problem using computer science techniques. These authors define critical thinking as a “skill or competence, by which the individual transcends, in a deliberate manner, in order to reach reasonable conclusions that can be corroborated using valid information”. It is a way of thinking that makes it possible to give rise to multiple solutions. In 2016, Soh et al. [23] indicated that CT complements critical thinking as a way of reasoning to solve problems, make decisions and interact with the world. CT therefore involves techniques such as abstraction, decomposition, algorithmic design, generalization, evaluation and interaction with the computer.

Soh et al. [23] and Bennet et al. [24] have linked CT with creativity, since it implies certain levels of creative thinking in formulating solutions. In 2015, DeSchryver and Yadav [25] defined creative thinking as “a cognitive activity that comprises several subsets of these components of thinking skills that are mediated by the most aesthetic components of traditional creativity.”

2.2. Children with Hearing Impairment

Studies have demonstrated that deaf children do not have the same level of reading skills as hearing children [26,27]. Cochlear implants represent a form of intervention performed on children and adults with profound deafness, an option that provides access to sound using electrical stimulation to the auditory system. A child with cochlear implants must therefore acquire knowledge about sound, sound sources, and correlations between objects. The process therefore usually begins with learning to listen and then to speak and write.

Researchers such as Marschark and Everhar [28] and Zevenbergen et al. [29] found differences between hearing and non-hearing children in problem-solving ability. The resolution of the problem as indicated by Simon [30] is related to the information processing system, the subject and the environment. It is important to construct a representation of a task to solve the problem [31,32]. Therefore, representation of the task, according to some authors [33,34], involves four aspects: (1) understanding the problem, (2) what should be achieved upon solving the problem, (3) the possible actions for task solving, and (4) knowing what is not allowed and the consequences of choosing a certain action.

A study was conducted in 2010 by González et al. [35], which involved planning a problem-solving situation for children of 4–5 years old with a hearing impairment who attended the Institute of Blind and Deaf Children of Valle del Cauca (Colombia). It was demonstrated that the children correctly performed the proposed task because the researchers were careful not to ask the child for verbal explanations, since actions rather than verbal skills were required. A study conducted in 2006 by Ebrahim [36] compared creative thinking skills and reasoning for deaf and hearing children, and found differences in creative thinking skills. The hearing children had better results in completing patterns, analogical reasoning and serial reasoning. The deaf children, however, had higher scores in spatial visualization.

2.3. Tangible Serious Games and Computational Thinking

Learning and teaching concepts related to CT gives rise to various challenges [37]. Tangible interaction enables end-users of systems to interact with the digital world by means of manipulation of real-life physical objects, instead of the traditional use of mouse and keyboard as input devices. This type of interaction can enhance learning, decision-making and retention of concepts, among other skills [38]. In addition, the interaction with physical objects could provide learning benefits, such as close links between activity and cognition; shared spaces that can allow users to readily monitor each other’s gaze; increasing the visibility of actions; facilitating increased awareness; enabling users to manipulate physical artifacts outside the interactive space to help social organization and planning; physical activity with digital effects may increase reflection in children; and interacting with various physical artifacts can increase the playfulness of learning [39,40].

Schez-Sobrino et al. [41] proposed RoboTIC, a serious game based on gamification and augmented reality that facilitates the learning of programming among students in lower levels of the education system. In 2020, Elshahawy et al. [42] developed a serious game called CodaRoutine to teach problem-solving skills to children with autism, where the game introduces sequential programming concepts. Cagin et al. [43], proposed a serious game for developing CT and learning introductory computer programming; the game is composed of action commands (such as go forward, turn left), which have a direct effect on the character represented by a robot, and programming commands indirectly supported by solutions developed by the player (such as repetition or making a decision on a condition). Another serious game proposed is Pic2Program [44], which teaches children the basic concepts of CT in a playful way. The avatar is controlled by a specific sequence of symbols. Therefore, the learner has to develop such a sequence suited for solving the maze. The different concepts involved in Pic2Program are algorithm, modularity, control structures, representation, hardware/software, design process and debugging. The interaction takes place through a camera with an android device to recognize the sequence of symbols that corresponds to a set of instructions to manipulate the character in the digital scenario. A tangible serious game was proposed by Berta et al. [45] to teach science, technology, engineering and mathematics (STEM) through smart objects, according to the Internet of Things. Such objects are called iblocks, which the learners can manipulate. An important aspect is that the environment supports the definition of game rules, so the users are stimulated and invited into the exploration, competition, and collaboration.

3. Methodology

Our study was qualitative and used a methodology called MECONESIS, which was proposed in 2016 by Cano et al. [13], and originally aimed to help to design serious games for children with hearing impairment. MECONESIS integrates four stages: analysis, pre-production, production and post-production (Figure 1). The analysis stage involves exploring and identifying users’ characteristics. The context is also analyzed, and the needs of children with cochlear implants are identified. Some relevant aspects are analyzed, such as pedagogical/playful strategies used by the teacher within the classroom, communication of the child with his/her peers, learning style, the child’s interaction within the classroom, and communication strategies used by the teacher with the child, among others. In this first stage, a set of instruments is used to identify aspects that can be addressed, taking into account that the children have a hearing impairment. The pre-production stage is related to the design of the game interface, where design guidelines focused on hearing-impaired children are considered based on the information collected in the first stage. The production phase is related to the implementation of the serious game. The production phase integrates components necessary for its operation and is more oriented to the developer, who must develop the game according to the different views and scenario models captured by the designer. Finally, the post-production stage is related to the evaluation of the serious game.

3.1. Analysis

This stage was based on an analysis model proposed by Cano et al. [46], where evaluation methods are applied to identify the child profile with hearing impairment, including direct observation, interviews and inquiry. These methods collect information about the child such as feelings (emotions), behaviors, motivation and limitations. The participants were children and teachers from the Institute of Blind and Deaf Children, Valle del Cauca (Colombia). Seven children participated (4 girls and 3 boys, with mean age = 10.3, and SD = 1.5), all of whom were enrolled in the third academic grade. Three teachers also participated. Other academic grades were not considered, as they included students in the very early stages of verbal and written language development. The group of children selected comprised three boys with hearing aids, three girls with a cochlear implant, and a hearing girl. For each child, informed consent for participation in the study was signed by the parents, at which point the conditions and protocol of the experiment regarding the publication of data were explained.

In this stage, evaluation methods were applied to understand a set of aspects of the user-child’s profile [46]. In this process, the user, product, activity, and context of use were included, in order to explore and identify the human factors, since not all users have the same level of experience of the game or learn at the same speed. Therefore, four qualitative methods were applied: (1) direct observation, (2) interviews, (3) thinking aloud, and (4) Wizard of Oz. The order of applying the evaluation methods is important since each method was designed based on the results of the previous method.

The first method applied was direct observation in the classroom, where it was observed that children with hearing impairment were at low reading and writing levels compared to a hearing child at the same grade level. This means they employ the phonological code less than hearing children due to the difficulty of linking the so-nest with the letter. However, they can use the phonological code when graphic information is presented to them, although they will never reach the level of a hearing child. This is because the deaf child is faced with the difficult and complex task of memorizing vocabulary, and when they find a new word they will not understand its meaning until it is visualized graphically. It was also observed that the children worked in groups, since they helped each other in the different activities assigned. In fact, the great majority of decisions that they made were first approved by the other group members. It was also observed that they worked with tangible objects within the classroom to facilitate their relationship with the real environment. Moreover, teachers used a visual learning style, using symbols, graphics, images and physical elements to help them relate concepts. During observation, the children were found to have a problem with spatial orientation in respect of how to identify laterality (to the right of, to the left of), depth (above, below), and anteriority (in front of, behind) [47,48].

The second method, an interview composed of five questions, was carried out with three teachers from the institute. The objective was to inquire about how much knowledge teachers had of the term CT. The questions were as follows: (1) what do you understand by CT? (2) What teaching/learning strategies or methods do you use to stimulate CT in the child? (3) What CT tasks involve the most difficulty for the child? (4) Do you have a way of measuring the CT of the child? (5) Are you familiar with Scratch programming and have you used it?

Some of the answers given by the teachers showed little knowledge related to CT, since they related it to mathematics, reasoning, and mental calculations. It was also observed that teachers did not know that CT can relate to other types of thinking, such as critical and creative thinking. Therefore, they related most of the activities that they worked on with the children to mathematics or mental calculations. It was also found that the means of evaluating the children was through written tests or participation in the classroom. All three teachers were unaware of the Scratch tool.

The interview explored the children’s learning style. It showed that children with cochlear implants require more effort to acquire speech compared to children with hearing aids. For children with a cochlear implant in the early school years, learning is more visual and they assimilate new ideas better when they can see. The children with a cochlear implant must learn to identify sounds, which is more difficult when the sounds are weak and they are required to discriminate what they hear, and when there is a lot of environmental noise, it is difficult to understand the meaning of words. Therefore, they learn to be more visual than auditory, but as they progress through the school year, the auditory canal becomes more relevant, so the teacher always addresses the child verbally with visual elements as support. In early years, the children use a method for language acquisition called the invariant method [49], which establishes an adequate word/image-meaning relationship.

The third method applied was thinking aloud. The group of children were asked several questions to find out about their likes, attitudes, and use of technology. This method is significant since it consists of talking with the users and observing them carefully. The following questions were asked: What is your favorite activity? What is your favorite color? What is your favorite animal? What do you do in your free time? Which technological devices have you interacted with? What would you do to improve your math results? Showing no inhibition, the children gave answers such as that the boys liked football, the girls liked skating, and their favorite colors were yellow, green or red. Most answered that dogs were their favorite animal, with some indicating tigers or elephants. In their free time, most said they watch TV or play with other children. Some commented that they play on their tablets, and others play games in the street. Finally, to the question of how they could improve their math results, they answered that they should pay more attention and study multiplication tables.

Finally, a fourth method (Wizard of Oz) was carried out to evaluate experience in the use of technology, in which the children interacted with a mobile device (a tablet with Android operating system without an internet connection). The applications used by the children were games that had previously been installed in the devices, such as Rosita Fresita’s Bakery [50] and Tiny Puzzle [51]. It was observed that the girls preferred bright colors such as pink, and the boys were attracted more by the kind of interaction using a mobile device. All the children were familiar with interacting with a touchscreen. In a study conducted by Vatavu et al. [52], the touch gesture involving children according to their age was analyzed, and it was observed that they have smaller fingers, less fine motor control, less manual dexterity and less experience with technology than adults. Furthermore, Antony et al. [53] investigated the touch patterns of children aged between 7 and 16 years. They observed touch behavior related to “touches located within the vicinity of the previous target”, noting that children tend to touch the screen additional times before noticing that the target has changed. These screen touches can change depending on the size of the mobile device.

It is important to mention that a child’s ability to concentrate is usually limited to approximately 30 min [54], so each of the above-described activities was limited to a maximum time of 30 to 40 min.

Figure 2 shows the relationships between CT, children and serious games.

3.2. Pre-Production

The pre-production stage is related to the design of the game interface using the information gathered. A serious game is proposed in which the interaction is cooperative. When we talk about a game, we do not just mean how useful and usable it can be; we must decide how to link aspects of the game such as story, character design, what the player feels and the game’s rules, among other factors. Therefore, the aspects to be included in a serious game are challenges/learning, control, rules, feedback, interaction, rewards, curiosity, communication/language and fantasy.

Based on the information that we collected in the analysis stage, we designed the following story: “Lupe, a Golden Retriever, lived in a beautiful house that had a huge courtyard at the back. Lupe always went out in the afternoon with her three doggy friends, Tobby, Tina, and Firulais. However, one afternoon, a cat called Kira appeared, and Lupe and her three friends thought it was fun to chase her, leaving their houses far behind. When Lupe and her friends stopped chasing Kira, they realized that they did not know how to get back home. So, Lupe and her friends need your help to get home”. The game story is related to the aspect of fantasy, which represents an activity that is separate from real life. Therefore, the scenario and characters involve the player in social and imaginary situations.

Another aspect that we considered was the interaction. Presently, children grow up immersed in technology, as was evidenced by the answers provided by the children in our study. They thus bring with them experience in using technology. Moreover, including technology as a learning material can motivate them. Therefore, we integrated both physical and digital environments. In order to include the curiosity aspect, we used a QR (Quick Response) code through a physical object and the mobile device. Therefore, the digital interface is a mobile application.

Two physical boards were designed (Figure 3), both composed of four dogs that are lost and must find their way home, thereby enabling four children to play cooperatively. Both boards (Figure 3a,b) were presented to the children; the one they liked best was the second prototype (Figure 3b). To evaluate the tokens (Figure 3c), three children were selected such that they understood the symbolic representation of each token.

Figure 3c shows the dog skills of Super Bark, Maximum Dig, and Mega Leap, which are the obstacles that represent challenges for the child in making the correct decision to help the dog get home. These challenges are Automobile, Stray Animal Control, and Kira. The rewards are represented as dog lives and are associated with bones: for each challenge completed by the child, a bone is awarded.

Once the physical board (Figure 3b) was selected and each token (Figure 3c) was evaluated, gameplay began, with each child advancing a number of steps during each turn. It was noted that the children found counting and orientation difficult. It was also noted that the high number of squares on the board caused a degree of visual strain. Figure 4 shows the flow diagram of the serious game “Perdi-Dogs”.

In the design of the physical board game and the mobile application, various design recommendations proposed by Cano et al. [55] and Tania et al. [56] were considered: (1) texts should be supported by pictograms; (2) a character/hero should be considered that takes account of both the gender and age of the child (Figure 5); (3) an interface ought to use the language and concepts familiar to the user; (4) metaphors should build on children’s existing knowledge so that they can easily see what to do and predict the outcomes of their actions; and (5) the design of the representations should allow children to easily see how they relate to the world.

The only token that the children did not understand was the Super Bark representation. The challenge of the child (player) is to help the dog to find its way home. On the way, they will meet obstacles that the child must overcome with the help of the dog skills. Therefore, the child must correctly select the skill to overcome the obstacle in question.

The rules aspect was also considered. Several rules were established for the serious game (

Table 1. Rules for play.

Rule	Description
The game must be supervised by a teacher.	Children are not very autonomous in their decisions because of their disability. Therefore, they always require teacher approval.
The game may be played by between one and four players.	The child can be competitive with other children during the game.
The teacher can control the number of obstacles that arise in the game; the more obstacles, the greater the level of difficulty.	The teacher has control over the game. Therefore, the game has a dynamic behavior, in which the teacher decides when the obstacles can change place and the difficulty level can increase.
The number of lives that the dog has reflects the number of correct actions and mistakes made by the child.	According to the set of competencies or skills of the player to present the challenges.
For the automobile or animal control obstacles, either the Maximum Digging or Super Leap abilities should be used, while for the Kira obstacle (a cat), the Super Bark ability should be used.	The child must select the correct skill card to advance in the game. If the child does not select the correct card, they are penalized by losing lives.

). The rules describe the structure of the objective of the serious game, how to interact with others and with each element and which behaviors are considered to develop a set of skills related to CT. The rewards aspect of the game is represented by the number of lives of the character, which is represented in bones. When a player makes a mistake, a life is subtracted.

In addition, the instructions concerning direction (left or right) and the number of steps to be taken are given from the mobile application. It was therefore proposed to design a roulette wheel that randomly determines the orientation and the step count (Figure 6).

Feedback is important in the problem-solving process. In order for children to understand the problem and what strategies to use, as Prensky [57] has indicated, the player must know the rules of the game, so they know what may be done and what the restrictions are. Similarly, they must learn about the consequences of their actions, which in this case relate to the number of lives that the dog is given. Therefore, the feedback helps the child to identify what is allowed and what is not. Vygotsky [58] states that the child needs language support to control their environment.

In the design of the serious game, five design recommendations for children proposed by the Norman Group [59] were also considered: (1) give children clear and specific instructions by explaining the goal of the game; (2) tailor instructions to the child’s level of understanding; (3) use existing mental models and knowledge about the world to help the children accomplish tasks; (4) reduce cognitive load by designing self-explanatory interfaces and preventing errors; and (5) make instructions clear and specific but not overly prescriptive.

Table 2. Computational thinking (CT) skills using the serious game.

Action	CT Skills	Description
Storytelling, sentences to move the character, e.g., “move to Lupe 2 steps”	Communication Language	The teacher communicates verbally with the children so that they listen to the story. The child must also read the instructions on the mobile app.
Moving the character considering the number of steps and orientation	Problem-solving Planning	Children must make decisions about which skill cards are appropriate to advance the game. They also learn orientation, since many of them have spatial orientation problems. Planning is very much about following the proper instructions so that the dog can find its home.
Selecting the skill card	Problem-solving Reasoning	The child must select which skill they need to overcome the obstacle. Therefore, they must analyze and evaluate the obstacle.
Competition	Socializing	Each child competes, as each child controls a character and must find the best way to get home.

shows the relationship between the serious game and CT skills.

3.3. Production

At this stage, as a result of the observations made, the new symbolic representation of Super Bark was proposed (Figure 7b), and the physical board was adjusted accordingly (Figure 6a), as were the obstacles (Figure 7c). The interaction with the physical board was then integrated with a digital interaction using a QR code (Figure 7b), and interfaces for the mobile application were designed (Figure 8). The vibrotactile system feedback used in the interface consisted of direct tactile feedback through the mobile device. Every time the child made a mistake in the selection of cards, the device vibrated. In addition, the child received visual feedback, which corresponded to a sad face.

To move around the physical board, at each step, the child received instructions about movements. A roulette wheel therefore contained all steps: turn right, turn left, go back and go forward (Figure 8d). The physical board interacts with a mobile application using a QR code. The mobile application enabled the children to go about performing each movement on the board. The mobile application (Figure 8) thus began by selecting the character (Figure 8b). A roulette wheel (Figure 8c) then randomly assigned the movements that the character (Figure 8d) should make through the physical board.

At this stage, as a result of the observations made, the new symbolic representation Super Bark was proposed, and the physical board was adjusted accordingly (Figure 7b). The interaction of the physical board was then integrated with a digital interaction using a QR code (Figure 8f), and interfaces for the mobile application were designed (Figure 8). The vibrotactile system feedback used in the interface consisted of direct tactile feedback through the mobile device. Every time the child made a mistake in the selection of cards, the device vibrated. In addition, the player received visual feedback, corresponded to a sad face (Figure 8g).

The digital interface was a mobile application, with which the child could interact using a QR code, through which the application indicated the steps to be followed. The application would therefore ask the child if there was an obstacle that prevented him or her from advancing in the game (Figure 8e). The child had to select the obstacle presented and choose the appropriate ability to overcome the obstacle. Alternatively, if no obstacle was presented, this was indicated with a “No”. If carried out correctly, the application would show a happy face; otherwise, it would show a sad face. According to the number of incorrect selections, lives would be taken away from the respective dog. Each life was represented by a bone. At the start, the dog had five lives; in other words, the player had five opportunities to make a mistake.

3.4. Post-Production

This stage corresponds to the evaluation of the game. Seven children from 7 to 11 years old participated in the evaluation. The serious game was designed to be played by 1 to 4 players. According to the observations made in the first stage, the children worked in groups of three or four. We therefore decided to evaluate the game by forming two groups: one group with four and another with three players. They were shown all the elements of the game (Figure 9a), with the characters printed in 3D-Pen (Figure 9b).

The goal of the evaluation was to assess the child’s experience when interacting with the game and other children. Therefore, we decided to play the game three times with both groups and to observe each experience.

3.4.1. First Round

In the first experience of playing, we observed that children did not know how to move around the board, due to the hexagonal shape of the board itself, and did not know what to do when they saw an obstacle. Meanwhile, the hearing child had no problems understanding.

During the first round of the game, the teacher intervened to explain the rules of the game, what children should do when the application told them to move in a certain direction, and what to do when they encountered an obstacle. By the end of the first round of play, when a winner safely guided their dog home, some children already had a better understanding of the game. The children were asked if they wanted to continue playing, and they answered positively, so a new experience was started.

In this first experience, children did not sufficiently understand the relationship between lives and bones, since the characters began with five lives, represented by bones, and every time they made a mistake they were punished by losing a bone.

Regarding the CT skills related to the serious game (see Table 2), the child with cochlear implants had problems with language development. Therefore, it was necessary to stimulate the child by having them understand the story by listening. In addition, rules helped them to inhibit behavior regarding what they should do and should not do to win the game. However, some children did not understand everything, but relied on what they saw from the other children playing to understand the movements and instructions.

3.4.2. Second Round

The children showed greater independence in the second experience. They had already employed strategies for the use of the application; for example, one spun the roulette wheel to determine how many steps to take, and another used it to find out which direction to move in. When the children encountered an obstacle, they looked at the challenge with fear, because if they chose the wrong power, it would take away their lives and keep them from the main objective. In this second experience, the children learned and understood how they should interact with the roulette wheel, as well as the movements and the decisions to make when an obstacle was presented to them. However, the spatial orientation aspect was still not very clear, so when the application showed the representation to the right, the children sought the teacher’s approval to confirm whether or not it was correct.

It is important to mention that although it is a competitive game, the children supported each other in moving forward. For example, when a child was not sure if the skill card was correct, the other children would indicate that it was. They also communicated with each other, for example, if a child was not sure of the right/left orientation.

In this second experience, the children had more control of the game. At the same time, they understood the instructions given by the application, the number of steps and orientation in which to move the character. Therefore, some could differentiate if the instruction they were going to perform helped the character advance towards the house, or just moved them away from it.

3.4.3. Third Round

The children were more confident in their decisions during the third experience. They no longer validated their decisions with the teacher. The children supported each other even though it was a game in which they competed. In all three experiences, the children expressed interest in playing again when the game was over, and they supported each other to reach the goal. It is important to mention that it was more difficult for children with cochlear implants than for those with hearing aids.

After interacting with the game three times, we applied the user satisfaction evaluation questionnaire (USEQ) proposed by Gil-Gómez et al. [60]. However, some questions were changed to elicit a better understanding of the children’s language. We asked the following five questions:

Q1: Did you enjoy your experience with the system?

Q2: Did you have to make a big effort to complete the activity?

Q3: Did you understand the activity you were required to carry out?

Q4: Did you feel discomfort during your experience with the system?

Q5: Are the images provided by the game clear?

Children’s scores were also modified, and a Likert scale was used featuring three emotional states using emoticons for sad, happy and neutral.

Table 3. Scores FOR questions in the Questionnaire.

Question	Mean	SD
Q1	3.00	0.0
Q2	2.85	0.4
Q3	3.00	0.0
Q4	3.00	0.0
Q5	2.85	0.4

shows quantitative results, where emotional states were related with scores ranging from 1 (sad) to 3 (happy).

The first question produced happy faces from everyone, indicating that all the children understood the activity to be performed. To question Q2, “Did you have to make a big effort to complete the activity?”, only one child expressed that he did not enjoy the game as it was very difficult for him to understand. The rest of the children responded with happy faces. To question Q5, “Are the images provided by the game clear?”, one child answered with a face in a neutral emotional state. The rest of the children answered with a happy face. In addition, the ready participation of the children interacting with the mobile application and the board could be seen, since they supported each other whenever a child did not understand the obstacles or skills well. The use of the QR code in the mobile application for them was something new and motivated the children to participate in the game. It was also observed that when playing in a group, they did not compete. On the contrary, they supported each other, and the teacher took on the role of supervisor.

4. Discussion

For the design of the skills and obstacles cards, an evaluation was carried out as to whether or not the children understood the action each image represented. Another problem was identified, related to the number of lives assigned to the dog. The application shows the number of lives as represented by bones but does not represent that number of lives through the dog character. It was found that the children often failed to see the life bar located at the top of the application and did not completely understand what it represented. As a result, they did not see that the number of lives or bones decreased as they made mistakes.

The application works in such a way that it gives a set of instructions to the child, which correspond to the movement that must be made: go forward, go back, turn left, or turn right. Each of these movements is represented by the direction of the arrows. When the children with a hearing impairment began to play, they were confused as to where to move and what to do when faced with an obstacle. A key aspect in the child learning from experience is the feedback that can be provided each time they make an interaction between the physical and digital devices. However, the presentation of the go back symbol was not very clear to some children, nor was the representation of go forward (Figure 10).

At the beginning of the game, children were asked the nature of the problem to be solved. The problem involves finding the correct way home for each lost dog. There is thus a set of instructions, but the children do not use these. Instead, the application recommends each instruction that the child should execute so that the dog can get home, with the player making decisions about how to help the dog to overcome obstacles along the way. To overcome an obstacle, the child must select the correct skill, and the application validates whether or not it is the correct one.

It is important to mention that solving a problem is related to creative thinking. According to Álvarez [61], for a child to be able to solve a problem, they must perform the following steps: first, they must read or understand the problem, while keeping in mind the relevant elements that can help the solution. However, most of the children with hearing problems have deficiencies in reading, so it is up to the teacher to explain to them the story of the game and what problem they need to solve. This is why a dog was chosen as the main character: since the character is framed in a real context, this can make it easier to reach the solution.

On the other hand, Arfé et al. [62] indicated that children between the ages of 5 and 7 undergo rapid changes in their cognitive functioning, and CT involves a set of higher-order cognitive abilities, such as (1) analyzing problems and decomposing them into smaller parts, (2) planning a sequence of steps or instructions for the solution, and (3) recognizing errors in the solution and fixing them. Therefore, in the serious game, the children follow a set of instructions that corresponds to the steps the character must follow to get home. The instructions are constructed as a sentence, in which the child must select the direction and number of steps, such as “Move Lupe 5 steps to the left” (Figure 11), representing a sentence for the child to read. However, in reading, the children had problems and needed support from the teacher to read the sentence; there were even children who counted with their fingers.

In addition, to allow the children to learn to recognize errors, the application asked them whether or not they had encountered one of the three obstacles. If children had found one of them, they had to tap the screen of the smartphone to indicate which of the three they found, or click on “No” if none of them were found. However, if the children found an obstacle, they had to tap on that obstacle, and the device’s camera would open at the bottom with a text indicating that they should choose the skill that they plan to use to overcome that obstacle. The skills cards have a QR code on the back that children needed to show to the camera, so that the game would know what skill the child was using. If the child responded correctly, according to the established rules, the next screen would show a happy face in green, indicating that they responded correctly. If they responded incorrectly, the screen showed a corresponding sad face and as a result, their dog will would lose a life. When children responded incorrectly, points were removed, where the points were associated with the dog’s life. When a life was taken, their facial expression showed sadness, and they sometimes did not understand why this occurred. However, if they responded correctly they would be happy. The reason for these two emotions is based on the way they are used to being graded. However, they must learn about the consequences of their actions, which in this case relates to the number of lives the dog is given. Therefore, the feedback helps the children to identify what is allowed and what is not. As the children gained more experience in play, they learned to better recognize mistakes and how to fix them.

It was also observed that in the analysis of the problem space, designing a multi-step plan required cognitive control over immediate and impulsive responses [63]. In the game, some control processes were needed, such as inhibitory control, in which the child should not respond until it was his/her turn. This proved to be one of the aspects that required the most effort at the beginning. Another control process was planning: as they gained more experience playing, they acquired more autonomy to make decisions and create strategies to reach the goal. It was also observed that the feedback for the child for each of their actions in the game ought to be immediate, and they should understand if the feedback they receive is positive or negative.

Complementary existing research [64] shows that experience with physical, tangible environments can significantly improve children’s working memory and inhibition skills. In the three experiences obtained with the serious game, the first experience was difficult because the children had to understand that it was no longer a single environment with which they had to interact, and to which they were accustomed in the classroom. They also had to learn how to interact through the digital environment. However, their biggest difficulty was not the digital environment, but rather the physical board and the different rules and rewards they got if they correctly selected the skill corresponding to the obstacle. However, they were motivated and were curious about the interaction with the QR cards.

5. Limitations

The validation that we performed had some limitations—the number of children, and the fact that and many of them did not possess a fluent vocabulary—so a longer questionnaire could not be applied to evaluate several aspects of user experience. There were also other limitations of the game, such as the short duration and a lack of long-term follow-up. In addition, there was a lack of related information on CT with hearing-impaired children. There was also insufficient information about how to evaluate the development of CT skills in children with special needs.

The serious game also has limitations, as it can only develop limited CT-related skills in hearing-impaired children. Most of the existing research we found related to problem-solving through a set of programmed block instructions, but these studies applied to hearing children. Thus, hearing-impaired children have differences in information processing compared to hearing children.

The methodology that we used is limited to serious game design, as it does not consider the integration of tangible objects. We would like to consider this as future work.

6. Conclusions and Future Work

Children with hearing impairment must learn how to listen before they can learn to speak and write. Teachers today are using play-based strategies in the classroom for learning cooperatively. In our study, a serious game for children with hearing impairment was proposed as a support tool in the acquisition of skills related to CT.

The serious game included the following aspects: (1) formulation of the problem in such a way that the use of a physical interface and a digital interface is used to help solve it; (2) the child must logically organize and analyze the correct path for a dog to get home; (3) the data are represented through abstraction, such that the simulation used is a physical board where the child performs the movement of the physical dog token; (4) the child must carry out a series of steps or follow a set of instructions presented by the mobile application; (5) in turn, the game analyzes whether or not the correct instruction was given to move towards the end goal.

Serious games allow the development of skills related to CT in a competitive manner, since the game allows more than one child to play at the same time. Being a competitive game, it allows children to gain confidence in playing and be competitive with others; therefore, they no longer require teacher approval. Children must solve a problem relating to a situation in which a dog gets lost and must find its way home. Therefore, to solve the problem, children must make correct decisions to retain the most lives and win the game.

Although our game had to have a winner, children did not compete, but supported each other in their decisions. In addition, as the children experienced the game again, changes were observed in their enhanced ability to plan and select skills according to the obstacles faced. This indicates that the game can help children to develop a set of skills related to CT.

In future work, it is hoped that serious tangible play will be applied to a larger group of hearing-impaired children to assess CT competencies quantitatively.