Teaching HCI Skills in Higher Education through Game Design: A Study of Students’ Perceptions

Abstract

Human-computer interaction (HCI) is an area with a wide range of concepts and knowledge. Therefore, a need to innovate in the teaching-learning processes to achieve an effective education arises. This article describes a proposal for teaching HCI through the development of projects that allow students to acquire higher education competencies through the design and evaluation of computer games. Finally, an empirical validation (questionnaires and case study) with 40 undergraduate students (studying their fifth semester of software engineering) was applied at the end of the semester. The results indicated that this teaching method provides the students with the HCI skills (psychology of everyday things, involving users, task-centered system design, models of human behavior, creativity and metaphors, and graphical screen design) and, more importantly, they have a positive perception on the efficacy of the use of videogame design in a higher education course.

1. Introduction

Human-computer interaction (HCI) courses often use the curriculum of the Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI) as a foundation [1]. As reported by Greenberg [2], this curriculum should not be used directly when it comes to an individual course within a career, rather as a complete curriculum to train HCI specialists.

In the software engineering program in the School of Telematics at the University of Colima, HCI is an individual course taught in a project-based approach to engage students in research of real problems [3], thus providing students with the basic knowledge of HCI and the different areas that converge in this field is ideal. Computer games (also called video games) are a rich opportunity to bring this approach into a traditional HCI course.

In this paper we present two main contributions: first, we present a proposal for using the design and evaluation of computer games as a learning tool to teach HCI to undergraduate students, and, second, we provide an empirical validation of this proposal to explore students’ attitudes to understand if the students believe that using video games allow them to learn higher education skills. The objective of this research was to explore HCI students’ perceptions regarding the use of game design to acquire the needed skills in the subject, and the research question that we aimed to answer was: does the use of videogame design improve the students’ perception of their learning?

2. Teaching Human-Computer Interaction

A report published by Hewett et al. [1], in addition to giving a formal definition of human-computer interaction, proposed a syllabus and contents for HCI courses in which students could understand the multidisciplinary nature of the field, leaving an open range of topics that interact in the academic training in HCI.

Many years have passed since the publication of that report and the research in the area has grown a lot. However, until recent years, the conclusions were the same. As Churchill et al. [4] states, there is still no agreement on the range of topics that integrate the area and on how to teach HCI courses, how much fundamental theory must be presented, and how much practice students should perform; there are divided opinions about it, and they are investigating the philosophies and best practices to update the ACM 1992 curriculum proposal and support the present and future of HCI education [1].

Despite the lack of agreement on the percentages of theory versus practice, there is a point where most converge since that first report [1]: the development of human-computer interfaces is a matter of engineering and design, and, the subsequent learning of theoretical content, is enriched with the cooperative experience and can advance to a more mature level when students are required to solve the problems of real-life projects. Hartfield et al. [5] also mentioned that a combination of readings, classes, and written assignments is appropriate for teaching topics such as solutions to typical design problems and theories of human skills that are related to HCI; however, they also considered that these activities do not help HCI students to acquire practical skills in the design of technological solutions, and recommended hands-on exercises be carried out with real projects so students can learn by doing, as suggested by Pastel [6]. His work also mentioned that learning through real projects in collaborative groups is becoming the standard for the teaching of the HCI, and he made the observation that the students prefer this method of teaching since they can learn by doing and carrying out a meaningful project with their projects designs. Pastel [6] also made an essential contribution by adding research techniques to work, this was because, as he mentioned, there is a significant similarity between the scientific research process and the software development process. In addition to being research projects, students can use unique HCI devices or implementations.

Lorés et al., Koppelman et al., Reimer et al., Solano, and Urquiza-Fuentes et al. [7,8,9,10,11] also used a combination of theory and projects with real-life applications using user-centered design methodologies. In fact, Lorés et al. [7] combined the teaching of formal content with the development of a real-life project in a compulsory course within the HCI curricula at the University of Lleida (Spain). The work of Koppelman et al. [8] consisted of providing projects with a realistic context, inviting people from industry to serve as clients for the student projects and allowing teams to evaluate their designs with real users. Reimer et al. [9] used the studio-based approach to teach competencies in designing interactive objects in an HCI design studio course. Solano [10] presented an analysis of the teaching experience in an HCI course at the Universidad Autónoma de Occidente of Colombia and identified that problem-based learning is a good way to construct knowledge in real scenarios that can be improved. Urquiza-Fuentes et al. [11] presented a study to test the effect of using practical exercises in a human-computer interaction course, and they found that realistic projects are a viable approach to teach HCI and that students involved in this approach are significantly more motivated.

On the other hand, as mentioned by Serrano-Cámara et al. [12], it is essential to keep the students motivated, especially in generating intrinsic motivation [13] (which refers to doing something because it is inherently interesting or pleasant) because motivation is a central factor in learning and, as mentioned by the authors, improves results in knowledge and creativity. Urquiza-Fuentes et al. [11] suggested that collaborative learning encourages the use of high-level cognitive strategies, critical thinking, deep learning, deep understanding, and positive attitudes towards education and teamwork, according to the research that real-life project development combined with collaborative learning approaches keeps the students motivated; therefore, it turns out to be a positive strategy for teaching the HCI.

3. Learning with Games

The last report from the Entertainment Software Association [14], showed that the videogame industry has maintained a steady growth and a prominent place in the market, being one of the most successful application domains in the history of interactive systems [15].

Video games have not only been used for entertainment but also as an educational tool since the design of games for education provides valuable help in skills acquisition [16]. The design of video games with and educational purpose is known as serious games, a relatively new discipline that combines learning design with game mechanics and logic [17].

Williamson et al. [18] argued that video games are a compelling context for learning because it is possible to create and interact with virtual worlds to develop situated and contextualized understandings. Robertson and Howells [19] mentioned that there is research that argue to give players a mental exercise that generates a series of cognitive abilities, such as the planning of strategies for the resolution of problems, and also mentioned that not only the use of games contributes to learning, rather, they suggested the benefits of designing and building games as part of learning strategies since it involves students and education, and thus becomes an active experience.

In addition, Skrzyszewski et al. [20] remarked on the benefits of including the design and construction of video games as a learning strategy: this activity creates the opportunity to develop and increase creativity, technical skills, and the ability to work as a team. In addition, students can acquire an update on state-of-the-art technologies and techniques used in the process, and gain practical experience in different areas of computing. Although there exist arguments against learning with games, centered upon the lack of empirical evidence to support their effectiveness [17], we found that according to [21,22,23], there is pedagogical support for learning through the development of video games and that, in addition to the aforementioned, it increases motivation and makes the courses more attractive for students.

There are also examples of the application of video game design and development activities as part of HCI courses, such as the case of Dyck et al. [15] who mentioned that these environments do not put restrictions on how things should look or how the interaction should be, but reward innovation and performance that often leads to novel interaction, which have led to the production of many ideas for future work. Bernhaupt et al. [24] agreed that the design of video games, as part of an HCI course, allows for exploring new interaction techniques and experiences, allowing faster adoption of novel technologies, and, as an example, they mention the creation of Oculus Rift and the Kinect, of which Villaroman et al. [25] have proposed the latter as projects for HCI courses.

In addition, as mentioned by Covaci et al. [26], modern educational theories describe effective learning as active, experiential, situated, problem-based, and providing immediate feedback. The development of games can be the balance between play and learning activities, including this type of events in HCI courses, which is consistent with the theories of effective learning since, with the design and development of games and real-world situations, that allows students to develop skills for solving problems in a way that they also stay motivated [27,28].

4. Experimental Study

This study was implemented in a fifth semester mandatory human-computer interaction course, taught at the University of Colima in Mexico for the software engineering major. This course has been taught since 2009 and, during these years, the learning has been mixed, using projects with video games and learning with utility projects or administrative software.

The experiment was between subjects [29], in which students were taking the course when video games were used. This evaluation approach is a study design in which each group is only exposed to a single setup (i.e., learning with utility software or learning with video games). At the time of the study, the student population of the fifth semester in software engineering were 40:29 (72.5%) were male, 11 (27.5%) were female, and none of them had taken an HCI course before. In this stage of their education, they already had skills in programming, data structures, and software development processes.

The study was designed to explore students’ perceptions of the efficacy of video game design to acquire the required HCI competencies: knowing the importance of the correct design, application and evaluation of human computer interfaces, identifying paradigms, and state-of-the-art in the design of technological developments with the purpose of producing efficient, easy to use software interfaces to solve computing problems.

5. The Games

In this section we describe two examples of computer game projects that were designed and developed during the course. These games were selected because they were success cases since the students published their results in research conferences [30,31].

5.1. Fallbox: Game Controlled by Head Tracking

Fallbox is a 2D game (see Figure 1) played from the first-person perspective. It is based on head tracking using infrared led and the infrared camera of the Nintendo Wii Remote. The aim of this game is that players move their heads to control the game avatar (called Chilo) on the X axis. Chilo has to move to dodge the boxes that are falling down.

The game has 5 levels, with increasing difficulty in each level, because the boxes are falling faster. The player has a limited life (a 100% bar at start) that decreases each time Chilo is hit by a falling box.

5.2. Shooter: Game with Multimodal Interaction

This game is a 2D with a first-person perspective. The characteristics and goals of the game are simple: the user has to go through different scenarios in which he has to eliminate every enemy that appears within a time frame. The game integrated three different types of tangible multimodal interactions (see Figure 2):

• Pen-based interface: these kind of interfaces uses stylus-type devices to point to elements on the user interface or to simulate handwriting [32];
• Vision recognition: this recognition technique is used to simulate the mouse by wearing colored tapes on the user’s fingers for tracking the movements and controlling the cursor [33]; and
• Gesture interface: this interaction is achieved using a video camera and computer vision techniques to capture the hand shape and movement, in order to simulate the mouse motion [34].

6. HCI Teaching Process

The course has been oriented to teaching through the design and evaluation of computer games and their interaction devices. This way students learn not only basic game design principles, but also acquire skills from the different areas that converge in the HCI area.

The course follows the proposal of Greenberg [2] that presented the teaching of HCI as a usability engineering process (see Figure 3), which has the purpose for students to acquire sufficient skills to make a reasonable design, develop, and evaluate of human-computer interfaces. Upon completion of the course, students must understand what a good design means, and have experience designing systems that are usable for people, which must be achieved by implementing interfaces through prototypes and practicing methods of evaluating the quality of the product.

6.1. Process of Design

The design process in HCI is fundamental and designing includes learning and acquiring important knowledge by the students [2]: psychology of everyday things, involving users, task-centered system design, models of human behavior, creativity and metaphors, and graphical screen design.

The design of a game starts with a plan. It will help define the game story and its elements. The game plan contains the designs of characters and scenarios, in addition to the types and levels of interaction that the player will have with the game. To achieve this, the sketching technique proposed by Buxton [35] was used. This technique has proved to be an excellent means for people to start thinking about the process of product design in a simple way [36].

To evaluate the students’ success during this stage of the course, students must generate the sketches with the initial design of the game and the interaction mechanisms (for examples, see Figure 4 and Figure 5).

6.2. Process of Implementation

The emphasis of the development process is on the behavioral domain: the tasks are concerned with the user interaction with the application and the view of the user. A variation of the user-centered methodology is used based on user-task analysis, where task-oriented methodologies ask that users attempt to complete a task in the system [37]. The first stage in the process is to identify a complete description of tasks, subtasks, and methods required to use a system, in addition to having a complete understanding of the user. With this process, designers have a clear understanding of the user requirements, and, therefore, could design better functionality according to the user. After gathering the users’ task analysis, the constructional domain starts, which involves modelling, source code generation, and integration.

For the Fallbox game, in this stage the students identified the scenario of the game as a complete description of the user task:

“Chilo is the youngest kid in the family. For this reason, his brothers throw boxes to stop him inside their house. The player must help Chilo avoid these boxes by moving his head and show his brothers that he can beat them.”

Once the tasks were defined, the subtasks were defined: the avatar will interact only with falling boxes, the boxes will fall with increased speed, the scenarios will include non-playable objects (i.e., tables, chairs, beds, frets, stove, refrigerator), and the avatar will be controlled by the users’ head movements.

At this point, the students have a clear understanding of the requirements, and they can start to build the game and the interaction devices. For the aforementioned game, they envisioned a device that should be placed on the player’s head for head tracking that consisted of an infrared LED (see Figure 6) that points to a Wii Remote and is the responsible for indicating the position of the player to the avatar in the game. So, the player has to move her head to control the game and dodge the boxes that are falling down.

6.3. Process of Evaluation

To learn the competencies for the usability assessment, the students used a summative evaluation following the methodology proposed by Santana-Mancilla [38], called “IHCLab Usability Test for Serious Games”.

The evaluation included the followings phases:

• The moderator (course instructor) opens the session with an introductory text and applies the questionnaire for user characterization;
• Each team gives a live demo of the game they developed in the course. The aim of this is to put into context the use of the controllers for the users;
• The participants should be given a task list to complete on the game. Each team will evaluate the usability of the games developed by the other teams; and
• The players answer the questionnaires to collect their opinions:

  (1)

  Game heuristics questionnaire: an adapted and generalized game heuristics instrument based on previous applications reported by the literature [39,40].

  (2)

  Game experience questionnaire (GEQ): The GEQ is divided in two dimensions: (1) four questions, where the learners had to give a score from 1 to 10, where 10 is the most significant, and (2) seven questions that measured some important indicators with a 5-point Likert scale.

7. Case Study

To validate the effectiveness of computer games use in the learning of HCI, as mentioned in Section 4, a between-subjects study was conducted with the two groups of students who followed the HCI course at the School of Telematics of the University of Colima in Mexico.

7.1. Subjects

Participants were 40 undergraduates studying their third year (fifth semester) with a software developer background (programming, algorithms, and data structures) and a basic hardware knowledge background (embedded systems and digital electronics). The school gave the informed consent for inclusion before the students were included in the study.

7.2. Questionnaire

A questionnaire (designed by our research group) to know the opinion of the students about their perception of the use of video games in HCI teaching was applied.

To validate the questionnaire contents, a group of experts in the area of HCI (expert judgment) used item analysis. The validity of content is essential when making inferences or generalizations from the results of the evaluation [41].

For this evaluation, four criteria were established:

• Sufficiency: The items that measure an indicator are enough to obtain the measurement of it;
• Clarity: The item is easily understood, that is, its syntactic and semantics are adequate;
• Coherence: The item has a logical relationship with the indicator that it is measuring; and
• Relevance: The item is essential or important, that is, it must be included.

To evaluate each criterion, the following response options were established:

• Does not comply.
• Low level.
• Moderate level.
• High level.

A total of five teachers, working in different HCI areas, validated the questionnaire. From the evaluation, a set of changes were introduced in the structure of the questionnaire (language used, closing items, rearrangement of some questions), developing the version that was used in this study, which consists of six questions and is shown in Appendix A. It is important to mention that this questionnaire had already been applied in other studies by our research team [42], but is still in the process of further validation using the Cronbach’s Alpha coefficient to validate the internal consistency of the instrument since it is calculated from the covariance between the elements analyzed. The results obtained in this study will contribute to the validation process of the questionnaire.

Each of the students in the intervention groups filled out the questionnaire and, since all students accomplished the instrument, the sample includes all the perceptions of the subjects under study and, thus, meets the requirement for saturation.

7.3. Results

95% of the students participating in the course thought that they learned the required HCI skills using the design and development of video games (see Figure 7). The remaining 5% considered that it was necessary to integrate additional content so that learning was more complete. The topics most commonly proposed for improvement were evaluation of video games, novel interaction devices, and graphic design.

This result showed a largely positive perception of the efficacy of use video games for skills development in higher education.

When asked how they thought their learning would have been if the teaching process were not with video game design, 100% said that the knowledge acquired would have been lower. Very related, the 85% percent mentioned that they would attend other courses that use video games as a learning tool.

Regarding if they enjoyed learning using computer games, 100% of the students enjoyed the course, and the following is some representative feedback:

• “I enjoyed from the moment of creating the idea of what game to develop, the design, the development of the new interaction and playing the new videogame.”
• “The implementation of high technology interfaces and the evaluation to improve our project.”
• “The use of videogames to compare the course topics with experiences closer to the common and everyday things, the work with devising a new form of interaction for a game was good and challenging.”

Regarding the problems, difficulties or frustrations they could have during the project, they said the following:

• “On some occasions it was difficult for me to understand how interaction will work.”
• “Understand the interaction that was going to be implemented and how it would be applied.”
• “Actually, the biggest difficulty is learning to develop this type of applications, but as is more fun we were more motivated to learn.”

Finally, the students made some suggestions to improve the experience.

• “I would suggest that projects like this used on the HCI course, be proposed as integrative projects for all the courses in the semester, because these projects motivate us to be involved deeper with the technologies implied.”
• “I think that this way of learning is very good.”
• “I would like there to be more time for the implementation of the interactions.”

8. Conclusions

Forty students who had participated in the HCI course were interviewed with an instrument following the conclusion of the semester. Participants were drawn from the human-computer interaction course of their undergraduate degree.

The use of computer games applied to HCI education has given us, in addition to learning, a motivating way of transmitting knowledge to students. The findings contribute to the learning-by-games literature by showing that students perceived value in learning by game design at the university level.

The approach of learning HCI through the design and evaluation of computer games has given results outside the course. To date we have nine undergraduate students who have created a computer game as a final project of their engineering studies based on the HCI course they attended. In addition, students and professors have published more than 15 research works (book chapters, journals, and conferences) in the field of video games and serious games with these projects and the lessons learned from the HCI course in the years that it has been offered.

By its nature, games offer constant challenges that students must be overcome to acquire the required skills, and these challenges allow the students to perceive that games can improve their required skills.

The evaluation has shown a broadly positive student attitude on how this game approach was perceived, demonstrating that, according to the perception of the students, games can have a positive effect on higher education.

The students’ perceptions described here appear to require further investigation. These results have motivated a long-term randomized controlled trial where participants will be assigned to either an intervention or a control group to offer compelling evidence for the potential of designing video games to aid in the development of higher education competencies on HCI.