Next Article in Journal
Dependence of PV Module Temperature on Incident Time-Dependent Solar Spectrum
Previous Article in Journal
Exploring the Evolution of Drought Characteristics in Balochistan, Pakistan
Previous Article in Special Issue
Meaningful Learning Through Virtual Reality Learning Environments: A Case Study in Materials Engineering
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 10
Issue 3
10.3390/app10030915
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Technological Obsolescence of Virtual Reality Learning Environments

by
Diego Vergara
1,*,
Jamil Extremera
2,
Manuel Pablo Rubio
3 and
Lilian P. Dávila
4
1
Technological Department, Catholic University of Ávila, 05005 Avila, Spain
2
Computer Science and Automatics, University of Salamanca, 37008 Salamanca, Spain
3
Construction Department, University of Salamanca, 49029 Zamora, Spain
4
Department of Materials Science and Engineering, School of Engineering, University of California at Merced, Merced, CA 95343, USA
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(3), 915; https://0-doi-org.brum.beds.ac.uk/10.3390/app10030915
Submission received: 26 December 2019 / Revised: 16 January 2020 / Accepted: 24 January 2020 / Published: 31 January 2020
(This article belongs to the Special Issue Technology Enhanced Learning Through Extended (Augmented, Virtual & Mixed) Reality)
Browse Figures
Versions Notes

Abstract

:

Featured Application

The process of technological obsolescence suffered by the virtual reality learning environments used in higher education and its consequences in educational fields are analyzed in this paper.

Abstract

The concept of technological obsolescence that affects computer programs is a readily observable phenomenon that has been widely studied over the past half century. The so-called virtual reality learning environments (VRLEs) which are used to support university classes are significantly affected by this technological obsolescence, decreasing their formative effectiveness as the obsolescence process advances. In this study, the technological obsolescence of two VRLEs is analyzed by means of an empirical research based on survey results (N = 135) after using the VRLEs in engineering classes. Several key performance indicators (KPIs) were analyzed during seven academic courses, including motivation, interactivity, ease of use and usefulness. Since both VRLEs were updated during this research work, the influence of these improvements is discussed in detail from a technological obsolescence point of view. Results suggest that the technological obsolescence negatively affects the students’ opinion regarding motivation and interactivity, but the other KPIs (ease of use and usefulness) are hardly affected. In contrast, results indicate that the technological obsolescence can be reversed if periodic updates of educational tools are carried out using modern development software.

1. Introduction

Technological obsolescence affects any device that uses some kind of computer program since any software is constantly updated. Just look at the continuous updates that developers carry out on well-known computer programs or changes that web pages undergo over the years. An example of software evolution is the MS-DOS operating system, which based its operation on the introduction of written commands and was later replaced by Windows, whose operation is based on user interaction with graphic elements which the user must manually click. Another clear example can be seen if one visits a website developed for a large company in the late nineties and compares it with a more recently developed website for that same company: a modern website of this type has an aesthetic adapted to current preferences, links to social networks (nonexistent in the nineties), modern security protocols, etc. Similarly, today almost all the websites of universities and scientific journals are generally updated in a period not exceeding 5 years. This evolution of software is applicable to many other areas such as video games, operating systems and smartphone applications, user interfaces, and so on.
As noted by Lehman and Ramil [1], the transformation in the use of hardware (i.e. computers, smartphones, etc.) means that the domain of use and its applications change. For instance, a mobile telephone in the nineties was mainly used to make phone calls, while a current smartphone is commonly used for multiple tasks. In response to these changes, software must be able to keep up with the pace set by the social needs through the expansion and refinement of functionalities, the correction of failures and the improvement of performance [1], among others.
The importance of the software evolution process is reflected in the vast amount of money that development companies apply to evolve their programs after the first version has been launched [2]. As reported by Neamtiu et al. [2], the costs of maintenance and evolution of a software program can amount to several times the cost of developing the first version [3]. Knowing the evolutionary process of software and the causes that originate it is in fact of great importance for those who must create, maintain and update any type of application. As Lehman and Ramil [1] indicate, software evolution is a phenomenon that can be systematically studied since to some extent it is a phenomenon in which regularity patterns can be found. Software, in any of its many forms, is present in many aspects of today’s life worldwide and thereby a massive number of programs are constantly being updated. Given the magnitude of any activity related to software maintenance and updating, it is not surprising that there is great interest in knowing this matter in depth. Indeed, there are numerous studies that investigate, among others: (i) the way in which software evolves [4,5]; (ii) the development of methods to help carry out the evolution of a certain software [6]; and (iii) the techniques to monitor such evolution [7,8].
Among all the reported studies concerning the evolution of software, the work that Lehman started in the late sixties has a special relevance. Lehman studied different program parameters that had gone through various updates, such as system size, number of modules added, deleted or modified, economic cost, etc. identifying rules that seemed to govern the software update process [9]. In 1980, Lehman defined as E-type software those computer programs written to carry out an activity in the real world (e.g. an operating system or a program to manage the stock of a warehouse [10]), being included in this definition most of the computer programs used to date. While Lehman listed the first 3 laws of software evolution in 1974 [9], it was not until 1996 when he extended this list to 8 laws [11]. Lehman’s laws have been used in numerous works focused on the evolution of software, either to study them [12,13,14,15,16] or to use them as support for other studies [17,18,19,20,21].
As technology evolves steadily the use of virtual reality learning environments (VRLEs) has rapidly emerged as a promising technology that provides opportunities for flexible, adaptable, interactive and personalized learning experiences. If teaching methods are approached from a constructivism viewpoint, students shall play an active role in their own learning process [22], not merely being passive receivers of information [23]. The basic idea of constructivism is that problem solving is at the heart of learning, thinking, and development. As students solve problems and discover consequences of their decisions, they can build their own understanding and gain effective skills to solve real problems [22]. The premise is that students only deeply understand what they have constructed. Consequently, the student interaction with the environment (real or virtual) [23] is necessary. Thus, virtual reality (VR) is an effective support in the application of the constructivist strategy [24]. The use of VRLEs and VR in higher education and industry using varied computing infrastructures provide viable means to stimulate innovation, teamwork and cost-effective options while providing quality education and training. This is understandable if one considers that the emergence of these educational tools (which, in turn, is the result of technological evolution) has allowed students to interact with virtual environments in a way that would have been impossible few decades ago. This fact has brought about the possibility of implementing new teaching concepts and methods from a constructivism point of view, where the student’s interaction with the virtual environment plays a key role. Enabling the learning and teaching through VRLEs will help make students be more competitive, eager to learn new concepts, ready to work collaboratively with other people in different disciplines and expertise, adaptable in new job environments and innovative-driven to seek entrepreneurship opportunities. For this reason, VRLEs have been the subject of different studies that have indicated which characteristics are those that arouse greater motivation among students [25,26,27,28,29,30,31]. One of the main conclusions drawn from these studies indicates the technological obsolescence of VRLE as a factor that decisively influences the motivation that students have when using these educational tools. Thus, it has been observed that if a VRLE is not updated periodically (and therefore becomes obsolete), students increasingly feel less motivated to use it, hence decreasing its effectiveness at a formative level [32].
For this reason, in this article the authors analyze the influence of the technological obsolescence process of VRLEs in some of its key performance parameters (KPIs), such as motivation, interactivity (i.e., the way in which the user manipulates the VRLE, as well as the type and quality of actions that he/she can perform in the virtual environment), ease of use and usefulness. These specific KPIs have been chosen because they are the most representatives of the VRLEs according to previous studies [33,34,35,36,37,38,39,40,41,42,43]. In addition, the relationship between technological obsolescence of VRLEs and Lehman’s laws is raised. The results obtained are based on a 7-year study (from the 2013–2014 academic year to the 2019–2020 course) using different VRLEs. The use of different VRLE designs–based on software from different years – and the comparison of results from surveys presented to 135 engineering students, have demonstrated the negative influence of the process of technological obsolescence on VRLEs. In this way, it has been verified that through a periodic update of VRLEs using current development tools it is possible to restore their KPIs to the levels before the obsolescence process began.

2. Virtual Reality Learning Environments

2.1. Development of the VRLEs

Learning ternary phases diagrams (TPD) and crystal lattices (CL) usually generate problems of spatial visualization in students reading topics in Materials Science and Engineering [36,44,45], since they must be able to mentally recreate in three dimensions complex structures and overlapping elements. To solve this problem, the authors of this article developed and used in classes two VRLEs that aim to improve the spatial understanding of TPD and CL, respectively. Some years later, these VRLEs were updated using development tools different than the ones used for the creation of the first versions, as described below, and applied again in the classroom. In order to differentiate between the two VRLEs previously developed from those developed later, the first ones are named “VRLEs-1” (first version of both VRLEs of TPD and CL), and the second type are named “VRLEs-2”, describing the two VRLEs of TPD and CL developed afterward (Figure 1). The workflow followed during the lifecycle of all VRLEs has been the same in all cases (Figure 2), similarly to that used in previous studies such as those by Ren et al. [46] and Rubio et al. [47,48].
The development of each of the stages in Figure 2 was identical for each VRLE, with the exception of the third stage. During that stage (application development) the 3D modeling of the virtual environment, assignment of materials, lighting application to the scene and programming of VRLE interactivity are carried out. However, the programs used to develop VRLE-2 were either updated versions of those employed with VRLE-1 or were different programs altogether.
To create virtual environments (i.e., 3D modeling, application of materials and lighting) 3DS Max® (version 2013, Autodesk, San Rafael, CA, USA, 2012) was used in all cases, but a more recent version (version 2017, which was released on 2016) of this program was used to create the VRLE-2. This latest version of 3DS Max® allows the creation of virtual environments that are more realistic and aesthetically more attractive than those created with the older version because it offers a larger library of materials as well as more lighting options.
The interactivity of the VRLE-1 systems was programmed using Quest 3D® (version 5.0, Act-3D, Warmond, The Netherlands, 2012), while Unreal Engine 4® (UE4, version 4.13, Epic Games, Cary, NC, USA, 2016) was employed to program the VRLE-2 systems. Quest 3D® is a basic platform that allows the programming of three-dimensional environments with low levels of graphic realism and interactivity. On the contrary, UE4 is a game engine that is currently used to program a large number of video games. This game engine allows one to program complex environments with great graphic realism and high interactivity. UE4 also offers the possibility of subjecting virtual environments to physical laws, which allows simulating more realistic collisions or the effects of gravity, among others. Furthermore, UE4 uses a physics-based rendering system, which calculates the interaction of light with materials through physical equations [49]. In addition, programming on this platform is comparatively simpler with respect to others as it is based on a visual scripting system, thus eliminating the need to write complex computer codes [50].

2.2. Description of the VRLEs

The VRLE-1 systems (Figure 3) are non-immersive virtual reality (VR)-based applications which run generally on a personal computer, the user interaction is carried out using a keyboard and a mouse and the virtual environment is displayed via a monitor [26,36,44]. The VRLE-1 dedicated to the TPD understanding (Figure 3a) shows a three-dimensional illustration of an ideal TPD model where it can be distinguished different parts (each part corresponding to a specific phase, resulting from the combination of three different components at several ranges of temperature). A given user can manipulate, by means of a mouse, each part to understand the TPD model and its individual phases spatially. Thus, the user of this VRLE can perform on the virtual TPD model operations such as: separate and identify different individual phases, rotate elements or apply transparencies to the surfaces to visualize hidden areas, etc. These operations allow users to see –and hence better understand– invariant points (e.g. eutectic, peritectic, etc.) as well as to have an improved view of the relation between the concentration of the three components, temperature and phases formation. On another hand, the VRLE-1 dedicated to CL (Figure 3b) offers the possibility of exploring different types of crystalline networks such as those that are part of the elementary structure of metallic materials. To do this, the VRLE allows users to choose and explore one of the 14 Bravais lattices. Once a crystal lattice has been selected, the user can perform several operations on the cell (by means of the mouse) such as: do 3-axis free rotation, create section views (allowing user to clearly see atoms within the lattice, among others), expand unit cells, etc.
The development tools used to create the VRLE-1 systems do not allow to generate environments with high levels of graphic realism, so both TPD and CL results are displayed in empty spaces, using lighting and smooth and textured colors, an evidence of the technology of the time in which they were implemented. Moreover, the development tools used at that time (Quest 3D® and an older version of 3DS Max®) do not allow the VRLEs to deliver a high level of interactivity, so that the actions and visualization options offered to the user are rather limited.
The VRLE-2 systems (Figure 4) are non-immersive VR-based applications that also run on a personal computer, are controlled by a keyboard and a mouse, and are displayed on a monitor. These VRLEs allow users to explore the virtual environment in a similar way as he/she would do in a first-person shooter video game [51,52]. Since these later versions were developed with more modern tools, the VRLE-2 systems have a considerable improvement in both their visual appearance and interactivity. Hence, in the new version of the VRLE dedicated to TPD studies (Figure 4a), the user can move freely through a laboratory environment similar to that in any university, in which a hologram of a TPD can be placed in the middle of the room. In addition, this version of the VRLE offers more possibilities for interaction than in the previous versions (such as obtaining isothermal sections of a TPD virtually). Likewise, the new version of the VRLE dedicated to CL (Figure 4b) allows any user to move freely through a virtual museum whose exhibition halls show the 14 Bravais lattices [51].
As the user approaches any of these lattices, the VRLE offers more possibilities for exploration and interaction than in the case of the previous version (Figure 3b), for instance revealing the unit cell and the expanded set with its geometric parameters, directions and crystallographic planes, octahedral and tetrahedral gaps, coordination indices, sections and families of planes and directions, etc. A complementary video is available in the supplemental section of this paper, which better clarifies all the technical options offered by this last VRLE. Therefore, both VRLE-2 systems (Figure 4) offer a more realistic and attractive appearance and more interaction options than prior versions (VRLE-1 systems in Figure 3).

3. Application in the Classroom

The use of VRLEs, both the former versions (Figure 3) and the later ones (Figure 4), have followed the same overall procedure, which is illustrated in Figure 5 and is similar to that used in prior studies reported by Mirauda et al. [53]. This procedure has been implemented as an assignment regarding Materials Science and Technology, which is part of the program of the second course of Mechanical Engineering Degree at the Catholic University of Avila (Spain), between the academic courses 2013–2014 and 2019–2020, and consist of the following main stages:
  • The instructor teaches master classes, during which he/she addresses among others, both the theory and solution of practical exercises concerning TPD or CL problems. This stage lasts approximately two weeks (8–10 class hours) for teaching binary and ternary phase diagrams, and one week (2–4 hours) to explain the concepts related to CL.
  • The use of VRLE in the classroom under the supervision of the instructor (0.5–1 hour). In addition, the student can continue using the VRLE on his/her personal computer after class as needed.
  • Solution of exercises related to CL in groups of 2–4 students (in the case of TPD there is no solution of exercises). The duration of the solution process of these exercises, plus the corresponding correction by the instructor, typically involves 2 hours in the classroom.
  • Survey fulfillment by students. These surveys aim to evaluate specific KPIs of the VRLEs: motivation, degree of interactivity, ease of use and usefulness (Table 1).
  • Analysis of the data obtained through the above surveys.
The authors of this article have verified that solely applying the above stage corresponding to master classes (theory explanation and practical exercises solution) the teaching-learning process is incomplete, so that often students forget quickly what they have seen or heard in the classroom. Despite this, using VRLEs has demonstrated to improve the spatial understanding of students particularly in both TPD and CL exercises, so that they later assimilate related concepts. This certainly results in students achieving better meaningful learning and retaining concepts learned in class for longer periods of time [36].

4. Results

In total 135 students participated in the study between 2013 and 2020. From this set, 103 students used the older versions of the VRLEs (VRLEs-1, Figure 3), of which 82 used the VRLE-1 dedicated to the TPD exercises (Figure 3a) and 81 the VRLE-1 dedicated to CL studies (Figure 3b). The average scores assigned by the students to each question (Table 1) are summarized in Table 2. Furthermore, as shown in Table 3, 53 students used the later versions of the VRLEs (VRLEs-2), of which all of them used the VRLE-2 dedicated to solve TPD exercises (Figure 4a) but only 32 students used the VRLE-2 system (Figure 4b) dedicated for CL studies. The average scores assigned by the students to each question (Table 1) are shown in Table 3. The results of both Table 2 and Table 3 are illustrated in Figure 6 (data corresponding to both VRLE versions of the TPD) and Figure 7 (data corresponding to both VRLE versions of the CL).
As can be seen in Figure 6 and Figure 7, there are two similar evolutions: (i) the values of both motivation (question 1) and interactivity (question 2) decrease over time until the VRLE is updated (VRLE-2); (ii) the values of both ease of use (question 3) and utility (question 4) remain constant throughout the period of time evaluated. Looking at the VRLE-1 case, the KPI corresponding to motivation (question 1) has an average value of 8.6 (TPD) or 7.5 (CL) in the first academic year evaluated, decreasing this value year after year until reaching a value of 7.5 (TPD) and 6.9 (CL) in the last year of use. The interactivity KPI (question 2) is similar to the aforementioned, that is, at the beginning of the VRLE-1 usage, the interactivity KPI has an average value of 7.5 (TPD) and 9.0 (CL), which drops in each course until reaching a value of 6.2 (TPD) and 7.5 (CL) in the last course evaluated. By contrast, as seen in Table 2, this decreasing trend changes in both KPIs just at the moment when the period of use of VRLEs-2 begins. In particular, the motivation KPI reaches a value of 9.2 (TPD) and 9.5 (CL) at the beginning of the period of use of these new versions of VRLEs. Similarly, one can observe in the interactivity KPI of the VRLE that the values reach 8.3 (TPD) and 8.3 (CL) at the beginning of the use of the updated version (VRLE-2). Regarding standard deviation values, results shown in Table 2 indicate a low dispersion of the students’ feedback (the maximum value is 1.03).
From Table 2 and Table 3, as well as Figure 6 and Figure 7, it is verified that the KPIs corresponding to the ease of use (question 3) and usefulness (question 4) have values that vary little throughout the period of time studied, regardless of the version of VRLE used. Thus, the ease of use KPI acquires values between 7.6–7.0 (TPD) and 9.3–8.9 (CL). In addition, the usefulness KPI acquires values ranging between 9.8–9.7 (TPD) and 9.8–9.6 (CL). In both of these KPIs the standard deviation is low, never exceeding 0.83, which indicates that there is little dispersion in the individual assessments taken from the surveys and, consequently, the opinion of the student body is quite homogeneous.

5. Discussion

From Figure 6 and Figure 7 it can be seen that the motivation generated by the VRLEs-1 systems decreases as time goes by. It is also noticeable that from the moment the VRLEs-2 systems, which are the updated versions of the VRLEs-1 systems, began to be used the motivation of the students returned to high levels. This fact is related to what is stated in Lehman’s seventh law, which read as follows: “E-type programs will be perceived as of declining quality unless rigorously maintained and adapted to a changing operational environment” [11] (p.3), i.e., a program that has been satisfactory for users during a certain period of time can be negatively valued by those same users. This phenomenon is due to the fact that the criteria of acceptance and satisfaction of the users often change with the passage of time, thus using feedback from the users to avoid the loss of quality of a program also acquires a great relevance [11].
Examining Figure 6 and Figure 7, it is concluded that the assessment given by the students to the level of interactivity follows a trend similar to that observed in motivation, that is, it decreases continuously until the old VRLEs-1 are replaced by the newer VRLEs-2, at the moment when assessment given by the students to this parameter increases again. This fact could be related, on the one hand, to the seventh law of Lehman described above, and on the other hand with the first law of Lehman, which reads as follows: “An E-type program that is used must be continually adapted, else it becomes progressively less satisfactory” ([11], p.1). This law is easily understood from everyday experience: when a program that is intended to be useful over the years must be adapted to the requirements set by the environment in which it is used. For example, it is observed that in the last decade a large number of programs and web pages for the general public have incorporated functionalities related to social networks in response to the boom in their use (e.g. Twitter® or Facebook®). Regarding the case of the VRLEs studied here, it is likely that the students frequently demand the possibility of more interaction with any program that they already find in other similar VR-based applications.
Based on the two Lehman’s laws set forth herein, and in accordance with the knowledge and experience of the authors in this article, the decrease in motivation and level of interactivity that students perceive throughout the years in which VRLEs-1 are used has its origin in the technological obsolescence to which they are exposed from the moment of their creation. When the first versions of VRLEs (VRLEs-1) were created with the development tools available at that time, they presented an aspect and handling similar to non-immersive VR-based applications that were available then. Consequently, students perceived VRLEs-1 as attractive and with highly interactive applications. However, over time the VRLEs-1 were at a disadvantage when compared to other similar applications. Therefore, students rated the VRLEs-1 worse and worse, as they perceived them to be outdated and with non-interactive applications.
When the VRLEs-2 were created, more modern development tools were used than those implemented to create the VRLEs-1, thereby achieving virtual environments with an aspect and a degree of interactivity matching the needed applications that could be found in that time. This means that the student body values again with high scores the aspects of motivation and level interactivity. As can be easily deduced, this is closely related to what is indicated in the seventh law by Lehman [11], which establishes that a constant work of adapting an application to the changing environment can make the perception of quality by users not decay over time. It can also be observed that Lehman’s first law is fulfilled in the case of interactivity because by adding new possibilities for interaction in VRLEs-2, students have reacted positively in their assessments (Figure 5 and Figure 6).
If the graphs of Figure 6 and Figure 7 are analyzed in greater depth, it can be observed that the decreasing slope of the motivation and interactivity plots is more pronounced as time goes by. That is, in the following year the creation of the VRLEs-1 the scores provided by the students hardly vary, but over time the decrease in the overall score given by the students is more pronounced. These results suggest that VRLEs suffer speedy technological obsolescence since in just four years they can lose the level of motivation and interactivity they had at the time of their creation.
Lastly, it was observed that the assessment given by the students to the ease of use and usefulness of VRLEs hardly varies over the years or with the change in the VRLE version (i.e. with the passage of VRLE-1 to VRLE-2). In the authors’ opinion, the invariability of such KPIs (ease of use and usefulness) is mainly due to two main factors: (i) the majority of students are used to using this type of virtual environment since they are very similar to video games and (ii) all the participating students in this study had used in the past, or were using at that time, other VRLEs in the classroom as tools that supported their master classes [26,27,33,35,37,38,52,54], hence students were familiar with their use as they had proven to be useful tools.

6. Conclusions

The results obtained in this study as we have seen are consistent with two of the laws enunciated by Lehman (in particular, with the first and the seventh law), which allow us to affirm that the assessment that students have of a VRLE is closely related with the degree of updates of the software used in its development. In other words, the technological obsolescence suffered by virtual educational platforms has a negative impact on the motivation they generate in students and the level of interactivity they perceive, which means that the educational tool loses part of its effectiveness at the formative level. This study has revealed that the use of current development tools allows the creation of educational tools that the students value positively in terms of motivation and interactivity. On the contrary, the assessment made by the students regarding both the ease of use and the usefulness of an educational platform does not seem to depend on the updating of the development tools used in its creation, but probably on other factors such as, for example, the fact the students are used to using similar modern computer applications.
Whenever an instructor develops an educational computer application in support of classes, he/she may not take into account that this type of program is subject to a rapid process of obsolescence nor of the negative consequences that this has on the efficiency of the teaching tool. Due to the great influence that the obsolescence process has on the formative effectiveness of VRLEs, this research paper highlights the need to update them periodically using current development tools. This way, instructors will ensure that their educational platforms maintain their formative effectiveness over the years.

Supplementary Materials

The following are available online at: https://0-www-mdpi-com.brum.beds.ac.uk/2076-3417/10/3/915/s1, Video S1: The Technological Obsolescence of Virtual Reality Learning Environments.

Author Contributions

Conceptualization and methodology, D.V.; software, J.E. and M.P.R.; validation, D.V., J.E. and M.P.R.; formal analysis, D.V., J.E., M.P.R. and L.P.D.; supervision, D.V., J.E. and L.P.D.; writing – original draft, D.V. and J.E.; writing – review & editing, D.V., J.E. and L.P.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

Authors wish to acknowledge the collaboration made by the engineers Gonzalo Mezquita, María Sánchez and Alberto Garcinuño in the design process of the VLs used in this paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lehman, M.M.; Ramil, J.F. Software evolution—Background, theory, practice. In Information Processing Letters; Elsevier Science: Amsterdam, The Netherlands, 2003; Volume 88, pp. 33–44. [Google Scholar]
  2. Neamtiu, I.; Xie, G.; Chen, J. Towards a better understanding of software evolution: An empirical study on open-source software. J. Softw. Evol. Process 2013, 25, 193–218. [Google Scholar] [CrossRef] [Green Version]
  3. Sommerville, I. Software Engineering, 7th ed.; Pearson Addison Wesley: Boston, MA, USA, 2004. [Google Scholar]
  4. González-Barahona, J.M.; Robles, G.; Michlmayr, M.; Amor, J.J.; German, D.M. Macro-level software evolution: A case study of a large software compilation. Empir. Softw. Eng. 2009, 14, 262–285. [Google Scholar] [CrossRef] [Green Version]
  5. Koch, S. Software evolution in open source projects—A large-scale investigation. J. Softw. Maint. Evol. 2007, 19, 361–382. [Google Scholar] [CrossRef]
  6. Cosmo, R.; Ruscio, D.; Pelliccione, P.; Pierantonio, A.; Zacchiroli, S. Supporting software evolution in component-based FOSS systems. Sci. Comput. Program. 2011, 76, 1144–1160. [Google Scholar] [CrossRef]
  7. Arbuckle, T. Studying software evolution using artefacts’ shared information content. Sci. Comput. Program. 2011, 76, 1078–1097. [Google Scholar] [CrossRef] [Green Version]
  8. Ambros, M.; Lanza, M. Visual software evolution reconstruction. J. Softw. Maint. Evol. 2009, 21, 217–232. [Google Scholar] [CrossRef]
  9. Lehman, M.M. Programs, cities, students—Limits to growth? Programming methodology. In Texts and Monographs in Computer Science; Gries, D., Ed.; Springer: New York, NY, USA, 1978; pp. 42–69. [Google Scholar]
  10. Lehman, M.M. Programs, life cycles, and laws of software evolution. Proc. IEEE 1980, 68, 1060–1076. [Google Scholar] [CrossRef]
  11. Lehman, M.M. Laws of software evolution revisited. In Proceedings of the Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 5th European Workshop on Software Process Technology, EWSPT, Nancy, France, 9–11 October 1996; Montangero, C., Ed.; Springer: New York, NY, USA, 1996; Volume 1149, pp. 108–124. [Google Scholar]
  12. Barry, E.J.; Kemerer, C.F.; Slaughter, S.A. How software process automation affects software evolution: A longitudinal empirical analysis. J. Softw. Maint. Evol. 2007, 19, 1–31. [Google Scholar] [CrossRef]
  13. Israeli, A.; Feitelson, D.G. The Linux kernel as a case study in software evolution. J. Syst. Softw. 2010, 83, 485–501. [Google Scholar] [CrossRef] [Green Version]
  14. González-Barahona, J.M.; Robles, G.; Herraiz, I.; Ortega, F. Studying the laws of software evolution in a long-lived FLOSS project. J. Softw. Evol. Process 2014, 26, 589–612. [Google Scholar] [CrossRef] [Green Version]
  15. Oliveira, R.P.; Almeida, E.S. Evaluating Lehman’s laws of software evolution for software product lines. IEEE Softw. 2016, 33, 90–93. [Google Scholar] [CrossRef]
  16. Oliveira, R.P.; Santos, A.R.; Almeida, E.S.; da Silva Gomes, G.S. Evaluating Lehman’s laws of software evolution within software product lines industrial projects. J. Syst. Softw. 2017, 131, 347–365. [Google Scholar] [CrossRef]
  17. Kemerer, C.F.; Slaughter, S. An empirical approach to studying software evolution. IEEE Trans. Softw. Eng. 1999, 25, 493–509. [Google Scholar] [CrossRef] [Green Version]
  18. Chapin, N.; Hale, J.E.; Khan, K.M.; Ramil, J.F.; Tan, W.G. Types of software evolution and software maintenance. J. Softw. Maint. Evol. 2001, 13, 3–30. [Google Scholar] [CrossRef]
  19. Lehman, M.M.; Ramil, J.F. Software evolution and software evolution processes. Ann. Softw. Eng. 2002, 14, 275–309. [Google Scholar] [CrossRef]
  20. Bennett, K.H.; Rajlich, V.T.; Wilde, N. Software evolution and the staged model of the software lifecycle. Adv. Comput. 2002, 56, 1–54. [Google Scholar]
  21. Rocchi, P. An essay on the origin of software evolution. IEEE Trans. Hum. Mach. Syst. 2014, 44, 281–285. [Google Scholar] [CrossRef]
  22. Huang, H.M.; Liaw, S.S. An analysis of learners’ intentions toward virtual reality learning based on constructivist and technology acceptance approaches. Int. Rev. Res. Open Distance Learn. 2018, 19, 91–115. [Google Scholar] [CrossRef] [Green Version]
  23. Aiello, P.; D’Elia, F.; Di Tore, S.; Sibilio, M. A constructivist approach to virtual reality for experiential learning. E Learn. Digit. Media 2012, 9, 317–324. [Google Scholar] [CrossRef]
  24. Concannon, B.; Esmail, S.; Roberts, M. Head-mounted display virtual reality in post-secondary education and skill training. Front. Educ. 2019, 4, 80. [Google Scholar] [CrossRef] [Green Version]
  25. Vergara, D.; Rubio, M.P.; Lorenzo, M. On the design of virtual reality learning environments in engineering. Multimodal Technol. Interact. 2017, 1, 11. [Google Scholar] [CrossRef] [Green Version]
  26. Vergara, D.; Rubio, M.P.; Lorenzo, M. A virtual resource for enhancing the spatial comprehension of crystal lattices. Educ. Sci. 2018, 8, 153. [Google Scholar] [CrossRef] [Green Version]
  27. Vergara, D.; Rubio, M.P.; Prieto, F.; Lorenzo, M. Enhancing the teaching-learning of materials mechanical characterization by using virtual reality. J. Mater. Educ. 2016, 38, 63–74. [Google Scholar]
  28. Doblack, B.N.; Flores, C.; Matlock, T.; Dávila, L.P. The emergence of immersive low-cost 3D virtual reality environments for interactive learning in materials science and engineering. Mater. Res. Soc. Symp. Proc. 2011, 1320, mrsf10-1320-xx04-01. [Google Scholar] [CrossRef] [Green Version]
  29. Flores, C.; Matlock, T.; Dávila, L.P. Enhancing materials research through innovative 3D environments and interactive manuals for data visualization and analysis. Mater. Res. Soc. Symp. Proc. 2012, 1472, mrss12-1472-zz01-03. [Google Scholar] [CrossRef] [Green Version]
  30. Meagher, K.A.; Doblack, B.N.; Ramirez, M.; Dávila, L.P. Scalable nanohelices for predictive studies and enhanced 3D visualization. J. Vis. Exp. 2014, 93, e51372. [Google Scholar] [CrossRef] [Green Version]
  31. Doblack, B.N.; Allis, T.; Dávila, L.P. Novel 3D/VR interactive environment for MD simulations, visualization and analysis. J. Vis. Exp. 2014, 94, e51384. [Google Scholar] [CrossRef] [Green Version]
  32. Vergara, D.; Extremera, J.; Rubio, M.P.; Dávila, L.P. Meaningful learning through virtual reality learning environments: A case study in materials engineering. Appl. Sci. 2019, 9, 4625. [Google Scholar] [CrossRef] [Green Version]
  33. Vergara, D.; Lorenzo, M.; Rubio, M.P. On the use of virtual environments in engineering education. Int. J. Qual. Assur. Eng. Technol. Educ. 2016, 5, 30–41. [Google Scholar] [CrossRef] [Green Version]
  34. Hashemi, J.; Austin-Stalcup, K.A.; Anderson, E.E.; Chandrashekar, N.; Majkowski, A. Elements of a realistic virtual laboratory experience in materials science: Development and evaluation. Int. J. Eng. Educ. 2005, 21, 534–545. [Google Scholar]
  35. Vergara, D.; Rubio, M.P. The application of didactic virtual tools in the instruction of industrial radiography. J. Mater. Educ. 2015, 37, 17–26. [Google Scholar]
  36. Vergara, D.; Rubio, M.P.; Lorenzo, M. A virtual environment for enhancing the understanding of ternary phase diagrams. J. Mater. Educ. 2015, 37, 93–101. [Google Scholar]
  37. Vergara, D.; Rubio, M.P.; Lorenzo, M. Interactive virtual platform for simulating a concrete compression test. Key Eng. Mater. 2014, 572, 582–585. [Google Scholar] [CrossRef]
  38. Vergara, D.; Rubio, M.P.; Lorenzo, M. New approach for the teaching of concrete compression tests in large groups of engineering students. J. Prof. Issues Eng. Educ. Pract. 2017, 143, 05016009. [Google Scholar] [CrossRef]
  39. Salinas, I.; Gimenez, M.H.; Cuenca-Gotor, V.P.; Seiz-Ortiz, R.; Monsoriu, J.A. Design and evaluation of a three-dimensional virtual laboratory on vector operations. Comput. Appl. Eng. Educ. 2019, 27, 690–697. [Google Scholar] [CrossRef]
  40. Rodríguez-Martín, M.; Rodríguez-Gonzálvez, P.; Sánchez-Patrocinio, A.; Sánchez, J.R. Short CFD simulation activities in the context of fluid-mechanical learning in a multidisciplinary student body. Appl. Sci. 2019, 9, 4809. [Google Scholar] [CrossRef] [Green Version]
  41. Violante, M.G.; Vezzetti, E. Virtual interactive e-learning application: An evaluation of the student satisfaction. Comput. Appl. Eng. Educ. 2015, 23, 72–91. [Google Scholar] [CrossRef] [Green Version]
  42. Chang, C.W.; Yeh, S.C.; Li, M.; Yao, E. The introduction of a novel virtual reality training system for gynecology learning and its user experience research. IEEE Access 2019, 7, 43637–43653. [Google Scholar] [CrossRef]
  43. Iacob, R.; Popescu, D.; Noel, F.; Masclet, C. Implementation and evaluation of a model processing pipeline for assembly simulation. Assem. Autom. 2017, 37, 400–410. [Google Scholar] [CrossRef]
  44. Vergara, D.; Rubio, M.P.; Lorenzo, M. New virtual application for improving the students’ understanding of ternary phase diagrams. Key Eng. Mater. 2014, 572, 578–581. [Google Scholar] [CrossRef]
  45. Vergara, D.; Rubio, M.P.; Lorenzo, M. On the use of PDF-3D to overcome spatial visualization difficulties linked with ternary phase diagrams. Educ. Sci. 2019, 9, 67. [Google Scholar] [CrossRef] [Green Version]
  46. Ren, S.; McKenzie, F.D.; Chaturvedi, S.K.; Prabhakaran, R.; Yoon, J.; Katsioloudis, P.J.; Garcia, H. Design and comparison of immersive interactive learning and instructional techniques for 3D virtual laboratories. Presence Teleoperators Virtual Environ. 2015, 24, 93–112. [Google Scholar] [CrossRef]
  47. Rubio, M.P.; Vergara, D.; Rodríguez, S.; Extremera, J. Virtual reality learning environments in materials engineering: Rockwell hardness test. In Advances in Intelligent Systems and Computing; Di Mascio, T., Ed.; Springer: Cham, Switzerland, 2019; pp. 106–113. [Google Scholar]
  48. Vergara, D.; Rubio, M.P.; Lorenzo, M.; Rodríguez, S. On the importance of the design of virtual reality learning environments. Adv. Intell. Syst. Comput. 2020, 1007, 146–152. [Google Scholar]
  49. Jakob, W.; Arbree, A.; Moon, J.T.; Bala, K.; Marschner, S. A radiative transfer framework for rendering materials with anisotropic structure. ACM Trans. Graph. 2010, 29, 1–13. [Google Scholar] [CrossRef]
  50. Valcasara, N. Unreal Engine Game Development Blueprints; Packt Publishing: Birmingham, UK, 2015; ISBN 1784397776, 9781784397777. [Google Scholar]
  51. Vergara, D.; Sánchez, M.; Garcinuño, A.; Rubio, M.P.; Extremera, J.; Gómez, A.I. Spatial comprehension of crystal lattices through virtual reality applications. In Proceedings of the ICERI2019, Seville, Spain, 11–13 November 2019; pp. 1291–1295. [Google Scholar]
  52. Vergara, D.; Rodríguez-Martín, M.; Rubio, M.P.; Ferrer-Marín, J.; Núñez-García, F.J.; Moralejo-Cobo, L. Technical staff training in ultrasonic non-destructive testing using virtual reality Formación de personal técnico en ensayos no destructivos por ultrasonidos mediante realidad virtual. Dyna 2018, 93, 150–154. [Google Scholar] [CrossRef]
  53. Mirauda, D.; Capece, N.; Erra, U. StreamflowVL: A virtual fieldwork laboratory that supports traditional hydraulics engineering learning. Appl. Sci. 2019, 9, 4972. [Google Scholar] [CrossRef] [Green Version]
  54. Vergara, D.; Lorenzo, M.; Rubio, M.P. Virtual environments in materials science and engineering: The students’ opinion. In Handbook of Research on Recent Developments in Materials Science and Corrosion Engineering Education; Lim, H., Ed.; IGI Global: Hershey, PA, USA, 2015; pp. 148–165. [Google Scholar]
Applsci 10 00915 g001 550
Figure 1. Classification of the VRLEs analyzed in this study: (i) former versions, VRLE-1 (left-hand side); and (ii) more recently developed versions, VRLE-2 (right-hand side).
Figure 1. Classification of the VRLEs analyzed in this study: (i) former versions, VRLE-1 (left-hand side); and (ii) more recently developed versions, VRLE-2 (right-hand side).
Applsci 10 00915 g001
Applsci 10 00915 g002 550
Figure 2. Workflow followed during the VRLEs lifecycle, based on previous works.
Figure 2. Workflow followed during the VRLEs lifecycle, based on previous works.
Applsci 10 00915 g002
Applsci 10 00915 g003 550
Figure 3. Illustration of previously developed VRLEs-1: (a) VRLE-1 system dedicated to TPD calculations and (b) VRLE-1 system focused on CL approximations.
Figure 3. Illustration of previously developed VRLEs-1: (a) VRLE-1 system dedicated to TPD calculations and (b) VRLE-1 system focused on CL approximations.
Applsci 10 00915 g003
Applsci 10 00915 g004 550
Figure 4. Screenshots of the VRLEs subsequently developed (VRLEs-2): (a) VRLE-2 system dedicated to TPD studies and (b) VRLE-2 system designed for CL investigations.
Figure 4. Screenshots of the VRLEs subsequently developed (VRLEs-2): (a) VRLE-2 system dedicated to TPD studies and (b) VRLE-2 system designed for CL investigations.
Applsci 10 00915 g004
Applsci 10 00915 g005 550
Figure 5. Procedure used to implement the VRLEs in the classroom.
Figure 5. Procedure used to implement the VRLEs in the classroom.
Applsci 10 00915 g005
Applsci 10 00915 g006 550
Figure 6. KPIs values of the VRLE of TPD over a period of academic years (former version, VRLE-1, and later version, VRLE-2).
Figure 6. KPIs values of the VRLE of TPD over a period of academic years (former version, VRLE-1, and later version, VRLE-2).
Applsci 10 00915 g006
Applsci 10 00915 g007 550
Figure 7. KPIs values of the VRLE of CL over another period of academic years (former version, VRLE-1, and later version, VRLE-2).
Figure 7. KPIs values of the VRLE of CL over another period of academic years (former version, VRLE-1, and later version, VRLE-2).
Applsci 10 00915 g007
Table
Table 1. Questions (related to this article) designed on surveys for students.
Table 1. Questions (related to this article) designed on surveys for students.
QuestionNumberKPI
Rate from 1 to 10 the following features of the VRLE (1 the lowest rate and 10 the highest)1Motivation felt when using the VRLE
2Interactivity level of the VRLE
3Ease of use of the VRLE
4Usefulness of the didactic tool
Table
Table 2. Mean and standard deviation (σ) of students’ scores from surveys given from 2013 to 2017 to each question of the Table 1 for VRLEs developed formerly (VRLE-1, Figure 3).
Table 2. Mean and standard deviation (σ) of students’ scores from surveys given from 2013 to 2017 to each question of the Table 1 for VRLEs developed formerly (VRLE-1, Figure 3).
VRLE-1Academic YearNumber of StudentsQuestion 1Question 2Question 3Question 4
MeanσMeanσMeanσMeanσ
TPD2013–2014228.60.507.50.967.10.759.70.48
CL- -- -- -- -- -- -- -- -
TPD2014–2015208.50.617.20.897.20.839.70.47
CL7.50.619.00.859.00.749.70.59
TPD2015–2016208.10.726.50.897.00.739.70.49
CL7.50.698.50.768.80.759.60.60
TPD2016–2017207.50.876.20.777.10.799.70.47
CL7.30.738.20.829.10.659.80.44
TPD2017–201821- -- -- -- -- -- -- -- -
CL6.91.007.51.038.90.769.70.46
Table
Table 3. Mean and standard deviation (σ) of students’ scores from surveys given from 2017 to 2020 to each question of the Table 1 for VRLEs developed later (VRLE-2, Figure 4).
Table 3. Mean and standard deviation (σ) of students’ scores from surveys given from 2017 to 2020 to each question of the Table 1 for VRLEs developed later (VRLE-2, Figure 4).
VRLE-2Academic YearNumber of StudentsQuestion 1Question 2Question 3Question 4
MeanσMeanσMeanσMeanσ
TPD2017–2018219.20.758.30.607.40.809.70.46
CL- -- -- -- -- -- -- -- -
TPD2018–2019219.10.748.30.647.30.739.80.44
CL9.50.469.80.339.20.589.80.54
TPD2019–2020118.80.758.20.607.60.669.60.50
CL9.60.389.80.349.30.349.70.47

Share and Cite

MDPI and ACS Style

Vergara, D.; Extremera, J.; Rubio, M.P.; Dávila, L.P. The Technological Obsolescence of Virtual Reality Learning Environments. Appl. Sci. 2020, 10, 915. https://0-doi-org.brum.beds.ac.uk/10.3390/app10030915

AMA Style

Vergara D, Extremera J, Rubio MP, Dávila LP. The Technological Obsolescence of Virtual Reality Learning Environments. Applied Sciences. 2020; 10(3):915. https://0-doi-org.brum.beds.ac.uk/10.3390/app10030915

Chicago/Turabian Style

Vergara, Diego, Jamil Extremera, Manuel Pablo Rubio, and Lilian P. Dávila. 2020. "The Technological Obsolescence of Virtual Reality Learning Environments" Applied Sciences 10, no. 3: 915. https://0-doi-org.brum.beds.ac.uk/10.3390/app10030915

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop