Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review

Abstract

In this systematic umbrella review we aggregate the current knowledge of how virtual and augmented reality technologies are applicable to and impact remote learning in higher education; specifically, how they impact such learning outcomes as performance and engagement in all stages of higher education from course preparation to student evaluation and grading. This review was done as part of a state wide research effort of Latvia, to mitigate the impact of COVID-19 and specifically to provide a framework for a technological transformation of education in this context. In this work we search the Scopus and Web of Science databases for articles describing the use of virtual and/or augmented reality technologies in remote learning for higher education and their impact on learning outcomes. We identified 68 articles from which, after multiple screening and eligibility phases, nine review articles were left for extraction phase in which 30 structural elements with corresponding interventions and measured effects were extracted. Of these, 24 interventions had a measured effect on student performance (11 positive, seven negative, six no impact) and six interventions had a measured effect on student engagement (all six positive).

1. Introduction

The COVID-19 pandemic has highlighted the need for transformation of remote learning to not only survive a wave of crisis, but to potentially fit the new normal. A trend among governments across the world has been emerging to emphasize the potential for new technologies such as artificial intelligence and virtual/augmented reality to mitigate the problems remote learning has compared to on-site learning, such as academic dishonesty, decreased social aspects of studying, lack of practical kinesthetic interactions, problems keeping students’ attention, practice of technological boundaries, etc. As these are complex and expensive technologies, a decision for their use must be based not on technological hype but scientifically validated outcomes.

When it became clear that remote learning will have to be extended after the first wave, the government of The Republic of Latvia initiated a research programme in technological transformation of remote education. This is a 6 months long research programme providing a 500,000 EUR grant to an interdisciplinary team of researchers from multiple research institutions to evaluate how the Latvian society dealt with the coronavirus crisis and to provide recommendations for societal resilience in the future.This project has several work packages including study of societal dynamics in Latvia during this crisis, evaluation of labour market and employment structures, psychological effects of COVID-19 on individuals and families, evaluation of media and health communication, strategic communication and governance and finally education transformation. This work is part of this project, specifically the last work package and is aimed at finding evidence of the impact of virtual reality (VR) and augmented reality (AR) technologies on remote learning in higher education-specifically impact on performance and engagement. This is done through a systematic umbrella literature review-a review of literature reviews. This review conforms to PRISMA guidelines. Our research question is defined as "Which interventions using virtual and/or augmented reality technologies for students in higher education in remote learning have measured impact on student performance and engagement" and this question is defined according to PICOS approach in

Table 1. Research question of our umbrella review defined according to PICOS approach.

P	patients	students in higher education
I	intervention	use of virtual and/or augmented reality technologies in remote learning
C	comparison	none
O	outcome	impact on student performance and engagement
S	study design	systematic reviews

.

2. Methodology

In this section we describe the methodology of this systematic umbrella review (see flow diagram in Figure 1 for an overview).

This research conforms to PRISMA guidelines with the exception that the review protocol was not registered beforehand due to the time sensitive nature of the funding project related to the pandemic situation.

2.1. Identification

To identify the articles for inclusion in the review, a search was conducted in September, 2020 in two databases indexing peer-reviewed articles: Scopus and Web of Science. The scope was defined as “Use of virtual and/or augmented reality technologies in remote learning of higher education and their impact on learning outcomes”. For query results see

Table 2. Identification search queries and results.

Database	Query	Results
Scopus	TITLE-ABS-KEY ((“Virtual reality” OR “Augmented reality”) AND (((online OR distance OR remote) AND (study OR education OR learning)) OR e-learning) AND students) AND PUBYEAR > 1999 AND (LIMIT-TO (DOCTYPE, “re”))	66
Web of Science	TOPIC: ((“Virtual reality” OR “Augmented reality”)) AND TOPIC: (students) AND TOPIC: ((“e-learning” OR ((“online” OR “distance” OR “remote”) AND (“study” OR “education” OR “learning”)))) AND YEAR PUBLISHED: (>1999)	2

.

2.2. Screening

After removing duplicates 66 articles were screened by reading their titles and abstracts. The screening criteria was:

• only review articles are included;
• only articles about higher education are included;
• only articles using AR/VR technologies are included;
• only articles about remote learning are included

During the screening process 36 articles were discarded leaving 30 articles for the Eligibility phase.

Out of these 36 excluded articles 16 articles were not review articles, 13 articles were not about AR/VR technologies, four articles were not available, two articles were not in English and one article was not about higher education.

2.3. Eligibility

During the eligibility phase, the articles were randomly distributed among the authors for a full text analysis. The eligibility criteria was as follows:

• The article full text is available in English;
• The article contains a review of multiple articles;
• The article is about higher education;
• The article is about remote learning;
• The article is about AR/VR technologies.

In this stage 12 articles were excluded as ineligible and 18 articles were deemed eligible for inclusion and data extraction.

Out of these 12 excluded articles 10 were not review articles, one article was not available, and one article was not about higher education.

2.4. Included

Finally, the eligible articles were processed to extract all interventions that affect one of two variables: student performance or student engagement.

In this way 30 interventions were extracted. These interventions were then divided by the stage of the remote education process. The following stages were considered:

• Course design, content planning;
• Development of digital learning materials;
• Cognitive load and time management;
• Remote lecturing and content delivery;
• Feedback and interactivity;
• Social involvement, interaction;
• Remote practice, labs, kinesthetic learning;
• Remote evaluation.

3. Results

In total 68 articles were first identified, of which 66 were left after removing duplicates, 30 were left after the screening, 18 were left after eligibility checks and finally, 9 review articles were included in the final data extraction.

From those articles 30 structural elements and related interventions were extracted that contained AR or VR intervention in remote learning in higher education and had a measured impact on either performance or engagement.

The interventions related to augmented reality (AR) in these results are defined as interactive technology that allows to combine/complement/enhance real-world objects by computer-generated perceptual information on some sort of smart device, whule interventions related to virtual reality (VR) are defined as technology that allows to simulate real-world objects, events and interactions in digital computer generated domain/world/environment. Usually, VR term is used to describe virtual reality experience that could be obtained using VR-Headset, but in this paper VR term is used in a broader sense. The VR term also includes virtual laboratories on mobile devices or PC for experiments demonstration and performance.

In these interventions the impact on performance refers to observed change or no change to either the efficiency of accomplishing assignments, cumulative grade obtained during course or a metric that represents student ability to accomplish given task, using previously obtained knowledge. The impact on engagement refers to change or no change to the tendency of students to participate in study process, student enjoyment, satisfaction and feel of meaningfulness of ongoing process, willingness of students to attend classes, participate in in-class/after-class, course related, activities and interest inobtainment of additional materials.

Specifically 24 interventions described measured impact on performance (of those 11 had a positive impact, 7 had a negative impact and 6 had no effect) and 6 interventions described measured impact on engagement with all 6 reporting positive impact.

The education stage with the most identified interventions was “Remote practice, labs, kinesthetic learning”. There was one stage without any matching interventions-“Development of digital learning materials”. In 6 out of 9 reviewed papers VR/AR technologies were used in the field of medicine. Other fields where VR/AR technology was used were engineering, physics, chemistry. In two papers the field wasn’t specified and it was used for study purposes. The following percentages of review articles and original articles contained use of AR/VR technologies for the specific stages of remote educational process: Course design, content planning 11/2%, Development of digital learning materials 0/0%, Cognitive load and time management 11/8%, Remote lecturing and content delivery 11/6%, Feedback and interactivity 11/4%, Social involvement, interaction 11/8%, Remote practice, labs, kinesthetic learning 89/63% and Remote evaluation 11/8% respectively.

The identified interventions together with the intervention stage, intervention value, intervention effect, review article in which the intervention was identified and list of original articles supporting the intervention results can all be seen in the results in

Table 3. Extracted structural elements with corresponding interventions and measured effect.

No.	Stage	Intervention	Variable	Effect	Review Article	Original Articles
1	Course design, content planning	Use of AR technologies with insufficient pedagogue  training	Performance	Decrease	[1]	[2]
	Development of digital learning materials	-	-	-	-	-
2	Cognitive load and time management	Usage of complex AR simulations for students, who are not familiar with this complex technology, leading to confusion and astoundment	Performance	Decrease	[1]	[3]
3	Cognitive load and time management	Use of AR with insufficient support that can confuse learners and delay the learning process	Performance	Decrease	[1]	[4]
4	Cognitive load and time management	Unassisted AR experience with high load/complex course leading to cognitive overload	Performance	Decrease	[1]	[3,5,6]
5	Remote lecturing and content delivery	Use of AR in lecturing and content delivery improves focus, attention levels, study process becomes more enjoyable, fun and satisfying	Engagement	Increase	[1]	[7,8,9]
6	Feedback and interactivity	Use of AR for calculus and abstract concept visualisation promotes mathematical and cognitive skills in engineering students	Performance	Increase	[1]	[10,11]
7	Social involvement, interaction	Use of AR for better face-to-face and remote interactions and collaborations	Engagement	Increase	[1]	[12]
8	Social involvement, interaction	Use of AR enables interactions and collaborations which are more similar to natural face-to-face collaboration than screen-based interaction	Engagement	Increase	[1]	[13]
9	Social involvement, interaction	Use of AR in academic settings improces learners’ motivation and engagement, especially when game-based approaches are utilized	Engagement	Increase	[1]	[14,15]
10	Remote practice, labs, kinesthetic learning	Use of AR had positive influence on learning rate and memorization process of medical students	Performance	Increase	[1]	[16]
11	Remote practice, labs, kinesthetic learning	Use of AR increases motivation, engagement, interest and knowledge retention	Engagement	Increase	[1]	[16,17,18]
12	Remote practice, labs, kinesthetic learning	Use of virtual worlds promotes student motivation and engagment	Engagement	Increase	[19]	[20]
13	Remote practice, labs, kinesthetic learning	Use of virtual worlds promotes spatial knowledge and capability to transfer the knowledge to real world skills	Performance	Increase	[19]	[20]
14	Remote practice, labs, kinesthetic learning	Use of VR for interactive presentation and visualization of complex physical experiments has positive effect on learning process	Performance	Increase	[21]	[22]
15	Remote practice, labs, kinesthetic learning	Wirtual worlds are as effective for learning as the more traditional Human Patient Simulator	Performance	Increase	[23]	[24]
16	Remote practice, labs, kinesthetic learning	Use of virtual labs in physics and chmeistry is as efficient as traditional labs	Performance	No change	[25]	[26]
17	Remote practice, labs, kinesthetic learning	Use of virtual partner simulation for medical students reduced performance	Performance	Decrease	[27]	[28]
18	Remote practice, labs, kinesthetic learning	Use of virtual partner simulation for medical students didn’t change the performance	Performance	No change	[27]	[29]
19	Remote practice, labs, kinesthetic learning	Use of VR for medical students leads to faster mean completion time, lower directional error in Flexible Sigmoidoscopy	Performance	Increase	[30]	[31]
20	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students reduces the mean score and the number of individually completed retroflexion cases	Performance	Decrease	[30]	[32]
21	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students did not change the average task time and patient satisfaction	Performance	No change	[30]	[32]
22	Remote practice, labs, kinesthetic learning	use of VR simulation for medicine students in flexible sigmoidoscopy increases patient comfort level	Performance	Increase	[30]	[33]
23	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students in flexible sigmoidoscopy did not change procedural skills such as independence, identifying pathology, landmarks, performing biopsies, adequate visualization	Performance	No effect	[30]	[33]
24	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students improves colonoscopy capabilities	Performance	Increase	[30]	[34,35,36,37,38,39,40,41,42,43]
25	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students made student Esophagogastroduodenoscopy capability worse	Performance	Decrease	[30]	[44]
26	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students improved student Esophagogastroduodenoscopy capability	Performance	Increase	[30]	[45,46,47,48]
27	Remote practice, labs, kinesthetic learning	Use of VR simulation for medical students improved endoscopic retrograde cholangiopancreatography (ERCP) capabilities	Performance	Increase	[30]	[49,50,51]
28	Remote practice, labs, kinesthetic learning	Use of VR simulation for students - nurses shows that after investment in training new intermediary students and a group of experts had equivalent performance	Performance	Increase	[52]	[53]
29	Remote evaluation	Use of VR in remote evaluation is able to discriminate between expert and novice performers	Performance	No change	[54]	[55,56,57]
30	Remote evaluation	Computer based simulations and virtual standard patient examinations were unable to distinguish between different exeprience levels	Performance	No change	[54]	[58]

.

4. Discussion

In order to determine if AR/VR technologies might be beneficial to technological transformation of remote learning, in this article we describe an umbrella review of related literature.

The main limitations of the review are the inability to access 5 of the identified articles and inability to properly analyze 2 identified articles which were not in English.

The results show that most of the current experiments pertain to organizing laboratory or practical exercises within virtual or augmented reality in cases when physical presence is not feasible. This overall seems to provide positive results, except for a few cases [28,32,44]. In cases where practical, spatial or kinesthetic skills are required the results were very encouraging, especially in medicine related education [23,30,52].

In addition to the specific results extracted, the literature also suggests that virtual/augmented reality is not capable of completely replacing on site studies, because whenever it was tried, the student grades suffered [32,44].

As can be seen in Table 3 in multiple studies the mere fact of VR/AR usage already created an impact on performance or engagement. This could be explained by multiple mechanisms, the three more plausible ones are (a) either the AR/VR technologies actually impact the learning process directly, or (b) they impact the outcomes indirectly e.g., these technologies might improve social contact, which in turn improves overall outcomes or (c) the result might be due to a novelty and thus diminish in time as well as stop functioning if new novelty technique is introduced. The latter can only be distinguished if the same group of students is followed through several semesters.

The fact that in all interventions where engagement was measured, the engagement increased, leads us to speculate that novelty of technology used has a direct positive impact on engagement. If this is the case, it means that novelty itself is a potential intervention, and any newly hyped technology could provide similar results. If this is true then another question should be researched—whether there exists a cumulative novelty resistance and whether it accumulates for a person in general with any novelty, or just a subset. Does “acumulative novelty resistance”-the effect when introducing next new technology to study process with purpose of increasing the engagement and/or performance of students-have any effect due to satiation.

The possibility of such novelty requirements could lead to future experiments to determine the best way to keep the engagement and performance of students until the end of the study year.

In every study that showed increase of performance or engagement, the course was well designed and teachers had good qualification to use benefits of AR/VR for learning purposes, however, AR/VR is not a panacea. In cases when students or teachers were not familiar with AR/VR technologies or when courses were not adapted well for AR/VR usage or when teacher of the course was not prepared enough to work with AR/VR, a noteable decrease in performance was noted in the articles explored [2,3,4,5,6,28,32,44]. This leads to a highly vital conclusion-an unprepared teacher can’t prepare a student well.

The potential solution is:

• create courses for teachers and lecturers on how to prepare/adapt courses for AR/VR;
• create a framework that would allow teachers easily prepare/adopt their material for AR/VR;
• Do not overload students with need to get familiar with AR/VR in a short time. there should be a possibility to use classical methods to get through the course;

At the same time AR/VR proved that it could help to understand abstract and complex content more easily due to good visualisation capabilities and interactivity. In multiple of the reviewed articles it was shown that kinesthetic learning, when instead of a classic lecture, students are working in 3D world, performing experiments alone or together with a teacher, is much more efficient than, previously mentioned, classic method [10,11,16,17,18,20,24,30,31].

The creation of AR/VR adopted courses could have a great effect on knowledge availability. An opinion in the educational community and society at large that has been reinfoced by the 2020 lockdown, is that online learning currently could be the future of education. If this is the case, then based on the fact that multiple papers show that AR/VR labs are of similar benefits as traditional “offline” labs with real equipment [24,26], it could be argued that properly adopted AR/VR based courses could, potentially, rise good, qualified specialists all around the globe, not only in local regions, democratizing education in hands on skills.

The performance is not the only factor that we need to take into account, emotional wellness is at least as important, as performance in terms of grades. Scientific groups that were researching Virtual Worlds as substitution for university environment showed that students feel much better if they could see their avatar in some virtual world, they could associate with, walk around virtual campus and explore it, like it would be real university [12,13].

It also must be noted that VR is still relatively complex and expensive technology and even though the prices are going down, still outfitting each student with VR/AR systems for remote learning is a complex and expensive task, which suggests that some of the future remote learning could happen from semi-centralized labs outfitted with VR/AR technologies, where students could arrive to work, but educators would connect remotely.