Next Article in Journal
An Enactivist Account of Mind Reading in Natural Language Understanding
Next Article in Special Issue
Association of the Big Five Personality Traits with Training Effectiveness, Sense of Presence, and Cybersickness in Virtual Reality
Previous Article in Journal
Designing Tangible as an Orchestration Tool for Collaborative Activities
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Temporal Development of Sense of Presence and Cybersickness during an Immersive VR Experience

1
Department of Psychology, NTNU—Norwegian University of Science and Technology, 7034 Trondheim, Norway
2
Department of Social Studies, University of Stavanger, 4021 Stavanger, Norway
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2022, 6(5), 31; https://0-doi-org.brum.beds.ac.uk/10.3390/mti6050031
Submission received: 17 February 2022 / Revised: 8 April 2022 / Accepted: 11 April 2022 / Published: 22 April 2022
(This article belongs to the Special Issue 3D Human–Computer Interaction (Volume II))

Abstract

:
Following the advances in modern head-mounted displays, research exploring the human experience of virtual environments has seen a surge in interest. Researchers have examined how to promote individuals’ sense of presence, i.e., their experience of “being” in the VE, as well as to diminish the negative side effects of cybersickness. Studies investigating the relationship between sense of presence and cybersickness have reported heterogeneous results. Authors that found a positive relation have argued that the phenomena have shared cognitive underpinnings. However, recent literature has reported that positive associations can be explained by the confounding factor of immersion. The current study aims to investigate how cybersickness and sense of presence are associated and develop over time. During the experiment, participants were exposed to a virtual roller coaster and presented orally with questions aimed to quantify their perceived sense of presence and cybersickness. The results of the experiment indicate that cybersickness and sense of presence are both modulated by the time spent in the virtual setting. The utilized short measures for sense of presence and cybersickness were found to be reliable alternatives to multi-item questionnaires.

1. Introduction

Following the technological and development of modern head-mounted displays (HMD), research investigating the effects of virtual environments (VE) on the human psyche and behavior has seen a surge in interest [1,2,3,4]. Recently, VEs has been successfully used in many applied fields, such as medicine and education [5,6,7], and work safety training [8]. The possibility of inducing the subjective perception that a simulation is reality has been a critical goal in the development of Virtual Reality (VR) systems [4]. To achieve this goal, developers have attempted to promote the user’s sense of presence—where the conscious awareness of the simulated environment ceases [9,10]. This is not to be confused with factors of the technology itself, as the quantifiable factors of VR are often referred to as immersion and the immersive qualities of the system. These objective qualities are responsible for promoting the user’s sense of presence [11,12].
The use of modern HMD-mediated VR has often been reported to induce unpleasant side effects, such as nausea and disorientation as well as other uncomfortable symptoms [13,14,15,16]. These symptoms are often referred to as “cybersickness” (CS; e.g., [17]); however, the term “Simulator Sickness” is also often used in the scientific literature. Despite this, a differentiation between the aforementioned terminologies has been proposed by the scientific literature (see [18]). These terms have often been used interchangeably [4]. For simplicity, in the present study, we will use the terminology cybersickness (CS) throughout the manuscript. In addition to being subjectively uncomfortable, symptomatology related to CS has generally been found to decrease the user’s overall enjoyment of the VR, quality of experience, and performance [19,20,21,22].
As both CS and sense of presence are pivotal aspects of the human experience of a VE, understanding their nature, causes, and interactions is essential for developing effective, usable, and enjoyable VR systems. However, the literature examining the relationship between sense of presence and CS has been plagued by discrepant findings [9].

Related Works and Study Objectives

Some lines of evidence have suggested that CS and sense of presence are inversely related. It has been proposed that individuals who experience high CS become more focused on their negative physiological symptoms related to CS and therefore become distracted from the VR experience and less present in it [23]. Evidence for such theory was presented by Usoh et al. [24], who examined the nature of different modes of moving in a VE, finding the oculomotor facet of the Simulator Sickness Questionnaire (SSQ; [25]) to be higher when sense of presence was low. The authors proposed that symptoms such as eyestrain and headaches are the CS symptoms that most strongly lead to an internal focus. A similar argument for an inverse relationship between sense of presence and CS was proposed by Witmer et al. [23] that suggested that CS symptoms may lead to distraction from the VE, impairing the experienced sense of presence.
The recently published literature review of Weech et al. [9] concluded that an inverse relationship between CS and sense of presence is commonly described in empirical studies reporting data for both sense of presence and CS. The authors concluded that most studies point toward a negative correlation between sense of presence and CS. The authors also proposed that the results of these published studies reporting a positive relationship between sense of presence and CS potentially could be explained by uncontrolled covariate variables (e.g., the levels of immersion provided by different VR systems employed in the studies). This conclusion is supported by studies finding immersive factors modulating both sense of presence and CS, such as an increased field-of-view [26,27,28]. Similarly, higher fidelity stereoscopic 3D viewing has been shown to promote both sense of presence and CS [29,30].
Evidence for a positive association between sense of presence and CS has been reported in the literature as well. Liu and Uang [31] found a strong positive correlation (r = 0.67) between sense of presence and CS. Moreover, Lin et al. [20] examined the relationship between enjoyment, sense of presence, and motion sickness, showing that sense of presence and CS may be positively associated. Kim et al. [32] found a positive correlation between CS and the involvement factor of the Immersive Tendencies Questionnaire (ITQ; [10]). However, they found a negative correlation between CS and the “user control” factor of the Presence Questionnaire (PQ; [6]). The authors proposed that individuals experiencing a high level of sense of presence during the VR simulation pay less attention to the negative symptoms of CS. It has also been suggested that the experience of CS enhances the user’s sense of presence, as the negative symptoms further strengthen the individuals’ understanding of being influenced by the VE [33].
While several studies have examined the association of CS and sense of presence, it has mostly been performed using retrospective questionnaires, asking the participants about their experience in VR after the VR experience (see also [9]). The current experiment aims to further the knowledge and understanding of how CS and sense of presence develop and how they are associated during a VR exposure.
Within the framework of the current experiment’s goal of exploring temporal development of sense of presence and CS, measurements must be made while the participants are experiencing the VR while attempting to disrupt the sense of presence of the user as little as possible. Multi-item post-test questionnaires are commonly used to assess sense of presence and CS [34]. The most used questionnaires for measuring sense of presence are the PQ [10,34] and the Simulator Sickness Questionnaire (SSQ) [25]. However, multi-item questionnaires may be time consuming to administer, and therefore, completing the questionnaire during a VR experience could significantly disrupt the sense of presence experienced by the user. Several studies have examined continuous physiological indices of sense of presence (for a recent study, see [35]); however, these have several drawbacks, such as being highly modulated by confounding or secondary effects of presence such as arousal, stress, or emotion [36]. The current experiment employs short, single-item measurements administered orally by the experimenter during the VR session. Short measurements have been used in research on sense of presence (see the Brief Measure of Presence) [36] and CS (see the Fast Motion Sickness Questionnaire) [37,38]. Research has shown single items to be valid and reliable, even if they do not address all the aspects of the psychological constructs that it aims to measure [34,39].
Little work has been completed to specifically examine the impact of exposure time on the development of sense of presence. Melo et al. [40] found evidence that sense of presence will increase until reaching a threshold, arguing that users might need some time to adapt to the VE. Melo et al. [40] examined the effect of different exposure times (1, 3, 5, and 7 min) on sense of presence and CS, finding no significant differences. However, further analyses indicated that male users required more time compared to female users to develop a higher level of sense of presence. It has also been proposed that for instance habituation and boredom negatively affect users’ sense of presence in prolonged exposures [41,42].
Several studies have shown that long exposure times in a VE generally are associated with increased CS symptom severity [43]. Previous research has found CS to steadily increase over time [36], but it has also shown the existence of a time threshold, after which the users report CS levels to plateau or decrease [44,45,46]. CS has also been found to have a lower threshold at which symptoms do not negatively affect the VE experience [47,48].
The discrepant findings concerning the relation between CS and sense of presence are a challenge for evaluating their relationship for practical purposes. While the recent systematic review of Weech et al. [9] suggested that the immersion qualities of the VR systems may explain the positive correlations, they were unable to explain the full complexity of the relationship. Moreover, most studies examining the relationship between sense of presence and CS are based on retrospective questionnaires. Therefore, the effect of exposure time has rarely been accounted for. It is therefore unknown if sense of presence is positively associated only at certain time-points during the simulation (e.g., at the beginning or at the end) or if the association is generalized for the whole duration of the VR experience.
The current experiment sets out to examine the effect of exposure on the relationship between CS and sense of presence, and how the phenomenon develops throughout time. More specifically, we aim to explore the following questions: (1) How do sense of presence and CS develop over time during a virtual roller-coaster scenario? (2) How do CS and sense of presence relate throughout the VR experience? (3) Are single-item questionnaires of CS and sense of presence valid measures compared to its equivalent, namely the multi-dimensional questionnaires?

2. Methods

2.1. Participants

Forty participants (8 males) took part in the study. Their age ranged from 18 to 26 years (M = 21.23, SD = 0.34). Participants were selected from the student population at the Norwegian University of Science and Technology (Trondheim, Norway). Only participants with corrected to normal vision and with no prior history of epilepsy participated in the experiment. The study was conducted after approval from the Norwegian Data Protection Agency (NSD).
Participants signed the informed consent and completed the Simulator Sickness Questionnaire (SSQ) before the beginning of the experiment (baseline SSQ). The experiment was described as an investigation into users’ experience during a virtual roller-coaster simulation. Then, participants were fitted with the HMD, the Oculus Go, which has a single screen with a resolution of 2560 × 1440 running at 60 Hz. The software used for the VE simulation was “Epic Roller Coaster” by B4T Games using the T-Rex Kingdom environment (version as of 1 June 2020; see Figure 1). A roller-coaster environment was chosen, which is in line with the goal of examining the association of CS and sense of presence, because of a roller coaster’s intrinsic ability to induce sickness.
The specific roller-coaster environment was chosen as it contains highly engaging elements, such as stones falling toward the participants. The copyright holders of the software gave written consent to the use of the software for research purposes. During the experiment, participants were presented with two one-item questionnaires, a Modified Fast Motion Sickness Scale (MFMS) and the Brief Measure of Presence (BMP).
Once the HDM was fitted, participants viewed a pause screen. While in the pause menu, participants were able to look around at a static image of the simulation. Participants were then (orally) asked the questions of the MFMS and BMP questionnaires. The first questions were asked while participants were watching the pause screen at the beginning of the simulation, and subsequent questions were asked by pausing the simulation every 65 s for a total of 7 measures. Each pause had the duration of around 20 s, depending on the time participants needed for answering the two questions. Including pauses, the duration of the experiment was around 10 min. The simulation was paused to give participants the opportunity to reflect on their experience while attempting to minimize the impact on their sense of presence.
A short oral explanation was given for sense of presence and CS prior to the experiment to ensure that participants understood the questions. The simulation varied in its intensity throughout the experiment. After the VR exposure, participants were asked to fill out the Presence Questionnaire (PQ), the SSQ once more, and report their age and sex.

2.2. Questionnaires

2.2.1. Simulator Sickness Questionnaire

The SSQ by Kennedy et al. [25] was used to measure participants’ degree of CS after the experiment. The tool is widely used in VR research to evaluate the subjective severity of CS [13]. The questionnaire was developed as a customization of the “Pensacola Motion Sickness Questionnaire” to distinguish cybersickness from motion sickness as well as provide subscales and a scoring method for tracking and monitoring symptoms [25].
The questionnaire consists of items describing 16 common VR-related physiological symptoms that participants rate on a scale from 0 (no symptoms) to 1 (slight symptoms), 2 (moderate symptoms), and 3 (severe symptoms). According to Kennedy et al. [25], the 16 symptoms can be divided into three factors: oculomotor, disorientation, and nausea. The final score is obtained by multiplying the total score for each of the three factors by constant values. The SSQ is considered to be a reliable scale for assessing overall simulator sickness severity with reasonably powerful subscale scores [49].
To control for any pre-existing symptoms the participants might experience (e.g., fatigue), the SSQ was performed both before and after the VR exposure, following the suggestions reported in Kennedy et al. [25]. The score was calculated by subtracting the pre-experiment measure from the post-experiment measure.

2.2.2. The Presence Questionnaire

The Presence Questionnaire (PQ version 2.0) was administered to examine the users’ sense of presence at the end of the VR experience. The PQ 2.0 consists of 32 items across six subscales: auditory stimulation, resolution, involvement/control, naturalness, haptic response, and interface quality [10]. Each item uses a Likert scale ranging from 1 to 7, with 1 describing the lowest level of sense of presence and 7 describing the highest level. The PQ was chosen as it is the most used questionnaire in VR research [50]. The current VE does not encompass interaction with the environment or other people; thus, 13 items were discarded (19 items were kept). The PQ has tested to be a reliable measure of sense of presence [51] and has established a good internal consistency with a Cronbach’s alpha of 0.81 [10].

2.2.3. Modified Fast Motion Sickness Scale

In the current study, a modified version of the single-item Fast Motion Sickness Scale (FMS) [38] was used for measuring CS repeatedly and verbally during the simulation. The questionnaire has a scale from 0 (no sickness at all) to 20 (frank sickness). However, to create parity between the short measure of CS and sense of presence, and to make the wording more understandable, the current experiment used a scale from 1 (no sickness at all) to 10 (very sick). The modified version of the FMS used in the current experiment will be named MFMS (Modified Fast Motion Sickness).
As suggested by Keshavarz and Hecht [38], participants were advised to focus on nausea, general discomfort, and stomach problems when assessing their degree of CS. The authors argue that FMS is a reliable and valid measurement for use in real and virtual scenarios. Moreover, Van Baren [52] found single-item scales to capture the moment’s experience more accurately and also to be less sensitive to memory deterioration compared to multi-item inventories. Keshavarz and Hecht [38] found peak FMS scores to correlate with total SSQ (r = 0.785), with the highest association being with the nausea subscale (r = 0.828).

2.2.4. Brief Measure of Presence

The brief measure of presence (BMP) by Bouchard et al. [53] was used to measure participants’ sense of presence orally. During the experiment, participants were asked to rate their degree of presence on a scale from 1 to 10, answering the question: “To what extent do you feel present in the virtual environment, as if you were really there?”
As suggested by Bouchard et al. [54], participants were informed that sense of presence is defined as their subjective experience of being in the virtual roller coaster. Bouchard et al. [53] validated the measure in two studies, finding the test–retest reliability to be 0.81 (p < 0.001) and 0.83 (p < 0.001). The BMP was found to correlate with the PQ (r = 0.45 to 0.53) in an experiment comparing anxiety and sense of presence [54].

2.3. Data Analysis and Statistics

The software used for the statistical analyses was IBM SPSS Statistics (version 26) and Matlab (2019a). To assess the association between sense of presence and CS during the simulation, skipped correlation analysis (Pearson’s r) was performed between the MFMS and BMP. The analyses were computed separately between each of the seven measures in time using the Robust Correlation Matlab toolbox (function “robust_correlation”). Skipped correlations were performed to control for outliers by performing a bivariate outlier analysis and calculating Pearson’s correlation on the remaining data. Skipped correlations are calculated by estimating the robust center of the data, identifying outliers by estimating the interquartile range of each data point from the central data point. Significance levels are estimated based on confidence intervals (95%). For more information, see Pernet et al. [55]. To investigate the reliability of the short measures, skipped correlations were performed between SSQ and each measure of the MFMS and between the PQ and each measure of the BMP. To further examine the association between sense of presence and CS, skipped correlation analyses were performed between the PQ and SSQ.
Sample cross-correlations were calculated using cross-correlation analysis (Matlab function crosscorr()). Such analysis was performed to assess the relationship between sense of presence and CS over time. Results are displayed graphically, with the x-axis representing time delays and the y-axis representing the correlation coefficient for each measure of time lag.

3. Results

3.1. Descriptive Statistics

The questionnaires were first analyzed descriptively. The used 19 items of the PQ had a mean score of 74.45 (SD = 0.11.97) for a maximum possible score of 133. The SSQ had a mean score of 21.41 (SD = 31.04). Descriptive data for the MFMS and BMP are shown in Table 1.

3.2. MFMS and BMP Scores across Seven Time Intervals

The average scores of short questionnaires (MFMS and BMP) were plotted on a graph across the seven time intervals (Figure 2) to assess the association of CS and sense of presence. The graph shows reported CS and sense of presence to develop comparably across time. These results suggest that both sense of presence and CS are similarly modulated (same direction of the effect) during the VR simulation.

3.3. Correlation Analyses

Pearson’s correlation analyses were performed to assess the relationship between sense of presence and CS at each measured time-point. Due to the relatively small sample size, and to reduce the effect of univariate and bivariate outliers affecting the results, correlations were performed using the robust correlation algorithm (skipped correlation method) [55]. Such an analysis method was used for estimating linear correlation first for individuate univariate and bivariate outliers using the mathematical method described in Wilcox [56], and then performing Pearson’s correlation analyses returning 95% CIs to estimate statistical significance.
The results yielded a significant correlation at the 5th (r = 0.30, CI = [0.0055, 0.05644]) and 7th time interval (r = 0.37, CI = [0.0379, 0.6334]). All correlations trend positively, except between the pre-exposure measures that trend negatively. Figure 3 shows the correlation analyses at every time-point (seven) when the short questionnaires were orally administered.

3.4. Post-Experimental Measures of PQ and SSQ

Skipped Pearson’s correlation analyses were performed between the post-experiment questionnaires, the PQ, and the SSQ. The analyses yielded a non-significant relationship. Scatter plot of the correlation analysis is presented in Figure 4.

3.5. Short Measures of CS and Sense of Presence

To investigate whether MFMS and BMP are reliable measures of their respective constructs, skipped correlations were performed between each measure of MFMS and the SSQ, and between each measure of the BMP and the PQ. Between the SSQ and MFMS, the correlations showed significant association between time intervals, 3 (r = 0.47, CI = [0.1730, 0.7129]), 4 (r = 0.56, CI = [0.2827, 0.7651]), 5 (r = 0.56, CI = [0.2754, 0.7721]), and 6 (r = 0.53, CI = [0.2668, 0.7344]). Between the MFMS and the PQ, the analyses yielded significant positive associations between time interval 2 (r = 0.43, CI = [0.0319, 0.6975]), 3 (r = 0.49, CI = [0.1224, 0.7570]), 4 (r = 0.50, CI = [0.0976, 0.7789]), 5 (r = 0.49, CI = [0.1154, 0.7690]), 6 (r = 0.53, CI = [0.2815, 0.7192]), and 7 (r = 0.66, CI = [0.4332, 0.8187]). These results indicate that the short measures are reliable in comparison with multi-item questionnaires.

3.6. Cross-Correlations between MFMS and BMP

Figure 5 shows sample cross-correlation values for MFMS and BMP at various time intervals and for shifted ± 5 or −5 time intervals. The analysis was computed using the Matlab function crosscorr(). The first measurement (made before the beginning of the experiment as a control measure) was not included in the figure. The cross-correlation function shows that the highest correlation between the sense of presence and cybersickness measure is at 0-lag measure (i.e., when sense of presence and CS are at the same point of the VR). These results give evidence for the notion that CS and sense of presence are modulated similarly in time and might further indicate the existence of confounding variables causing the association of sense of presence and CS. These analyses also suggest that there is not an observable cause–effect relationship between sense of presence and CS (at least not identifiable taking into consideration the time windows used in the present experiment).

4. Discussion

In the present study, we aimed to study the association of CS and sense of presence during and after an exposure to an immersive VR environment. Specifically, the study explored how the phenomena of CS and sense of presence temporally develop and are associated, using repeating measures throughout the VR experience.

4.1. Temporal Development of CS and Sense of Presence

Descriptive statistics (Figure 2) show that both CS and sense of presence follow a similar pattern across the experimental session. Correlation analyses between each measure in time indicate significant correlations at the fifth- and seventh-time interval, suggesting that the two phenomena are associated according to the measures taken at the end of the experiment. The scores for both sense of presence and CS generally increased during the simulation. As immersive characteristics of the VR system are constant during the simulation, immersion alone cannot account for the positive correlations between CS and sense of presence. Such findings challenge the interpretation proposed by Weech et al. [9], according to whom different immersion characteristics of the VR systems could explain a positive association between sense of presence and CS.
The parallel development of sense of presence and CS in the current experiment might be specific for the contents of the simulation. Specifically, experiencing a roller coaster ride, either virtual or real, may be sickness inducing (even though in the case of a real roller coaster, we would refer more properly to the phenomenon of motion sickness). The onset and development of CS symptoms in the virtual roller coaster ride may have given users the suggestion of realism in the simulation and therefore increased their sense of presence. In other words, the feeling of sickness would reaffirm the idea that the participant is experiencing a state as if they should be in a real roller coaster.
If the specific type of scenario in our experimental setting explains the positive association in the current study, our findings may not be generalizable for different types of VR, such as static simulations or when the user is an active participant, as it may suggest that the content of the VE is crucial for determining the directional relation between sense of presence and CS. Future research should strive to uncover which aspects of a simulation influence the association of sense of presence and CS.
Questions remain as to what subjective measures sense of presence measure and whether the participants of our experiment rated specifically the degree to which they have experienced sense of presence in the VE or if their rating was unconsciously affected by other subjective experiences such as engagement, excitement, and attention. Subjective measures of sense of presence may not adequately differentiate between objective influences (immersion) and the perceived sense of being there [53].
The highly varying measures of sense of presence in the current experiment seemingly based on the content of the simulation questions whether sense of presence is practically separable from immersion.
The temporal changes found in the current study are in line with Melo et al. [40], who reported sense of presence to increase with the time spent in the VR.
Longer VR-exposure times have generally been found to increase CS, until a threshold is reached, after which CS level stabilize or decrease [44,45,47]. The current results align with these findings, indicating a linear increase in CS levels during the first phase of the experiment followed by a stabilization of the reported CS level. We found inconclusive evidence for the CS levels decreasing because of habituation effects.

4.2. The Impact of Time-Delayed Sense of Presence on CS

Cross-correlation analyses were performed between each measure for the scores of the one-item sense of presence and CS questionnaire. These analyses were performed to investigate whether time-shifted values of sense of presence correlate with CS, attempting to assess if a temporal cause–effect relationship between the variables could be evidenced from the data. In the current study, the cross-correlation at zero lag was found to be the time with the higher association between sense of presence and CS.

4.3. The Validity of MFMS and BMP

The current study used single-item questionnaires to measure sense of presence and CS. Single-item measures have been used in several study designs as a substitute for multi-item questionnaires [34]. However, only a few studies have used single-item measures together with multi-items inventories. In the current study, we compared the single-item measurements for CS (MFMS) and sense of presence (BMP) with widely used multi-item questionnaires to assess sense of presence and CS.
Correlation analyses between the PQ and the BMP were significant and showed a moderate correlation level, ranging from r = 0.43 to 0.66. This finding is similar to the data reported by Bouchard et al. [53], which found correlations of r = 0.45 and r = 0.53 to the PQ in two separate studies.
Questions remain about which strategy participants in the experiment used to rate the retrospective questionnaires. Participants may either rate their peak, average, or ending levels of sense of presence. In the current study, correlations were generally similar across all significant measurements but were the highest for the central part of the VR experience.
The correlations between the MFMS and the SSQ ranged from r = 0.47 to 0.56. These correlations are somewhat lower than what Keshavarz and Hecht [38] found with a peak MFMS score to correlate with SSQ at r =0.785.
The correlation coefficients were generally increasing throughout the experiment, with the highest correlation found at the last measurement. The roller-coaster experience starts out slow with a slow movement of the chart before it further develops into a more intense stimulation toward the latter parts of the simulation.
The reliability and validity of single-item measures may be questioned as they do not fully address all the aspects of the psychological constructs in question. However, when multi-item questionnaires are not applicable, the current studies suggest that both the MFMS and BMP seem to be reasonably valid measurements of the phenomena they aim to measure.

4.4. Limitations/Future Research

Although the current study significantly contributes to the understanding of the relationship between sense of presence and CS, it has several limitations. First, the data were obtained from participants using a specific VR system (Oculus Go). The system is, despite its relatively recent release (2018), quickly becoming obsolete, providing a lower quality of experience compared to more recent HMD systems. Future studies may seek to examine whether a higher fidelity and more immersive VR system would influence the relationship between sense of presence and CS compared to what is reported in the present study.
A second limitation of the study was that the experimenter had to pause every 65 s to ask the users to give their rating of sense of presence and CS. The effect of the several pauses could be especially significant for the participant’s sense of presence, as a pause in the flow of the experience would inevitably break the illusion of “being there.” Moreover, asking participants to contemplate on their own experienced sense of presence could make them more aware of the separation between the virtual and real world—reducing their sense of presence. A potential remedy for the effect of pauses influencing users rating might be to implement other types of continuous measuring techniques, such as a dial [44]. However, this may be too complicated to implement when measuring two simultaneous phenomena. The simulation’s length might have been too short for the detection of a possible habituation effect of the VR experience on sense of presence and CS.
Furthermore, the study sample population might be a limitation. The participant’s group is somewhat homogeneous, as they were mostly recruited from the same university campus. Moreover, the predominance of female participants might be a problem for the results generalization, as it has been reported in the literature that females are more sensitive to CS [57]. Individual factors that were not controlled for in the present study (see e.g., [58]) may have had an impact on the level of reported sense of presence or CS.
Lastly, in the analysis, we performed multiple correlations without controlling for type 1 errors. Therefore, the results reported in the present study are of an exploratory nature, and replication studies may be needed to confirm our results.

Author Contributions

S.T. contributed for the conceptualization of the experiment, the lab. work (including participants testing). He took the main responsibility in writing the first draft of the manuscript. A.S.R. helped in commenting and reviewing the manuscript. S.G. was responsible for the supervision of all stages of the scientific project, substantially contributing in all phases of the experiment, data analysis, and manuscript writing. All authors have read and agreed to the published version of the manuscript.

Funding

The present research was supported by internal fundings from the Department of Psychology (Faculty of Social Sciences) of the Norwegian University of Science and technology.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki. The Norwegian Centre for Research Data (NSD) was notified prior to the start of the project according to the Norwegian regulations.

Informed Consent Statement

Written informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data supporting the finding of the present research can be found in Zenodo, DOI:10.5281/zenodo.6451810, accessed on 16 February 2022.

Conflicts of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Bird, J.M. The use of virtual reality head-mounted displays within applied sport psychology. J. Sport Psychol. Action 2019, 11, 115–128. [Google Scholar] [CrossRef]
  2. Bouchard, S.; Rizzo, A. Applications of Virtual Reality in Clinical Psychology and Clinical Cognitive Neuroscience—An Introduction. In Virtual Reality for Psychological and Neurocognitive Interventions; Springer: New York, NY, USA, 2019; pp. 1–13. [Google Scholar] [CrossRef]
  3. Pastel, S.; Chen, C.-H.; Petri, K.; Witte, K. Effects of body visualization on performance in head-mounted display virtual reality. PLoS ONE 2020, 15, e0239226. [Google Scholar] [CrossRef] [PubMed]
  4. Grassini, S.; Laumann, K. Questionnaire Measures and Physiological Correlates of Presence: A Systematic Review. Front. Psychol. 2020, 11, 349. [Google Scholar] [CrossRef] [Green Version]
  5. Tang, Y.M.; Chau, K.Y.; Kwok, A.P.K.; Zhu, T.; Ma, X. A systematic review of immersive technology applications for medical practice and education-Trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ. Res. Rev. 2021, 35, 100429. [Google Scholar] [CrossRef]
  6. Fong, K.N.K.; Tang, Y.M.; Sie, K.; Yu, A.K.H.; Lo, C.C.W.; Ma, Y.W.T. Task-Specific Virtual Reality Training on Hemiparetic Upper Extremity in Patients with Stroke. Virtual Real. 2021, 1–12. Available online: https://0-doi-org.brum.beds.ac.uk/10.1007/s10055-021-00583-6 (accessed on 28 September 2021). [CrossRef]
  7. Grassini, S. Virtual Reality Assisted Non-Pharmacological Treatments in Chronic Pain Management: A Systematic Review and Quantitative Meta-Analysis. Int. J. Environ. Res. Public Health 2022, 19, 4071. [Google Scholar] [CrossRef]
  8. Grassini, S.; Laumann, K. Evaluating the use of virtual reality in work safety: A literature review. In Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment And Management Conference, Venice, Italy, 1–5 November 2022; pp. 1–6. [Google Scholar]
  9. Weech, S.; Kenny, S.; Barnett-Cowan, M. Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Front. Psychol. 2019, 10, 158. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Witmer, B.G.; Singer, M.J. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence Teleoperators Virtual Environ. 1998, 7, 225–240. [Google Scholar] [CrossRef]
  11. Heeter, C. Being There: The Subjective Experience of Presence. Presence Teleoperators Virtual Environ. 1992, 1, 262–271. [Google Scholar] [CrossRef]
  12. Ijsselsteijn, W.; Riva, G. Being there: The experience of presence in mediated environments. In Concepts, Effects and Measurements of User Presence in Synthetic Environments; IOS Press: Amsterdam, The Netherlands, 2003; pp. 3–16. [Google Scholar]
  13. Dużmańska, N.; Strojny, P.; Strojny, A. Can Simulator Sickness Be Avoided? A Review on Temporal Aspects of Simulator Sickness. Front. Psychol. 2018, 9, 2132. [Google Scholar] [CrossRef]
  14. Narciso, D.; Bessa, M.; Melo, M.; Coelho, A.; Vasconcelos-Raposo, J. Immersive 360° video user experience: Impact of different variables in the sense of presence and cybersickness. Univers. Access Inf. Soc. 2017, 18, 77–87. [Google Scholar] [CrossRef]
  15. Laviola, J.J. A discussion of cybersickness in virtual environments. ACM Sigchi Bull. 2000, 32, 47–56. [Google Scholar] [CrossRef]
  16. Grassini, S.; Laumann, K. Immersive visual technologies and human health. In Proceedings of the European Conference on Cognitive Ergonomics 2021, Siena, Italy, 26–29 April 2021; pp. 1–6. [Google Scholar]
  17. Davis, S.; Nesbitt, K.; Nalivaiko, E. A Systematic Review of Cybersickness. In Proceedings of the 2014 Conference on Interactive Entertainment, Newcastle, Australia, 2–3 December 2014. [Google Scholar]
  18. Stanney, K.M.; Kennedy, R.S.; Drexler, J.M. Cybersickness is Not Simulator Sickness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA, 1 October 1997; Volume 41, pp. 1138–1142. [Google Scholar]
  19. Almallah, M.; Hussain, Q.; Reinolsmann, N.; Alhajyaseen, W.K. Driving simulation sickness and the sense of presence: Correlation and contributing factors. Transp. Res. Part F Traffic Psychol. Behav. 2021, 78, 180–193. [Google Scholar] [CrossRef]
  20. Lucas, G.; Kemeny, A.; Paillot, D.; Colombet, F. A simulation sickness study on a driving simulator equipped with a vibration platform. Transp. Res. Part F Traffic Psychol. Behav. 2019, 68, 15–22. [Google Scholar] [CrossRef]
  21. Renganayagalu, S.K.; Mallam, S.C.; Nazir, S. Effectiveness of VR Head Mounted Displays in Professional Training: A Systematic Review. Technol. Knowl. Learn. 2021, 26, 999–1041. [Google Scholar] [CrossRef]
  22. Grassini, S.; Saghafian, M.; Thorp, S.; Laumann, K. User Individual Characteristics and Perceived Usability in Immersive HMD VR: A Mixed Method Explorative Study. In Proceedings of the International Conference on Applied Human Factors and Ergonomics, New York, NY, USA, 24–28 July 2021; pp. 157–165. [Google Scholar] [CrossRef]
  23. Witmer, B.G.; Singer, M.F. Measuring Presence in Virtual Environments; Defense Technical Information Center: Fort Belvoir, VA, USA, 1 October 1994. [Google Scholar]
  24. Usoh, M.; Arthur, K.; Whitton, M.C.; Bastos, R.; Steed, A.; Slater, M.; Brooks, F.P. Walking > walking-in-place > flying, in virtual environments. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques—SIGGRAPH, New York, NY, USA, 1 July 1999. [Google Scholar]
  25. Kennedy, R.S.; Lane, N.E.; Berbaum, K.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [Google Scholar] [CrossRef]
  26. Lin, J.J.W.; Duh, H.B.L.; Parker, D.E.; Abi-Rached, H.; Furness, T.A. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings of the IEEE Virtual Reality, Orlando, FL, USA, 24–28 March 2002. [Google Scholar]
  27. Mittelstaedt, J.; Wacker, J.; Stelling, D. Effects of display type and motion control on cybersickness in a virtual bike simulator. Displays 2018, 51, 43–50. [Google Scholar] [CrossRef]
  28. Seay, A.F.; Krum, D.M.; Hodges, L.; Ribarsky, W. Simulator Sickness and Presence in a High FOV Virtual Environment. In Proceedings of the IEEE Virtual Reality Conference 2001 (VR2001), Nagasaki, Japan, 13–17 March 2001; pp. 299–300. [Google Scholar] [CrossRef]
  29. Ling, Y.; Nefs, H.; Brinkman, W.-P.; Qu, C.; Heynderickx, I. The relationship between individual characteristics and experienced presence. Comput. Hum. Behav. 2013, 29, 1519–1530. [Google Scholar] [CrossRef]
  30. Yang, S.-N.; Schlieski, T.; Selmins, B.; Cooper, S.C.; Doherty, R.A.; Corriveau, P.J.; Sheedy, J.E. Stereoscopic Viewing and Reported Perceived Immersion and Symptoms. Optom. Vis. Sci. 2012, 89, 1068–1080. [Google Scholar] [CrossRef]
  31. Liu, C.-L.; Uang, S.-T. Effects of Presence on Causing Cybersickness in the Elderly within a 3D Virtual Store. In Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; Volume 6764, pp. 490–499. [Google Scholar] [CrossRef]
  32. Kim, Y.Y.; Kim, H.J.; Kim, E.N.; Ko, H.D. Characteristic changes in the physiological components of cybersickness. Psychophysiology 2005, 42, 616–625. [Google Scholar] [CrossRef]
  33. Grassini, S.; Laumann, K.; Topranin, V.D.M.; Thorp, S. Evaluating the effect of multi-sensory stimulations on simulator sickness and sense of presence during HMD-mediated VR experience. Ergonomics 2021, 64, 1532–1542. [Google Scholar] [CrossRef]
  34. Jovanović, V.; Lazić, M. Is Longer Always Better? A Comparison of the Validity of Single-item Versus Multiple-item Measures of Life Satisfaction. Appl. Res. Qual. Life 2018, 15, 675–692. [Google Scholar] [CrossRef]
  35. Grassini, S.; Laumann, K.; Thorp, S.; Topranin, V.D.M. Using electrophysiological measures to evaluate the sense of presence in immersive virtual environments: An event-related potential study. Brain Behav. 2021, 11, e2269. [Google Scholar] [CrossRef] [PubMed]
  36. Chalfoun, P.; Dankoff, J. Developing actionable biometric insights for production teams. In Oxford Scholarship Online; Oxford University Press: Oxford, UK, 2018; pp. 281–299. [Google Scholar] [CrossRef]
  37. Keshavarz, B.; Hecht, H. Visually induced motion sickness and presence in videogames: The role of sound. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2012, 56, 1763–1767. [Google Scholar] [CrossRef]
  38. Keshavarz, B.; Hecht, H. Validating an Efficient Method to Quantify Motion Sickness. Hum. Factors: J. Hum. Factors Ergon. Soc. 2011, 53, 415–426. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Cheung, F.; Lucas, R. Assessing the validity of single-item life satisfaction measures: Results from three large samples. Qual. Life Res. 2014, 23, 2809–2818. [Google Scholar] [CrossRef] [Green Version]
  40. Melo, M.; Sampaio, S.; Barbosa, L.; Vasconcelos-Raposo, J.; Bessa, M. The Impact of Different Exposure Times to 360° Video Experience on the Sense of Presence. In Proceedings of the 2016 23° Encontro Português de Computação Gráfica e Interação (EPCGI); Available online: https://massive.inesctec.pt/wp-content/cache/mendeley-file-cache/ede5337f-f8b3-35ed-927c-d95d3a9261c5.pdf (accessed on 30 November 2016).
  41. Huang, W.; Roscoe, R.D.; Johnson-Glenberg, M.C.; Craig, S.D. Motivation, engagement, and performance across multiple virtual reality sessions and levels of immersion. J. Comput. Assist. Learn. 2020, 37, 745–758. [Google Scholar] [CrossRef]
  42. Stanney, K.; Salvendy, G. Aftereffects and Sense of Presence in Virtual Environments: Formulation of a Research and Development Agenda. Int. J. Hum.-Comput. Interact. 1998, 10, 135–187. [Google Scholar] [CrossRef]
  43. Kennedy, R.S.; Stanney, K.M.; Dunlap, W.P. Duration and Exposure to Virtual Environments: Sickness Curves During and Across Sessions. Presence Teleoperators Virtual Environ. 2000, 9, 463–472. [Google Scholar] [CrossRef] [Green Version]
  44. Classen, S.; Owens, A. Simulator Sickness among Returning Combat Veterans with Mild Traumatic Brain Injury and/or Post-Traumatic Stress Disorder. Adv. Transp. Stud. Available online: https://www.researchgate.net/publication/288645986_Simulator_sickness_among_returning_combat_veterans_with_mild_traumatic_brain_injury_andor_post-traumatic_stress_disorder (accessed on 31 December 2010).
  45. McHugh, N.; Jung, S.; Hoermann, S.; Lindeman, R.W. Investigating a Physical Dial as a Measurement Tool for Cybersickness in Virtual Reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, Australia, 12–15 November 2019; p. 32. [Google Scholar] [CrossRef] [Green Version]
  46. Moss, J.; Scisco, J.; Muth, E. Simulator Sickness during Head Mounted Display (HMD) of Real World Video Captured Scenes. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, New York, NY, USA, 22–26 September 2008; Volume 52, pp. 1631–1634. [Google Scholar] [CrossRef]
  47. Sinitski, E.H.; Thompson, A.A.; Godsell, P.; Honey, J.; Besemann, M. Postural stability and simulator sickness after walking on a treadmill in a virtual environment with a curved display. Displays 2018, 52, 1–7. [Google Scholar] [CrossRef]
  48. Feenstra, P.; Bos, J.; van Gent, R. A visual display enhancing comfort by counteracting airsickness. Displays 2011, 32, 194–200. [Google Scholar] [CrossRef]
  49. Sevinç, V.; Berkman, M.I. Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Appl. Ergon. 2019, 82, 102958. [Google Scholar] [CrossRef] [PubMed]
  50. Hein, D.; Mai, C.; Hußmann, H. The Usage of Presence Measurements in Research: A Review. In Proceedings of the International Society for Presence Research Annual Conference (Presence); pp. 21–22. Available online: https://www.researchgate.net/publication/325793959_The_Usage_of_Presence_Measurements_in_Research_A_Review (accessed on 31 May 2018).
  51. Witmer, B.G.; Jerome, C.J.; Singer, M.J. The Factor Structure of the Presence Questionnaire. Presence Teleoperators Virtual Environ. 2005, 14, 298–312. [Google Scholar] [CrossRef]
  52. Van Baren, J.; IJsselsteijn, W. Measuring presence: A Guide to Current Measurement Approaches (OmniPres Project IST 2001-39237). Retrieved from Eindhoven University of Technology, IST FET OMNIPRES Project. Available online: https://www8.informatik.umu.se/~jwworth/presencemeasurement.pdf (accessed on 16 February 2022).
  53. Bouchard, S.; Robillard, G.; St-Jacques, J.; Dumoulin, S.; Patry, M.-J.; Renaud, P. Reliability and validity of a single-item measure of presence in VR. In Proceedings of the the 3rd IEEE International Workshop on Haptic, Audio and Visual Environments and Their Applications, Ottawa, ON, Canada, 2–3 October 2005. [Google Scholar] [CrossRef]
  54. Bouchard, S.; St-Jacques, J.; Robillard, G.; Renaud, P. Anxiety Increases the Feeling of Presence in Virtual Reality. Presence Teleoperators Virtual Environ. 2008, 17, 376–391. [Google Scholar] [CrossRef]
  55. Pernet, C.R.; Wilcox, R.R.; Rousselet, G.A. Robust Correlation Analyses: False Positive and Power Validation Using a New Open Source Matlab Toolbox. Front. Psychol. 2013, 3, 606. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Wilcox, R. Inferences Based on a Skipped Correlation Coefficient. J. Appl. Stat. 2004, 31, 131–143. [Google Scholar] [CrossRef]
  57. Munafo, J.; Diedrick, M.; Stoffregen, T.A. The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Exp. Brain Res. 2016, 235, 889–901. [Google Scholar] [CrossRef]
  58. Grassini, S.; Laumann, K.; Luzi, A. Association of Individual Factors with Simulator Sickness and Sense of Presence in Virtual Reality Mediated by Head-Mounted Displays (HMDs). Multimodal Technol. Interact. 2021, 5, 7. [Google Scholar] [CrossRef]
Figure 1. Example images from «Epic Roller Coaster» by B4T Games Scenario «T-Rex Kingdom»; reproduced with permission.
Figure 1. Example images from «Epic Roller Coaster» by B4T Games Scenario «T-Rex Kingdom»; reproduced with permission.
Mti 06 00031 g001
Figure 2. Mean scores for sense of presence and CS across seven time-intervals. The orange line represents MFMS scores, and the blue line represents MFMS scores. Black error bars represent standard deviations.
Figure 2. Mean scores for sense of presence and CS across seven time-intervals. The orange line represents MFMS scores, and the blue line represents MFMS scores. Black error bars represent standard deviations.
Mti 06 00031 g002
Figure 3. Skipped correlations between MFMS and BMP for each measure in time. Skipped Pearson’s correlations for each measure of MFMS and BMP. The top of the figure shows Pearson’s r coefficients and 95% confidence intervals (shown as the transparent red area in the figure). The numbering above each skipped correlation indicates the time interval. The x-axis represents the score for sense of presence (BMP; 1–10), and the y-axis shows the score for CS (MFMS; 1–10). Dots show each data point, where red dots represent outliers that were eliminated before the correlation analyses were computed. (ag) represent the chronological order in which the questionnaire answers were collected during and after the VR experience.
Figure 3. Skipped correlations between MFMS and BMP for each measure in time. Skipped Pearson’s correlations for each measure of MFMS and BMP. The top of the figure shows Pearson’s r coefficients and 95% confidence intervals (shown as the transparent red area in the figure). The numbering above each skipped correlation indicates the time interval. The x-axis represents the score for sense of presence (BMP; 1–10), and the y-axis shows the score for CS (MFMS; 1–10). Dots show each data point, where red dots represent outliers that were eliminated before the correlation analyses were computed. (ag) represent the chronological order in which the questionnaire answers were collected during and after the VR experience.
Mti 06 00031 g003
Figure 4. Skipped correlations between SSQ and the PQ. The x-axis represents PQ scores, and the y-axis shows SSQ total scores (final score minus score before the start of the experiment). The top of the figure shows Pearson’s r coefficients and 95% confidence intervals (shown as the transparent red area in the figure). Dots show each data point. The red line represents the correlation coefficient, and the blue line represents the distances from the orthogonal center of the data where data points outside are excluded.
Figure 4. Skipped correlations between SSQ and the PQ. The x-axis represents PQ scores, and the y-axis shows SSQ total scores (final score minus score before the start of the experiment). The top of the figure shows Pearson’s r coefficients and 95% confidence intervals (shown as the transparent red area in the figure). Dots show each data point. The red line represents the correlation coefficient, and the blue line represents the distances from the orthogonal center of the data where data points outside are excluded.
Mti 06 00031 g004
Figure 5. Cross-correlations between MFMS and time-shifted values of BMP. Horizontal blue lines display the upper and lower confidence bounds. The X-axis shows time-shifted values of sense of presence on the effect of CS. Y-axis indicates sample cross-correlation values.
Figure 5. Cross-correlations between MFMS and time-shifted values of BMP. Horizontal blue lines display the upper and lower confidence bounds. The X-axis shows time-shifted values of sense of presence on the effect of CS. Y-axis indicates sample cross-correlation values.
Mti 06 00031 g005
Table 1. Descriptive statistics for cybersickness and sense of presence reported across seven time intervals during the VE exposure.
Table 1. Descriptive statistics for cybersickness and sense of presence reported across seven time intervals during the VE exposure.
MFMSBMP
Time Interval
(n, Chronological Order)
MSDMSD
1 (pre-experiment)1.330.832.902.15
22.261.415.402.05
33.451.746.082.06
44.131.946.451.84
53.832.166.181.99
63.131.965.581.88
7 (post-experiment)3.952.216.031.98
Reported scores across the 7 time-intervals for Fast Motion Sickness Questionnaire (MFMS) and Brief Measure of Presence (BMP).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Thorp, S.; Sævild Ree, A.; Grassini, S. Temporal Development of Sense of Presence and Cybersickness during an Immersive VR Experience. Multimodal Technol. Interact. 2022, 6, 31. https://0-doi-org.brum.beds.ac.uk/10.3390/mti6050031

AMA Style

Thorp S, Sævild Ree A, Grassini S. Temporal Development of Sense of Presence and Cybersickness during an Immersive VR Experience. Multimodal Technologies and Interaction. 2022; 6(5):31. https://0-doi-org.brum.beds.ac.uk/10.3390/mti6050031

Chicago/Turabian Style

Thorp, Sebastian, Alexander Sævild Ree, and Simone Grassini. 2022. "Temporal Development of Sense of Presence and Cybersickness during an Immersive VR Experience" Multimodal Technologies and Interaction 6, no. 5: 31. https://0-doi-org.brum.beds.ac.uk/10.3390/mti6050031

Article Metrics

Back to TopTop