4.1. Students’ Simulation Performance
Overall, the SRNAs’ simulation performance scores in the areas of technical and non-technical skill performance had means of 80% to 97%. The high overall mean scores reflect the effective management of the simulated airway fire by the SRNAs. In addition, the study provides evidence to support the use of mixed reality as an effective, added layer for simulation-based education. An OR fire with visual effects such as smoke, flames, and pouring water over high-fidelity mannequins cannot be effectively simulated unless adding technical elements such as augmented reality headsets. The SRNAs’ experiences and anecdotal comments effectively captured the usefulness of this technology. The SRNAs, with such limited clinical experiences, were able to effectively demonstrate high proficiency in the management of the OR fire using both technical and non-technical skills in this mixed reality environment.
SRNAs obtained the highest scores in technical skills on fire prevention (5.81/6 = 97%) and the highest scores for non-technical skills in teamwork (3.80/4 = 95%). The lowest scores for technical skills were in fire management with a mean 3.2/80% and for non-technical skills were decision-making with a mean 8.94/ 89%. On the checklist, there were four questions that the raters scored differently for the same student. The questions were TM: check for early signs of fire; SA: recognition of early signs of fire; DM/TM: immediately stop the flow of all gases; and SA/DM/TM: examine endotracheal tube (ETT) for missing fragments in the airway (consider bronchoscopy). This difference can be attributed to the difference in confederates giving clues to the SRNAs, SRNAs’ incomplete or assumed actions, or simply raters’ variability. Inconsistency of the scenario interpretation such as when gases were considered stopped at the time of ETT removal vs. turning off inhaled gases prior to removal of the ETT was scored differently by the raters despite scoring the pilot simulation greater than 80% among the raters’ agreement. Another point in the TM that showed inconsistency among the raters was “check for early signs of airway fire”. When using augmented reality headsets, some areas of the periphery vision of the SRNAs were lost and not captured by the video recordings unless the student was directly looking at the source of smoke. In a real-life situation, sensory signs such as smell or hearing of a spark from the electrocautery were lost in the simulated OR fire. Even though the augmented reality headset displayed details such as smoke and fire, the addition of sensory stimulation such as sound and smell should be considered in the future when assessing early recognition of fire.
Another question—TM: Check for early signs of fire—may need to be revised in a checklist during simulation scenarios as such a question is difficult to define with respect to what constitutes “early sign of fire” in such cases. This interpretation may be subject to difficulties even in a real-life scenario when following the OR fire algorithm. The fourth question that raised inconsistent ratings of SRNAs’ performance was SA/DM/TM: examine ETT for missing fragments in the airway (consider bronchoscopy). Authors agree that when assessing this skill in future simulations, it may be beneficial to discolor or disfigure the tip of the ETT to simulate a burned tip, thereby enhancing the fidelity and prompting further examination of the airway by the SRNAs. Despite this disagreement in the skill rating, the majority of the SRNAs acted appropriately and used a video laryngoscope to re-establish the airway, which is consistent with past findings that early introduction of SRNAs to OR fire checklists for prevention and treatment is effective in simulation training necessary for effective OR fire treatment [20
Previous research supports the necessity and use of a virtual environment platform for fire in OR training of health professionals [24
]. The addition of a mixed reality platform in this study provided an additional layer of realism to the simulation scenario. The smoke and fire were highly realistic. However, auditory and olfactory stimuli would have increased the fidelity and added to the situational awareness of the overall experience. Additional stimuli such as the crackling of fire or the smell of smoke may have provided benefit to the student’s response time and ability to manage the situation at hand. These enhancements may be used in future OR fire simulations.
Over the past twenty-five years, hospitals and universities have invested millions of dollars into facilities, equipment, and personnel for the development of simulation-based education centers for the healthcare professions’ students and clinicians. With the growth of virtual, augmented, or mixed reality simulations, centers are now seeing a push to adopt these new technologies. The cost to outfit one learner and three to four role player participants for a mixed reality simulation scenario may incur $12,500 for augmented reality headsets, such as the Magic Leap One™, plus the development or purchase cost of the scenario software. By comparison, when this same scenario is run using theatrical fog as a high-fidelity immersive simulation, the cost of the fog machine and set-up is approximately $75. However, due to the time required to turn over the fog simulation, including exhausting the accumulated fog from the room, learners have to rotate through the experience in small groups to complete the training session in the scheduled block of time. With the technology-based, mixed reality simulation, the turnover time was under five-minutes and learners were able to be scheduled for an individual experience in the scheduled block of time. Beyond this specific use, the headsets are available for other training simulations, whereas the fog machine has a limited application. Future adoption of technology-based simulations should focus on the best attainment of learner outcomes and not solely on hardware and software costs.