Supporting User Onboarding in Automated Vehicles through Multimodal Augmented Reality Tutorials

Abstract

Misconceptions of vehicle automation functionalities lead to either non-use or dangerous misuse of assistant systems, harming the users’ experience by reducing potential comfort or compromise safety. Thus, users must understand how and when to use an assistant system. In a preliminary online survey, we examined the use, trust, and the perceived understanding of modern vehicle assistant systems. Despite remaining incomprehensibility (36–64%), experienced misunderstandings (up to 9%), and the need for training (around 30%), users reported high trust in the systems. In the following study with first-time users, we examine the effect of different User Onboarding approaches for an automated parking assistant system in a Tesla and compare the traditional text-based manual with a multimodal augmented reality (AR) smartphone application in means of user acceptance, UX, trust, understanding, and task performance. While the User Onboarding experience for both approaches shows high pragmatic quality, the hedonic quality was perceived significantly higher in AR. For the automated parking process, reported hedonic and pragmatic user experience, trust, automation understanding, and acceptance do not differ, yet the observed task performance was higher in the AR condition. Overall, AR might help motivate proper User Onboarding and better communicate how to operate the system for inexperienced users.

1. Introduction

Modern vehicles integrate more and more assistant systems. Today, systems like adaptive cruise control, lane-keeping assistants, and automated parking assistants are standard equipment of middle- to upper-class vehicles. However, not everyone who uses these systems has adequate knowledge about these systems’ potentials and boundaries. One reason for this is that modern assistant systems are not part of the education in driving school, and vehicle designers miss to educate their users, too. A misconception of such systems leads to (1) a non-use in adequate driving situations and a potential lack of comfort for the user, and (2) a use in an inadequate driving situation and potentially compromising safety for the user. Additionally, as trust in automation is partly formed through experience [1], the use in inadequate situations leads to (1) overtrust, if no misbehavior of the car is experienced (silent error), or (2) undertrust if misbehavior is experienced. Overtrusting behavior might lead to dangerous system misuse, whereas undertrusting behavior might lead to system non-use. Thus, users should understand the real capabilities of vehicle automation from the beginning. Traditional User Onboarding relies on text-based manuals, which are, in the case of dynamic software updates that change the vehicle functionality, already outdated by the time the user reads it. In this paper, we investigate the possibility to educate users about automated driving functions with multimodal augmented reality (abbr.: AR) tutorials on a smartphone. AR approach can improve the User Onboarding because it creates a richer user experience (abbr.: UX) and through a combination of text, video, and audio elements and can communicate information locally, e.g., by overlaying the windshield with exemplary driving situations or by highlighting interface elements in the car interior. To identify misconceptions of modern vehicle users and their assistant systems, we first conducted an online survey to understand where users of modern vehicle assistant systems have experienced misunderstandings and select potential candidates to improve the User Onboarding process. While most assistant systems would make suitable candidates because around 30% of the users report the need for training despite high trust in the systems, we focused on automated parking. In a between-subject real-world study in a Tesla with first-time users, we tested the User Onboarding approaches for the “autopark”-assistant with either the AR app or the more established owners manual. While the User Onboarding experience for both approaches shows high pragmatic quality, the hedonic quality was perceived significantly higher in AR. For the automated parking process, reported hedonic and pragmatic user experience, trust, automation understanding, and acceptance do not differ, yet the observed task performance was higher in the AR condition.

Contribution Statement

With our contribution, we extend current knowledge on educating first-time users by providing (1) insights into vehicle users’ current understandings of vehicle assistant systems and where they see a need for training and (2) a real-world evaluation of a multimodal AR User Onboarding process. Our findings will help vehicle designers with the integration and establishment of new ways for User Onboarding processes in future automated vehicles. In particular, AR might increase intrinsic motivation to use a system tutorial and better communicate how to operate the system for inexperienced users.

2. Background & Related Work

In this section, we first define User Onboarding in the context of automated vehicles. Then, we show that educating users from the start onwards can positively affect their experiences with the system. Lastly, we show how multimodal technologies like AR can contribute to User Onboarding.

2.1. Defining User Onboarding in Context of Automated Vehicles

The term Onboarding originally comes from the field of human resources management. If a new employee starts working for a company, s/he must first become acquainted with knowledge, skills, and behavior to succeed in the company [2]. Crumlish and Malone describe the Onboarding process as an entry point into a product that helps the user to overcome initial problems and hurdles [3]. The concept of Onboarding was also established in software development, mainly used in the context of user experience design, where it is also called User Onboarding. The goal is to motivate a new user to return to a website or system and become a future regular user [4]. Renz et al. [5] divide User Onboarding into three phases over time: (1) Onboarding, (2) help and support, and (3) re-entry. In the first phase, the goal is to support the user to become an effective user of the system. Furthermore, the user is supported and motivated to use the system in the second phase. The last phase intends to motivate the user to re-enter the system if s/he has not used it for a more extended period. Our paper addresses the first phase, the initial contact with a system, the Onboarding.

There are two critical components for a successful Onboarding process that lead to desired “quick wins” and “aha experiences” [6]: (1) users need to recognize the system’s value quickly [6], and (2) training users how to use the application to achieve their goals faster [7]. In our case, vehicle users need to recognize the additional comfort they get through automation and learn how and when to use an assistant system for that purpose. The marketing is excellent at selling the latest assistant systems’ comfort and value, in some cases even exaggerating the systems real capabilities–so-called “autonowashing” [8]. However, the training aspect is often neglected yet essential in the context of automated vehicles, and, thus, one should apply User Onboarding processes to automated vehicles.

2.2. Effects of User Onboarding on Experience with Automated Vehicles

In the context of automated vehicles, having adequate initial knowledge about the system can have beneficial effects on user acceptance, UX, trust, and task performance.

First, bad use experiences due to inadequate system knowledge might lead to the rejectance of the system. Missing the system’s potential lets users interact inefficiently, e.g., when the user does not know how to activate the automated cruise control (abbr.: ACC) assistant and thus misses a comfortable automated vehicle (abbr.: AV) feature. Missing knowledge about the system’s boundaries lets the user interact risky, e.g., s/he might activate the ACC in a non-safe situation or hesitate to retake the control. A robust mental model helps to detect such situations faster [9,10]. With frequent practical experience, drivers might get more and more aware of the system’s boundaries [11], but only if they experience these boundaries [12], and perceive it as an inappropriate behavior – otherwise they also might get used to automation failure [13].

Second, the degree of a user’s knowledge about the system is linked to automation trust. The widely used automation trust model of Hoff and Bashir [1] consists of three aspects: (1) dispositional trust based on personality and socio-cultural aspects, (2) situational trust based on the interaction context, and (3) learned trust based on (3a) preknowledge and (3b) experience with a system over time. Concerning (3a), preknowledge, ease of learning, and self-rated knowledge increase perceived trust [14]. User Onboarding might help to calibrate trust to an adequate level [15]. E.g., Tenhundfeld et al. found that a better understanding of the Tesla “Autopark”-assistant leads to higher trust ratings [16]. Further, providing transparent information about the system, the base to build a precise mental model, can prevent a decrease in trust [17,18]. Concerning (3b), experience with a system over time, Dikmen and Burns [14] found in a survey with Tesla drivers regarding the assistants “Autopilot” and “Summon” that trust in these systems increase over time – regardless of the actual experience. Kraus et al. [17] also show that trust decreases after experiencing system failures but is reestablished after a period of error-free interaction. Additionally, trust also predicts driving performance: Körber et al. [19] explicitly altered the information to promote or lower reliance on the system and found that with higher reliance on the system, the engagement in non-driving-related activities increases while takeover and driving performance decrease.

Multimodal User Onboarding in Augmented Reality

A popular theory of learning, the Cognitive Theory of Multimedia Learning (abbr.: CTML) [20], proposes that the human working memory contains independent auditory and visual channels for the short-term storage of information, each with a limited capacity. Further, meaningful learning occurs if the learner integrates information from both channels into an (existing) mental model. Relying on redundant channels leads to better memorization [21]. However, after purchasing a car, User Onboarding in the automotive domain is traditionally done through the text-based user manual that only uses the auditory channel. Multimodal approaches like AR allow the combination of the visual and auditory channels and, thus, improve the learning process. Through the coupling of virtual information with real-world objects, AR elements can communicate information fast, at the right time, and at the right location [22,23]. Processes can be experienced and thus better understood through AR [24]. Overall, learning in AR environments, e.g., for assembly tasks [25], is slower than manual-based learning but can lead to better task performance.

The increase in users’ performance through a richer learning experience in system tutorials has been found in the context of vehicle assistant systems: Ebnali et al. [26] compared different levels of interaction fidelity (video, low-fidelity virtual reality, and high-fidelity virtual reality) in a driving simulation over time. They found that tutorial interaction fidelity positively affects task performance and trust (cf. previous section). Similar results were found for simulator/video tutorials [27], VR and AR training [28], and interactive education processes, e.g., via quiz [29,30]. These findings underpin the importance of multimodal User Onboarding: The richer the pre-use tutorial’s interaction experience, the better the user’s performance later.

2.3. Conclusive Summary

Studies show that user education before the first time of using automation features helps form better mental models and therefore helps build initial trust and improve driving task performance, increasing the users’ experience. While using a rich multimedia environment would be possible, today, most User Onboarding processes in cars are based on written manuals. With multimodal AR tutorials, users can potentially integrate information from auditory and visual channels, locate information at the right time and the right place, and thus, better experience/predict the future use of an assistant system. In the next chapter, we compare current assistant systems and identify candidates for applying a multimodal AR User Onboarding process.

3. Online Survey on Users’ Experience with Current Vehicle Assistant Systems

To gain an understanding of users’ misconceptions regarding modern vehicle assistant systems, we conducted an online study. Therefore, we selected six current automated driving assistant systems (abbr.: ADAS) and asked participants if they used them, and if so, how they did get on with them.

3.1. Sample

The sample consisted of 58 participants ($m=54,f=3,NR=1$) with an average age of 41.11 years ($SD=12.42$). We recruited over Facebook special interest groups like “W213 S213 Mercedes Benz E-Klasse Deutschland” where we expected members to have access to a car with modern assistant systems. Most respondents (81%) personally owned a vehicle equipped with at least one of the ADAS of interest. Their cars were built mostly in 2017 ($SD=1.9$ years). The technical affinity of participants, measured with the ATI-S scale (four items on a 6-point Likert scale) [31], was relatively high ($M=4.53,SD=0.95$).

3.2. Method and Procedure

The questionnaire was accessed at home, provided on the soscisurvey.de (accessed on 19 April 2021) platform. First, participants were briefed about the study’s aim and purpose and informed about the data usage according to the EU General Data Protection Regulation [32]. Then, after giving their informed consent, they answered questionnaires regarding their experience with diverse automation technologies. We chose six modern driver assistant systems for the survey: (1) adaptive cruise control (ACC), (2) active lane keeping assistant (abbr.: aLKA), (3) active lane change assistant (abbr.: aLCA), (4) traffic jam assistant (combination of ACC and aLKA), (5) remote parking, and (6) automated parking. The first two were asked in one question because we were interested in the combination of both systems (level 2 automation). For each assistant system, we asked questions covering the following aspects: frequency of assistant use (single item on a 4-point scale: weekly, monthly, even less frequently, not at all), trust at first contact “How much trust did you have in the first use of...?” (single item on a 5-point Likert scale from “no trust” to “very high trust”), trust today “How much trust do you have today in the mentioned system above?” (single item on a 5-point Likert scale from “no trust” to “very high trust”), the need to receive training for the system “Would you have liked to receive training for the driver assistance system?” (single item with binary yes/no), incomprehensibility (multiple choice with three items for operation, system understanding, and system limits), experienced misunderstandings (free text answer), and hidden functions that were noticed only after using a system already for a while (free text answer). We also asked for the general trust in automated vehicles (single item on a 5-point Likert scale from “no trust” to “very high trust”) and the behavior in obtaining information about new functions in the vehicle. At last, participants were debriefed about the study aims and we thanked them for their participation.

3.3. Results & Discussion

Table 1. Results from the preliminary survey, ${}^a$ significantly higher than initial trust (p < 0.05), checked with dependent t-test (homogeneity of variances checked with Levene-Test).

	ACC ∪ aLKA	aLCA	Traffic Jam Assistant	Remote Parking	Automated Parking
Persons who own it $n(\%)$	44 (75)	17 (29)	27 (47)	11 (19)	45 (78)
Owners who use it...
...weekly	37 (84)	9 (53)	17 (63)	2 (18)	16 (36)
...monthly	2 (4)	3 (18)	4 (15)	1 (9)	5 (11)
...<monthly	3 (6)	5 (29)	3 (11)	5 (46)	15 (33)
...never	2 (4)		3 (11)	3 (27)	9 (20)
Trust at first contact $M\left(SD\right)$	3.5 (1.17)	3.47 (1.17)	3.26 (1.25)	3.64 (1.2)	2.96 (1.6)
Trust today $M\left(SD\right)$	4.23 ${}^a$ (0.83)	4.41 ${}^a$ (0.71)	4 ${}^a$ (1.14)	4.36 ${}^a$ (1.02)	4.16 ${}^a$ (1.1)
Training Needs	29%	29%	25%	36%	28%
Incomprehensibility	59%	52%	51%	36%	64%
Operation	9%	5%	25%	9%	31%
System Understanding	20%	17%	3%	9%	11%
System Boundaries	46%	47%	33%	36%	33%
Experienced Misunderstandings	7%	0%	8%	0%	7%
Hidden Functions	0%	0%	0%	0%	0%

summarizes the results from the survey: Participants state a high level of today’s trust for all assistant systems and rated it significantly higher than their initial trust. Given the increment of trust over time, we would have expected that most participants understand how to use a particular assistant system. However, in contrast, they report a high level of training needs and incomprehensibility for most assistants. This mismatch between the high level of trust and a low level of expertise can lead to misunderstandings that affect the whole user experience with a car or even lead to dangerous situations. The reported misunderstandings with assistant systems are unexpected error messages (“spontaneous error messages, not trackable”), unexpected non-possibility to make use of the assistant in some situations (user tried to use a traffic assistant at a construction site which is not permitted / use of ACC under high solar ray), or unexpected behavior of the assistant system (Autopark: Users found the fast steering and moving during the autopark situation irritating and they did not expect the car to completely stop as they intervened; “Had to intervene in the automatic parking process. The car could not pass a pillar. Parking too hard and too fast.”). None of the participants reported that they discovered a “new” function of an assistant, indicating either that all functions were evident from the start, or that they did not further educate themselves about the system use. However, this is in contrast to the reported training needs and incomprehensibility. Therefore, it is essential to educate users of automated vehicle functions from the first moment of use.

Whereas all assistant systems would make suitable candidates for inspecting and improving the User Onboarding processes given the training needs (around 30%), we decided to focus on the automated parking assistant in this paper for multiple reasons. First, because participants reported the least initial trust in this assistant system and trust could be improved. Second, related to the first point, because automated parking is not yet a widely implemented assistant system in middle-class cars, the chance to recruit first-time users would be higher. Third, participants mentioned the most incomprehensibility with automated parking, e.g., 30% of the users reported that they do not fully understand the vehicle’s behavior, so that User Onboarding would have the highest effect here.

4. Real-World Driving Study to Compare User Onboarding Mechanisms

To answer how multimodal AR User Onboarding impacts the users’ experience with the automated parking assistant, we aimed at extending the insights on trust and task performance by previous simulator-based research through an experiment with high ecological validity (cf. [33]) and aimed at testing the User Onboarding process in the real world. We, therefore, conducted a real-world study in a Tesla S 60 with its “autopark”-assistant and created a smartphone application according to Tesla design guidelines that presented the autopark process in a multimodal AR environment. In a between-subjects experiment with first-time users, 26 participants used either the multimodal AR app (AR group) or the text-based manufacturer manual (Manual group) to familiarize themselves with the autopark assistant. We did not compared the conditions to a non-onboarding group, since previous works have shown the beneficial effects of system tutorials, and we focus on the comparison of a new form of presenation to the traditional form.

4.1. Stimulus Material: AR User Onboarding Prototype vs. Manual for Autoparking

The autopark process requires the users to interact in the following procedure:

1.

Slowly pass through the parking space until a “P” appears in the instrument cluster

2.

Stop the vehicle

3.

Engage reverse gear

4.

On center screen, press button “Start”

Figure 1 shows step 1 and step 4 of this procedure.

This kind of spatially distributed interaction procedure might be difficult to understand for first-time users. To investigate the influence of our AR User Onboarding application on the actual user performance and experience, we compare it with the manual provided by the manufacturer. Both stimuli contain the same information about the operation of the autopark system. While the manual (see supplementary material at osf.io, link at the end of the article) contains a textual description (2 pages) of the autopark procedure, the AR app stimulates the user to consume the procedural information at the later location, i.e., it augments the instrument cluster display with a video overlay that shows the future state during the autopark process (“P”-notification), highlights the reverse gear, and augments the center display with the future state, too (“Start”-button). The app allowed to reverse steps and did not contain spoken instructions, but auditory feedback was used for interface elements, e.g., after completing a step or pressing a button. Figure 2 gives an impression of the tutorial in the AR app.

4.2. Experimental Procedure

The study consists of a preliminary survey, a driving test, and a follow-up survey. We explain each part of the study in more detail below.

4.2.1. Pre-Questionnaire

In preparation for the study, the test persons filled out an online questionnaire at home provided on the SoSciSurvey platform. The access link was distributed to the respondents via email two days before the experiment. The questionnaire began with an explanation of the experiment’s aims and procedures and an assurance that personal data will be processed anonymously and exclusively for scientific purposes. The test persons could give their consent for the processing of the questionnaire data and the recording of audio data later in the test drive. Furthermore, socio-demographic data (age, gender), technology affinity (ATI-S Scale [31]), driving experience (km/year), and experience with driver assistance systems (selection from 6 common systems) were recorded. The preliminary survey took about 20 min.

4.2.2. Driving Test

The driving test took place in a parking lot of the University (blinded for review). The test drive was done in a Tesla Model S60 from 2017 with an autopark assistant. The participants sat in the driver’s seat, the test leader was seated in the passenger seat, and another transcriber in the back seat. The setup of the test track consisted of three stations (cf. Figure 3 and Figure 4): (1) an introductory part, in which the test person receives the briefing and goes through the Onboarding process, (2) a driving training, in which the test person interacts with all relevant vehicle controls (accelerator, brake, gear lever), and (3) a parking space into which the system automatically parks. The test procedure at the three stations is described in more detail in the following.

Station I

In the first phase of the test, the test persons are received by the test director and asked to take a seat in the vehicle. The second researcher also briefly introduced himself there. After a short explanation of the experiment with the exact procedure of the stations, the test persons were informed about the insurance situation, and again their agreement to the audio recording of the experiment was obtained. The test persons were asked to express their thoughts aloud during the entire duration of the experiment (Think Aloud [34]). Afterward, the car’s automatic parking system was presented: Either by reading the manual or through the tutorial of the AR app. Participants were instructed to take their time with the tutorial and inform us when they feel ready to start the ride. The first station usually lasted 15 minutes.

Station II

After the autopark tutorial, the participants drove a course (see Figure 3). To familiarize the participants with accelerator and brake pedals and the acceleration behavior of the car, the car was started and stopped on the cordoned-off parking lot on an acceleration distance of about 30m. To familiarize the participants with the automatic transmission and reverse gear, the car was then parked forward and turned around again. Then the participants drove towards the parking space. The second station usually lasted 3 minutes.

Station III

At the last station, participants should start automated parking in a space between two artificial vehicle-look-alike limitations made of lightly filled moving boxes and real-sized prints of vehicles fronts on foam. The setup allowed that Tesla detected the parking space and reduced the risk of potential damages at the own and adjacent vehicles. The autopark interaction procedure that participants learned in Station I had to be applied. The correct sequence of action was as follows (cf. Section 4.1): At the parking space at a distance of about 1 m at max. 16 km/h and driving past one car length, a “P”-symbol appears on the instrument cluster; Engage reverse gear, as a result of which a new screen appears on the display of the center console, on which the autoparking process can be started; Start the autoparking process by pressing the “Start”-button on the center console. The car then started the automatic parking process, and the rearview camera on the center console was activated. The participants could interrupt the process by applying the brake once or moving the steering wheel, and end it by applying the brake twice. We used the number of interruptions as an indicator of mistrust. We intervened if participants got stuck and did not know how to proceed so that they could complete the experiment. We used the stuck and error-free interaction during the autopark procedure as an indicator of task performance. The third station usually lasted 2 min.

4.2.3. Post-Questionnaire

After completion of the third station, a follow-up survey of the participants was conducted. For this purpose, the test persons changed into a parked vehicle and fill out an online questionnaire on a tablet. Like the preliminary survey, the questionnaire was created using the SoSciSurvey platform and provided via a tablet. It covered the pragmatic and hedonic aspects of the user experience (UEQ-S Scale [35], eight items on a 7-point semantic differential scale) with Onboarding process and the autopark system, user acceptance of the system (TAM Scale [36], 14 items on a 7-point Likert scale), and trust in the parking system (Trust Scale [37], 11 items on a 7-point Likert scale), as well as a self-created quiz about the autopark system access the understanding of the system (6 questions with multiple choice). The quiz contained questions about the used sensors, the required supervision tasks, when not to use the system, distance to the parking space, required size of the parking space, and recommended speed. The follow-up questioning took about 20 min.

4.3. Sample

The study involved 26 participants ($m=23;f=3$), most of whom were students of the University. They were recruited through an university’s online forums. As a precondition, they should not have experienced autoparking before. The average age was 21.77 years ($SD=3.75$). The participants had a valid driver’s license, moderate driving experience (MDN = 5000–10,000 km/year), were somewhat familiar with driver assistance systems ($MDN=2$ out of 6), and a high affinity for technology ($M=4.53,SD=0.73$). These parameters did not significantly differ between experimental groups. In the order of their appointments, participants were alternately assigned to one of the two test conditions. Participants did not receive financial compensation.

4.4. Results of the Driving Study

We analyzed differences between experimental conditions regarding the self-reported questionnaires and observed behavior and uttered thoughts of participants. We used independent t-test (homogenity of variances checked with Levene’s test, $p>0.05$) to determine if the differences were statistically significant.

4.4.1. Observed Behavior

During the automated parking, we kept record of (1) the interruptions of the system behavior through pressing the break of the participants as an indicator of mistrust, and (2) their task performance, observed through the ability to start and complete the autopark process without help.

Mistrust: Interruption of the Automated System Behavior

In both groups, a few users (AR: $n=2$, manual: $n=3$) interrupted the autopark assistant due to skepticism if the system would handle the situation error-free (“It’s a little... One doesn’t quite trust that.”, “I hit the brake when I saw that it [the UI] was quite red”).

Task Performance: Completing the Autoparking Procedure without further Help

The AR group’s ability to perform the autopark process correctly had a comparably higher success rate of 54% ($n=7$) than the manual group 23% ($n=3$). Noticeably, in the manual group, test persons were often not sure, for example, what the instrument cluster is and on which screen or in which area of a screen interaction is necessary (“I think a key should be activated here”). It was particularly noticeable that many test persons tried to operate the parking symbol in the instrument cluster.

4.4.2. Subjective Questionnaires

Table 2. Results from the Questionnaires, * significant difference between groups ($p<0.05$).

		Experimental Condition
		Manual			AR
Scale	Factor	M	SD		M	SD	t-Test
Station I (Onboarding)
UEQ-S	Pragmatic Quality	5.26	0.97		5.24	1.4	$t\left(24\right)=-0.061,p=0.95$
	Hedonic Quality *	3.85	1.27		5.7	1.08	$t\left(24\right)=3.96,p=0.001$
	Overall *	4.5	1.05		5.47	1.05	$t\left(24\right)=2.196,p=0.038$
Station III (Autopark)
UEQ-S	Pragmatic Quality	5.52	0.87		5.87	0.85	$t\left(24\right)=0.061,p>0.05$
	Hedonic Quality	5.9	0.89		6	1.03	$t\left(24\right)=0.2,p>0.05$
	Overall	5.71	0.74		5.94	0.69	$t\left(24\right)=0.814,p>0.05$
Trust Scale	Trust	4.7	1.3		5.44	0.83	$t\left(24\right)=1.6,p=0.1$
	Mistrust	2.84	1.09		2.6	1.26	$t\left(24\right)=-0.463,p=0.64$
TAM	Perceived Usefulness	5.46	1.55		5.35	1.5	$t\left(24\right)=-0.179,p=0.85$
	Perceived Ease of Use	5.89	0.98		6.02	1.01	$t\left(24\right)=0.327,p=0.74$
	Attitude Toward Using	5.86	1.07		6.01	1.21	$t\left(24\right)=0.342,p=0.73$
	Behavorial Intention to Use	3.38	1.1		3.65	1.02	$t\left(24\right)=0.644,p=0.52$
Quiz	Overall (max 6 points)	3.69	1.31		3.61	0.86	$t\left(24\right)=0.176,p=0.86$

shows the results from the subjective questionnaires.

The user experience for the Onboarding process at Station I was significantly different between conditions. The manual condition leads to somewhat high UX ratings, while the AR leads to high UX ratings.

At Station III, during the automated parking process, the conditions do not differ significantly. Participants rate their user experience very high, show high trust, and somewhat low to neutral intention to use the system.

5. General Discussion

In the following, we discuss the driving study’s findings, compare it to previous work and the online survey, show limitations, and point out directions for future work.

5.1. User Acceptance and UX of Vehicle Assistant Systems

Both User Onboarding approaches, text-based manual, and multimodal AR app lead to high perceived usefulness, ease of use, and positive attitude towards using the vehicle assistant system. However, the intention to use the system is moderate in both conditions indicating that further factors influenced the acceptance of autopark. Here, the TAM questionnaire should be replaced with an acceptance questionnaire specific to automotive contexts. During User Onboarding, AR leads to significantly higher user experience ratings in the UEQ-S questionnaire. While the pragmatic quality is comparable, especially the tutorial’s hedonic quality improves through AR, indicating that users would more likely enjoy a multimodal AR User Onboarding process. Hedonic system experience is vital because it could motivate users intrinsically to use system tutorials more often, supporting the “re-entry” goal of User Onboarding [5]. In a longitudinal study, this could be further investigated.

5.2. Trust in and Familiarization with Vehicle Assistant Systems

The online survey showed that users trust for vehicle assistant systems increased compared to the initial level, supporting the learned trust facet in the model of Hoff and Bashir [1]. However, the simultaneously reported incomprehensibility, experienced misunderstandings, and the need for training indicate a not fully adequate mental model of the used vehicle assistant systems. This mismatch can be explained with the studies of Dikmen and Burns [13] (“Autopilot” trust increases over time—regardless of the actual experience), Beggiato et al. [12] (system failures have to be experienced), and Kraus et al. [17] (trust decreases after experiencing system failures but is reestablished after a period of error-free interaction).

During automated parking, both AR and manual conditions did not result in statistically significant trust ratings in the autopark assistant, yet there is a trend towards higher ratings in the AR group. Nevertheless, when investigating “appropriate trust” in automated systems, it is hard to define to what specific level the participants should calibrate their trust. In our case, participants showed high but not perfect trust and intervened in the autopark-process if deemed necessary. Future work could investigate how this light skepticism can be maintained to prevent overreliance on the system.

We tested the first time contact with the system. From the literature (e.g., [14]) and our online survey, we know that over time trust and performance measures might change, and more experienced users might have other requirements than first-time users. User Onboarding should be refreshed after a while but also be adapted to the expertise of users. As stated before, the higher user experience of the AR system has the potential of a higher revisitation system, and therewith additional mechanisms for trust calibration could be explored in future work.

Vehicle Assistant Systems’ Understanding and Operation

Multimodal AR User Onboarding leads to comparably lower interaction errors during automated parking. These findings are surprising at first sight when looking at questionnaire and quiz results because the understanding (potentials and boundaries) of the autopark system was comparable in both groups. When taking a closer look at the implementation of our Onboarding strategies, an explanation could be that the necessary information to answer the quiz (system-based knowledge) were coded textually in both conditions. The information about how to interact (interaction knowledge), in contrast, could be seen at the correct location in the car, while in the manual, this information had to be mentally visualized by participants, leading to an unprecise mental model of the interaction procedure (“I think a key should be activated here”, “On the display you should see a park symbol like this, but I can’t find where it is”). These findings support the dual coding assumption of CMTL [20,21] and previous work in other contexts that found AR leading to better task performance (cf. Blattgerste et al. [25]). Augmented reality tutorials allow us to visually concretize textual-encoded descriptions of the location of interaction processes at the right time and thus, help users building a more precise mental model of the interaction process. We see this investigation of a smartphone AR application as a first step towards creating multimodal User Onboarding experiences. Future work could also investigate other multimodal approaches, e.g., compare AR with a video-based tutorial on the center-screen, in a smartphone application, or on a virtual windshield—and test how these relations evolve over time with regard to long term retention of information.

6. Limitations

Like the sample of the online survey and the real-world study mainly consisted of male, technophile persons, so that results with other users, e.g., female or technophobe persons, might differ. Further, the sample size of the driving study was rather small, and the equally perceived trust in AR and manual conditions is likely to change with more participants.

The “autopark”-functionality is not as safety-critical as other assistants like the ACC or lane-keeping for users that require higher driving speed. Thus, the reported trust value might be influenced by the risk-taking behavior of participants. In this context, some users might find it more acceptable to not-be-sure how to interact or for the system to produce errors. Future work could investigate this effect.

The technical implementation of our AR approach relies on the smartphone screen, which is freely orientable, but not well usable over longer periods of time (holding the device). With technical advancement of AR glasses or lenses, these could be a viable alternative in the future - lightweight and without limiting the field of view to the screen. Therefore, it would also make sense to investigate these in future work. However, in this work, despite the technical limitations, we focused on AR implementation via smartphone, as these devices are already ubiquitous, and are easy to integrate the car manufacturers’ digital service ecosystems.

7. Conclusions

In this paper, we investigated the potential of multimodal AR User Onboarding for vehicle assistant systems. First, in a survey with modern vehicle owners, we learned that they show high reliance on those systems. However, they also report misunderstandings and misconceptions for most assistant systems and therefore see the need for further training. Then, in a real-world driving study, we tested for automated parking if a multimodal AR approach could improve the traditional text-based user manual approach. We learned that manuals (if read) are not inferior to AR tutorials in terms of system understanding. Users remembered facts well and showed high trust and user experience values for the automated parking process. However, AR’s potential is that it can convey knowledge about interaction procedures in a more precise manner by locally augmenting and guiding through spatially distributed interaction procedures leading to lower task-related errors. Moreover, given the higher hedonic experiences during Onboarding, AR might also increase the motivation to use a tutorial about a driving assistant system in the first place. Overall, vehicle designers could use multimodal AR User Onboarding as an additional opportunity to train customers and present new functions such as driver assistance systems in the car in an up-to-date, understandable, and interactive way.