The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios

Abstract

In the development of highly automated driving systems (L3 and 4), much research has been done on the subject of driver takeover. Strong focus has been placed on the takeover quality. Previous research has shown that one of the main influencing factors is the complexity of a traffic situation that has not been sufficiently addressed so far, as different approaches towards complexity exist. This paper differentiates between the objective complexity and the subjectively perceived complexity. In addition, the familiarity with a takeover situation is examined. Gold et al. show that repetition of takeover scenarios strongly influences the take-over performance. Yet, both complexity and familiarity have not been considered at the same time. Therefore, the aim of the present study is to examine the impact of objective complexity and familiarity on the subjectively perceived complexity and the resulting takeover quality. In a driving simulator study, participants are requested to take over vehicle control in an uncritical situation. Familiarity and objective complexity are varied by the number of surrounding vehicles and scenario repetitions. Subjective complexity is measured using the NASA-TLX; the takeover quality is gathered using the take-over controllability rating (TOC-Rating). The statistical evaluation results show that the parameters significantly influence the takeover quality. This is an important finding for the design of cognitive assistance systems for future highly automated and intelligent vehicles.

1. Introduction

Within recent years, human factors have become an important research topic in automating driving [1]. Approaching the Level 3 of automation [2], the driver may shift attention to a non driving related task (NDRT) during the automated drive. Still, the driver remains as fallback if the automation requests a takeover (TOR; [2]). Most takeover requests in Level 3 highly automated driving [2] will be non-critical [3], giving the driver sufficient comfortable transition time [4]. The focus in this study lies on non-critical takeover situations in different scenarios and the resulting takeover quality. In contrast to critical takeover situations, where drivers abbreviate the takeover process, the driver has enough time to properly perceive the driving environment before performing a maneuver. During the automated mode, the driver can engage into a non-driving related task. The takeover is a complex task. As soon as a TOR is triggered, the driver has to shift the attention back to the driving environment, perceive the surrounding traffic environment and take over the driving task. Hands and feet have to be relocated, situation awareness regained and the driving task has to be executed [5,6]. In addition the in-vehicle environment has to be perceived and filtered for relevant information. All these processes happen after the driver has been out-of the loop. In a small amount of time and a dynamic environment, these are several cognitive and motoric processes that have to happen in a very small amount of time. It is thus important to investigate aspects that affect a safe and comfortable takeover. In this paper, four relevant factors that influence the takeover quality are examined. In the following, the four factors are described separately and distinguished. Still, they are not independent from each other. First, the takeover process is influenced by the complexity of the surrounding traffic environment that can be defined as objective complexity. The objective complexity mainly varies in its amount of relevant objects in the surrounding environment. However, other factors, such as weather conditions, road structure and relative speed can also add up to objective complexity. Especially when taking over the driving task, the objective complexity can impact the quality of the takeover. Different studies [7,8] found that high traffic density leads to a reduced takeover quality when a lane change is required. A reason for this is that the choice of lane change is more complex than just braking as vehicles on the other lanes have to be perceived and time gaps and relative speeds estimated. Second, besides the objective complexity, individual differences have to be taken into account [8]. Not only the traffic situation but also the current state of the individual driver (e.g., stress level, vigilance, workload of non-driving related task) may differ in every takeover situation. This is called subjective complexity. The subjective complexity is task- and resource-dependent and describes an individuals’ subjective perception of complexity in a certain traffic situation [9]. Depending on the current attentional state of the driver, the perception of complex situations can vary. While one driver might be familiar and thus very comfortable with high traffic density and rate complexity of the situation as low, another driver might perceive the situation as more complex. Third, such an individual perception of complexity is influenced by the familiarity. Due to common driving routes of individual drivers, the familiarity with roads and therefore traffic situations (traffic jam, urban roads, villages etc.) varies. Reference [10] show that the overall response time is significantly lower for drivers who are familiar with the system. In unfamiliar situations, drivers thus have higher response times. This is highly important when dealing with safety aspects for takeover situations, as the takeover quality can be enhanced when lower reaction times are needed in familiar situations. Fourth, stable driver variables, such as driving style, driving frequency, driving routes and driving duration have an impact on the takeover quality. To improve the takeover quality, cognitive assistance systems can support the driver during a takeover. By integrating information about the surrounding traffic environment (objective complexity), the current state of the driver (subjective complexity), the customary traffic situations of individual drivers (familiarity) and stable driver variables, such as the driving style, the HMI as well as vehicle dynamics can be adapted. In a situation with high objective complexity and an unfamiliar driver, who perceives the situation as very complex, only relevant and supportive information would be presented to the driver (e.g., projection of best maneuver trajectory) and the automation would hand over the driving task gradually (e.g., handing over the steering but keeping adaptive cruise control activated). In highly familiar situations with low complexity, additional information, such as a radio channel, playing the favorite song or the time schedule of the next appointment could be presented to the driver to keep vigilance low. As it is already shown that the drivers’ familiarity with a situation and the objective complexity of the current traffic situation influence the subjective complexity [9,11,12], this study investigates the impact of the situational variables familiarity, objective complexity and subjective complexity on the takeover quality. Furthermore, stable individual variables are integrated. All variables are related to each other in different ways. Figure 1 represents the relationships that are investigated in the present study. Based on this, cognitive assistance systems can be developed to support individual drivers accordingly. The following hypotheses are examined in this study:

Hypothesis 1.

Higher familiarity with the situation is related to increased quality of a takeover.

Hypothesis 2.

Higher objective complexity is related to a decreased quality of the takeover.

Hypothesis 3.

Higher subjective complexity is related to a decreased quality of the takeover.

Hypothesis 4.

Situational and stable driver variables (driving style, driving frequency, driving routes and driving duration) together can best explain variance in takeover quality.

2. Methods

To rate the takeover quality, this study evaluates videos of a driving simulator study. The driving simulator consists of six monitors that create a 360${}^{\circ }$ surround view and a moveable driving unit to create a more realistic driving simulation. Six different traffic scenarios are built using the driving simulation SILAB [13]. Participants are tested in a controlled environment to enable measurements under exactly the same traffic conditions. A ten minutes learning session prior to the study is included for participants to get acquainted with simulator dynamics, notifications and the takeover itself. The implementation of the study is approved by the ethics committee of the TU Berlin in April 2019 and Robert Bosch GmbH.

2.1. Study Design

The study includes six scenarios with a different amount of relevant vehicles in the surrounding traffic environment (Section 2.3.1). In three blocks, each scenario is repeated once per block in randomized order. Overall, participants took over the driving task 18 times after an automated drive. Depending on each participant the global study duration lies between 90 and 120 min. After the mandatory documents, participants are theoretically instructed into the study (20–30 min). This is followed by a test drive in which participants get used to the simulator (5 min). Their main task is to drive onto the highway (starting from a parking lot) and onto the center lane, where they turn on the automation as soon as it is available. During the automated drive, they are instructed to play a quiz on a mounted tablet next to the center console until a takeover request is triggered. Each automated drive lasts around 2 min. As soon as the takeover is triggered, participants are instructed to immediately stop the quiz and take over. The takeover request is always triggered when the ego vehicle is driving on the center lane with a speed of 120 km/h. Participants are instructed to take over the driving task using the levers, and keep the speed at ca. 120 km/h. Each scenario triggers a certain maneuver that is the best solution in the given situation. Depending on the traffic situation (speed and position of relevant vehicles), participants should stick to the obligation to drive on the right and try not to break or accelerate enormously. Due to this, always one maneuver is most useful (right when the right lane is free; follow when the right is occupied and the leading vehicle faster or at the same speed; left when the right is occupied and the leading vehicle certainly slower than the ego vehicle). As soon as participants take an action decision the corresponding decision has to be indicated aloud. After each takeover, participants drive onto a parking lot to answer a rating sheet for subjective complexity (NASA-TLX; Section 2.3.3). From the parking lot, the next scenario starts as soon as participants finish the rating sheet. Depending on the time participants took to answer the rating sheet, each scenario lasts three to five minutes.

2.2. Participants

The simulator study took place in May and April 2019 after a successful pre-testing. Statistical evaluations base on N = 20 (13 male, 7 female) participants with a mean age of M = 26.2 years (SD = 2.69) who took part in the study. Most participants drive on average 30 min on a daily basis. They drive mostly on highways and indicate a moderate driving style (Figure 2).

2.3. Variables and Measurements

The study is designed to measure four main variables that are important for the takeover in highly automated driving. The connection between those variables is depicted in Figure 1. Variables and measurement methods are described in detail below.

2.3.1. Objective Complexity

The objective complexity is an independent variable (Figure 1) and based on the amount of relevant vehicles in the traffic environment. A vehicle is defined as relevant when it has a direct impact on the ego vehicle. Such a direct impact is either the necessity to react, the reason for a maneuver or a safety critical vehicle that has to be regarded during a maneuver (e.g., overtaking vehicles during a lane change to the left). Three different maneuver options are set up in the traffic simulation. The takeover is always triggered when the ego vehicle is in the highly automated mode on the center lane. Maneuver options are thus a lane change to the left, a lane change to the right or car following. Based on the obligation to drive on the right, the traffic environment is set up to trigger all three maneuvers. For every maneuver a complex and an easy traffic scenario exists. This results in overall six different scenarios that vary in their complexity based on the amount of vehicles relevant for the maneuver (0, 1, 2, 3, 6; Figure 3). Two scenarios have two relevant vehicles in the surrounding traffic environment that are similarly integrated into statistical analysis.

2.3.2. Familiarity

The second independent variable is the familiarity with a certain traffic situation (Figure 1). It is implemented by a repetition of the scenarios. Each scenario is represented three times for each participant in a randomized order. Therefore, the habituation to general traffic situations rises with repeated exposure.

2.3.3. Subjective Complexity

Subjective complexity is not a direct independent variable as it is not manipulated throughout the experiment. It indicates how complex participants perceive the scenario (individual perception of complexity in terms of “has this been a complex environment for you”). It is influenced by the objective complexity and the familiarity (Figure 1; [9]). To assess the subjective complexity, the multidimensional rating sheet NASA-Task Load IndeX /NASA-TLX; [14]) is used after each takeover. Originally the NASA-TLX is a rating scale in which information about magnitude and sources of six workload-related factors are combined to derive an estimate of workload. Due to its six sub-scales, the questionnaire is the most suitable to measure subjective complexity in takeover situations. On a 20-point likert scale, six different sub-scales are rated. The six sub-scales measure mental demand, physical demand, temporal demand, performance, effort and frustration. A weighting of the items as in [14] has been criticized in the past [15]. Reference [16] states that without the weighting of the scales a better differentiation and higher reliability can be achieved. Furthermore, it is stated that the weighting of the scales provides little informative value [17]. Another shortcoming of the weighting is the aspect of time that is additionally needed for the weighting. Based on this, the weighting is not used in this study. Participants are instructed to rate the complexity of the situation using the NASA-TLX after every trial, resulting in overall 18 ratings (six scenarios, three times each).

2.3.4. Takeover Quality

The takeover quality is the dependent variable (Figure 1). Both complexities and the familiarity are assumed to influence the takeover quality. The quality of the takeover is rated using the take-over controllability rating (TOC; [18]). The TOC is a procedure for an assessment of control transitions from automated to manual driving. It provides a standardized rating scheme on a scale from one to ten. Furthermore, it allows the integration of different aspects of driving performance during control transitions into a global measure when evaluating video material of a driving situation. The sub-scales of the TOC include braking response, longitudinal vehicle control, lateral vehicle control, lane change/lane choices, securing/communication, vehicle/system operation and the facial expression of the driver. The last sub-scale (facial expression of the driver) is not rated in this study as the video material does not include the face of the driver [18]. The sub-scales are rated on a 10-point scale. A perfect quality is rated with one. Values of two or three indicate imprecision. Those include jerky steering movement or imprecise lane keeping on the sub-scale of lateral vehicle control, unnecessary/wrong use of indicator on the sub-scale securing/communication, imprecision for vehicle/system operation and visible emotions on the sub-scale facial expression of the driver. Driving errors are rated between four and six, depending on the strength of the error. The following items indicate errors: too strong, too weak, too late, missing (braking response), safety distance too low, inadequate speed (longitudinal vehicle control), safety-distance too low, strong oscillation, crossing lane markings (lateral vehicle control), hesitant/interrupted, too late, missing, wrong lane (lane change/lane choices), missing/too late use of indicator, missing/too late control glance (securing/communication) and problems (vehicle/system operation). Endangerment is rated between seven and nine, including endangerment of others and self-endangerment over all the sub-scales. In cases of non-controllable events, the takeover is rated with a ten, including collision, lane departure/leaving road or loss of vehicle control over all the sub-scales [18]. Low values indicate a faultless takeover (= 1) and high quality. Higher values on the other hand indicate a bad quality of the takeover (10 = uncontrolled).

3. Results

Regression analysis is used to examine the influence of the independent variables on the dependent variable takeover quality. Residual vs. fitted, normal Q-Q, scale-location and residual vs. leverage plots are used to test on the model, normal distribution, homoscedasticity and outliers. To test on multicollinearity, the variance inflation factor is used. Mediation and moderation effects are tested as well, but no significant effects are found.

3.1. The Impact of Familiarity on Takeover Quality (H1)

To evaluate the impact that the familiarity with a traffic scenario has on the takeover quality, regression analysis is used. Results show that with a rise in familiarity, the quality of the takeover significantly improves ($\beta =-0.24,R^2=0.01,t\left(311\right)=-2,p<0.05$; Figure 4, right). The slope of the regression is with −0.24 not very high and only one percent of variance in the takeover quality can be explained by familiarity. This shows that familiarity has a significant impact on the takeover quality, but only a small one. It has to be stated though that all participants are regular highway drivers. Hence, the familiarity may have been high already.

3.2. The Impact of Objective Complexity on Takeover Quality (H2)

Additionally, the takeover quality is significantly influenced by the objective traffic complexity (Figure 4, left). With more relevant vehicles in the surrounding traffic environment, the takeover quality becomes worse. In scenarios that have a bad TOC rating, drivers do not hold enough safety distance, changed the lane very hesitant and interrupted, did not use the indicator or did not do the control glance. In cases where the low safety distance could have lead to a collision in real traffic, endangerment of self and others is rated. Wrong decisions did not influence the takeover quality when the maneuver was executed perfectly. Results show that the slope of the regression is 0.17. Three percent of variance can be explained by the amount of relevant vehicles in the surrounding traffic environment ($\beta =0.17,R^2=0.03,t\left(311\right)=3.44,p<0.001$). The small amount of variance that can be explained can again be due to the participants driving history. As all drivers are used to highway situations where the objective complexity is usually high, the impact might be reduced due to the increased familiarity. Furthermore, other aspects that add up to objective complexity (e.g., traffic signs) may also play an important role.

3.3. The Impact of Subjective Complexity on Takeover Quality (H3)

The subjective complexity measures how complex each individual perceives the situation. It is significantly influenced by the objective complexity of the environment ($\beta =0.55,p<0.001$) and the familiarity with the situation ($\beta =-0.83,p<0.001$; Figure 1; [9]). In addition, the aggregated subjective complexity has a significant impact on the takeover quality ($\beta =0.07,p<0.05$). A driver who perceives a situation as highly complex has a worse quality of the takeover (Figure 5). Although the impact is significant, only one percent of variance can be explained by the aggregated subjective complexity ($R^2=0.01,t\left(311\right)=2.33,p<0.05$). Subjective complexity consists of the six different sub-scales mental demand, physical demand, temporal demand, performance, effort and frustration. Mental demand and physical demand do not influence the takeover quality significantly. However, with a rise in temporal demand, the takeover quality decreases significantly ($\beta =0.1,t\left(306\right)=2.26,p<0.05$). In addition, the takeover quality decreases with a rise in frustration ($\beta =0.11,t\left(306\right)=2.86,p<0.01$). Surprisingly, with a rise in the perceived performance, the actual takeover quality also decreases ($\beta =0.08,t\left(306\right)=2.54,p<0.05$). Furthermore, the effort has a positive effect on the takeover quality ($\beta =-0.15,t\left(306\right)=-4.36,p<0.001$). Multiple linear regression analysis of the sub-scales can explain ten percent of variance in takeover quality ($R^2=0.1$; Figure 5). Figure 5 shows that many scores lie on the fourth marker. In the TOC rating, driving errors are rated between four and six. After taking over in this study, a lot of drivers make driving errors. These errors are mostly not enough distance, too strong braking, a missing use of indicators or a missing control glance. As these errors are not severe (e.g., low distance but no cutting in on other vehicles) in these cases, the lowest driving error rating is chosen.

3.4. Multiple Regression Analysis on Takeover Quality Including Stable Driver Variables (H4)

Separately, the variables show significant relationships, but the amount of variance in takeover quality that can be explained is not high. To estimate the impact of the combination of the variables, multiple regression analysis is used (Figure 6). Results show that a combination of stable (e.g., driving style) and situational variables (e.g., objective complexity) increases the amount of variance in takeover quality that can be explained to 58 percent. The stable variables that significantly influence takeover quality are indicated driving style, average driving frequency, most used driving routes and average driving duration. The takeover quality decreases with a more defensive driving style ($\beta =0.77,t\left(183\right)=5.85,p<0.001$), less driving frequency ($\beta =-0.41,t\left(183\right)=-8.03,p<0.001$) and longer average driving duration ($\beta =0.44,t\left(183\right)=4.77,p<0.01$). More frequent highway usage is related to a better takeover quality ($\beta =1.17,t\left(183\right)=8.4,p<0.001$). Situation familiarity is not significant in the multiple linear regression anymore. Similarly, the objective complexity is only significant on a .1 level ($\beta =0.09,t\left(183\right)=1.95,p<0.1$). The sub-scales temporal demand, effort and frustration from subjective complexity add to the multiple linear regression. The higher temporal demand ($\beta =0.09,t\left(183\right)=3.16,p<0.01$) and frustration ($\beta =0.1,t\left(183\right)=3.19,p<0.01$), the lower is the resulting takeover quality. The more effort is spent during a takeover on the other hand, the better is the resulting quality ($\beta =-0.14,t\left(183\right)=-4.67,p<0.001$). In contrast to the simple linear regressions, multiple linear regression shows that the combination of the above mentioned variables give a better understanding on how the variables influence takeover quality (Figure 6). Regression results can be used to compute predictions of takeover quality, depending on the input data that is available.

4. Discussion

Results show that a combination of stable and situational variables can be used to explain 58 percent of variance in takeover quality. This new finding is important for the development of highly automated driving. Depending on the variables that can be assessed, a prediction of the takeover quality can now be made and cognitive assistance systems for highly automated driving adapted accordingly. In a user profile for example, the stable driver characteristics can be stored and used for predictions. Based on previous rides, the profile can adapt and store information about the drivers familiarity with certain situations. In combination with that, sensors of highly automated vehicles are able to provide information about the objective complexity of the current traffic environment. In contrast to these variables, measuring subjective complexity is more challenging. To integrate subjective complexity measurements into such a system, a faster and more easily manageable measurement method than the NASA-TLX rating sheet is needed. A way to measure subjective complexity is via eye-tracking (e.g., saccade distance [19], fixation times [20]) or physiological data (e.g., heart rate [21], skin conductance [22]). However, eye-tracking has to be supported in the corresponding vehicle or the driver wears a smart-watch featuring health tracking. Considering the current trend, these two measurement techniques are very likely. By integrating eye-tracking or physiological data, information about the current subjective complexity can be collected. In combination with measurements of the other situational and stable variables, good predictions about the current situation and the drivers state can be made. Based on this, cognitive assistance can support the driver during a takeover situation. Vehicle dynamics and the HMI can be adapted to increase the takeover quality. The results of the study already provide a very good basis for variables that are relevant for the takeover quality. For future research, it is important to consider further variables that might be important. Other distracting objects in the environment that are not vehicles (e.g., traffic signs, roadside environment), current stress level, vigilance and other variables are important to consider in future research. Furthermore, investigation in eye-tracking and physiological measurement methods to capture subjective complexity is important. If these methods are able to measure subjective complexity validly, a next step towards cognitive assistance systems that can be adapted based on the needs of the individual on hand is made.

5. Conclusions

In sum, it can be shown that already 58 percent of variance in takeover quality can be explained by the observed variables of this study. Those are the stable variables driving style, driving frequency, driving routes and driving duration as well as aspects of the situational variable subjective complexity. Objective complexity and familiarity did not become significant in the multiple regression analysis, but show a significant impact when taken separately. In future research it is thus still important to consider these variables. Stable variables can easily be stored in a user profile. Situational variables on the other hand have to be updated and integrated permanently. Different measurement methods have to be used and their output combined to validly display situational variables. Such a combination could be for example the integration of high traffic density (high objective complexity), a high heart rate or skin conductance level and a low saccade distance (high subjective complexity), low familiarity and a defensive driving style. Based on this combination, cognitive assistance would support with relevant information (e.g., projection of optimal driving trajectory), but suppress irrelevant information (e.g., radio or weather information). In addition, the automation would adapt vehicle parameters, such as decelerating while handing over or handing over step by step (e.g., first lateral dynamics—steering, second longitudinal dynamics—acceleration and deceleration). This process has to be very fast as takeover times are short and cognitive assistance has to be given as soon as possible. This paper gives an important selection of relevant variables that influence takeover quality. Based on this it is important to consider valid and fast measurement methods for situational variables and find further variables that influence the takeover quality. Then, cognitive assistance can be developed, individualized and adapted instantaneously.