Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.8
3.1
Journals
Information
Special Issues
Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles
share announcement

Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles

A special issue of Information (ISSN 2078-2489). This special issue belongs to the section "Information Applications".

Deadline for manuscript submissions: closed (31 May 2020) | Viewed by 93035

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special Issue Editors

Dr. Frederik Naujoks

E-Mail Website
Guest Editor
BMW Group, 80809 Munich, Germany
Interests: usability; human–machine interaction; human factors; automated driving
Special Issues, Collections and Topics in MDPI journals
Dr. Sebastian Hergeth

E-Mail Website
Guest Editor
BMW Group, 80809 Munich, Germany
Interests: human-machine interaction; safety in use; distracted driving; usability; human factors; assisted and automated driving
Special Issues, Collections and Topics in MDPI journals
Dr. Andreas Keinath

E-Mail Website
Guest Editor
BMW Group, 80809 Munich, Germany
Interests: usability; human–machine interaction; human factors; automated driving
Special Issues, Collections and Topics in MDPI journals
Dr. Nadja Schömig

E-Mail Website
Guest Editor
Wuerzburg Institute for Traffic Sciences, 97209 Veitshöchheim, Germany
Interests: usability; human–machine interaction; human factors; automated driving
Special Issues, Collections and Topics in MDPI journals
Dr. Katharina Wiedemann

E-Mail Website
Guest Editor
Wuerzburg Institute for Traffic Sciences, 97209 Veitshöchheim, Germany
Interests: usability; human–machine interaction; human factors; automated driving
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Today, OEMs and suppliers can rely on commonly agreed and standardized testing and evaluating methods for in-vehicle human–machine interfaces (HMIs). These have traditionally focused on the context of manually driven vehicles and put the evaluation of minimizing distraction effects and enhancing usability at their core (e.g., AAM guidelines or NHTSA visual distraction guidelines).

However, advances in automated driving systems (ADS) have already begun to change the driver’s role from actively driving the vehicle to monitoring the driving situation and being ready to intervene in partially automated driving (SAE L2). Higher levels of vehicle automation will likely only require the driver to act as a fallback ready user in case of system limits and malfunctions (SAE L3) or could even act without any fallback within their operational design domain (SAE L4). During the same trip, different levels of automation might be available to the driver (e.g., L2 in urban environments, L3 on highways). These developments require new test and evaluation methods for ADS, as available test methods cannot be easily transferred and adapted.

For example, The ADS HMI should be capable of informing the user about the current mode and minimize confusion about the status of the ADS and the user’s current responsibilities (e.g., whether the ADS is functioning properly, ready for use, unavailable for use or requesting a transition of control from the ADS to the user). While ADS might allow new and more comfortable seating positions and engagement in nondriving-related tasks that were not allowed in manual driving, these might generate motion sickness or lower the user’s availability for a transfer of control. As the driving task is no longer actively fulfilled by the driver, distraction by nondriving-related tasks might turn into controlled engagement. ADS might behave differently than manually driven vehicles, which might generate a need for external HMIs or standardized motion patterns to adequately interact with non-equipped traffic participants.

This Special Issue welcomes theoretical papers as well as empirical studies that deal with these new challenges by proposing new and innovative test methods in the evaluation of ADS HMIs in areas such as (but not limited to) the topics below:

- Mode awareness and mode indicators;
- Testing of minimum HMI requirements;
- Driver state in the context of ADS (e.g., distraction or drowsiness);
- Trust in ADS;
- External HMIs for ADS;
- Guidelines for HMIs for ADS;
- Motion sickness in ADS;
- Validity of test settings (on-road, driving simulators, etc.);
- Learnability and usability of ADS;
- Comfortable and pleasurable user experience of ADS.

Dr. Frederik Naujoks
Dr. Sebastian Hergeth
Dr. Andreas Keinath
Dr. Nadja Schömig
Ms. Katharina Wiedemann
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Information is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Automated driving
  • Human–machine interface
  • Test methods
  • User studies
  • Evaluation

Published Papers (22 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Editorial

Jump to: Research, Review, Other

7 pages, 202 KiB  
Editorial
Editorial for Special Issue: Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles
by Frederik Naujoks, Sebastian Hergeth, Andreas Keinath, Nadja Schömig and Katharina Wiedemann
Information 2020, 11(9), 403; https://0-doi-org.brum.beds.ac.uk/10.3390/info11090403 - 20 Aug 2020
Cited by 1 | Viewed by 2233
Abstract
Today, OEMs and suppliers can rely on commonly agreed and standardized test and evaluation methods for in-vehicle human–machine interfaces (HMIs). These have traditionally focused on the context of manually driven vehicles and put the evaluation of minimizing distraction effects and enhancing usability at [...] Read more.
Today, OEMs and suppliers can rely on commonly agreed and standardized test and evaluation methods for in-vehicle human–machine interfaces (HMIs). These have traditionally focused on the context of manually driven vehicles and put the evaluation of minimizing distraction effects and enhancing usability at their core (e.g., AAM guidelines or NHTSA visual-manual distraction guidelines). However, advances in automated driving systems (ADS) have already begun to change the driver’s role from actively driving the vehicle to monitoring the driving situation and being ready to intervene in partially automated driving (SAE L2). Higher levels of vehicle automation will likely only require the driver to act as a fallback ready user in case of system limits and malfunctions (SAE L3) or could even act without any fallback within their operational design domain (SAE L4). During the same trip, different levels of automation might be available to the driver (e.g., L2 in urban environments, L3 on highways). These developments require new test and evaluation methods for ADS, as available test methods cannot be easily transferred and adapted. The shift towards higher levels of vehicle automation has also moved the discussion towards the interaction between automated and non-automated road users using exterior HMIs. This Special Issue includes theoretical papers a well as empirical studies that deal with these new challenges by proposing new and innovative test methods in the evaluation of ADS HMIs in different areas. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)

Research

Jump to: Editorial, Review, Other

15 pages, 1674 KiB  
Article
Measuring Drivers’ Physiological Response to Different Vehicle Controllers in Highly Automated Driving (HAD): Opportunities for Establishing Real-Time Values of Driver Discomfort
by Vishnu Radhakrishnan, Natasha Merat, Tyron Louw, Michael G. Lenné, Richard Romano, Evangelos Paschalidis, Foroogh Hajiseyedjavadi, Chongfeng Wei and Erwin R. Boer
Information 2020, 11(8), 390; https://0-doi-org.brum.beds.ac.uk/10.3390/info11080390 - 08 Aug 2020
Cited by 11 | Viewed by 4034
Abstract
This study investigated how driver discomfort was influenced by different types of automated vehicle (AV) controllers, compared to manual driving, and whether this response changed in different road environments, using heart-rate variability (HRV) and electrodermal activity (EDA). A total of 24 drivers were [...] Read more.
This study investigated how driver discomfort was influenced by different types of automated vehicle (AV) controllers, compared to manual driving, and whether this response changed in different road environments, using heart-rate variability (HRV) and electrodermal activity (EDA). A total of 24 drivers were subjected to manual driving and four AV controllers: two modelled to depict “human-like” driving behaviour, one conventional lane-keeping assist controller, and a replay of their own manual drive. Each drive lasted for ~15 min and consisted of rural and urban environments, which differed in terms of average speed, road geometry and road-based furniture. Drivers showed higher skin conductance response (SCR) and lower HRV during manual driving, compared to the automated drives. There were no significant differences in discomfort between the AV controllers. SCRs and subjective discomfort ratings showed significantly higher discomfort in the faster rural environments, when compared to the urban environments. Our results suggest that SCR values are more sensitive than HRV-based measures to continuously evolving situations that induce discomfort. Further research may be warranted in investigating the value of this metric in assessing real-time driver discomfort levels, which may help improve acceptance of AV controllers. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

21 pages, 46200 KiB  
Article
Efficient Paradigm to Measure Street-Crossing Onset Time of Pedestrians in Video-Based Interactions with Vehicles
by Stefanie M. Faas, Stefan Mattes, Andrea C. Kao and Martin Baumann
Information 2020, 11(7), 360; https://0-doi-org.brum.beds.ac.uk/10.3390/info11070360 - 11 Jul 2020
Cited by 10 | Viewed by 3665
Abstract
With self-driving vehicles (SDVs), pedestrians can no longer rely on a human driver. Previous research suggests that pedestrians may benefit from an external Human–Machine Interface (eHMI) displaying information to surrounding traffic participants. This paper introduces a natural methodology to compare eHMI concepts from [...] Read more.
With self-driving vehicles (SDVs), pedestrians can no longer rely on a human driver. Previous research suggests that pedestrians may benefit from an external Human–Machine Interface (eHMI) displaying information to surrounding traffic participants. This paper introduces a natural methodology to compare eHMI concepts from a pedestrian’s viewpoint. To measure eHMI effects on traffic flow, previous video-based studies instructed participants to indicate their crossing decision with interfering data collection devices, such as pressing a button or slider. We developed a quantifiable concept that allows participants to naturally step off a sidewalk to cross the street. Hidden force-sensitive resistor sensors recorded their crossing onset time (COT) in response to real-life videos of approaching vehicles in an immersive crosswalk simulation environment. We validated our method with an initial study of N = 34 pedestrians by showing (1) that it is able to detect significant eHMI effects on COT as well as subjective measures of perceived safety and user experience. The approach is further validated by (2) replicating the findings of a test track study and (3) participants’ reports that it felt natural to take a step forward to indicate their street crossing decision. We discuss the benefits and limitations of our method with regard to related approaches. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

24 pages, 2012 KiB  
Article
Feeling Uncertain—Effects of a Vibrotactile Belt that Communicates Vehicle Sensor Uncertainty
by Matti Krüger, Tom Driessen, Christiane B. Wiebel-Herboth, Joost C. F. de Winter and Heiko Wersing
Information 2020, 11(7), 353; https://0-doi-org.brum.beds.ac.uk/10.3390/info11070353 - 06 Jul 2020
Cited by 7 | Viewed by 3419
Abstract
With the rise of partially automated cars, drivers are more and more required to judge the degree of responsibility that can be delegated to vehicle assistant systems. This can be supported by utilizing interfaces that intuitively convey real-time reliabilities of system functions such [...] Read more.
With the rise of partially automated cars, drivers are more and more required to judge the degree of responsibility that can be delegated to vehicle assistant systems. This can be supported by utilizing interfaces that intuitively convey real-time reliabilities of system functions such as environment sensing. We designed a vibrotactile interface that communicates spatiotemporal information about surrounding vehicles and encodes a representation of spatial uncertainty in a novel way. We evaluated this interface in a driving simulator experiment with high and low levels of human and machine confidence respectively caused by simulated degraded vehicle sensor precision and limited human visibility range. Thereby we were interested in whether drivers (i) could perceive and understand the vibrotactile encoding of spatial uncertainty, (ii) would subjectively benefit from the encoded information, (iii) would be disturbed in cases of information redundancy, and (iv) would gain objective safety benefits from the encoded information. To measure subjective understanding and benefit, a custom questionnaire, Van der Laan acceptance ratings and NASA TLX scores were used. To measure the objective benefit, we computed the minimum time-to-contact as a measure of safety and gaze distributions as an indicator for attention guidance. Results indicate that participants were able to understand the encoded uncertainty and spatiotemporal information and purposefully utilized it when needed. The tactile interface provided meaningful support despite sensory restrictions. By encoding spatial uncertainties, it successfully extended the operating range of the assistance system. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

15 pages, 2218 KiB  
Article
How Much Space Is Required? Effect of Distance, Content, and Color on External Human–Machine Interface Size
by Michael Rettenmaier, Jonas Schulze and Klaus Bengler
Information 2020, 11(7), 346; https://0-doi-org.brum.beds.ac.uk/10.3390/info11070346 - 03 Jul 2020
Cited by 18 | Viewed by 4163
Abstract
The communication of an automated vehicle (AV) with human road users can be realized by means of an external human–machine interface (eHMI), such as displays mounted on the AV’s surface. For this purpose, the amount of time needed for a human interaction partner [...] Read more.
The communication of an automated vehicle (AV) with human road users can be realized by means of an external human–machine interface (eHMI), such as displays mounted on the AV’s surface. For this purpose, the amount of time needed for a human interaction partner to perceive the AV’s message and to act accordingly has to be taken into account. Any message displayed by an AV must satisfy minimum size requirements based on the dynamics of the road traffic and the time required by the human. This paper examines the size requirements of displayed text or symbols for ensuring the legibility of a message. Based on the limitations of available package space in current vehicle models and the ergonomic requirements of the interface design, an eHMI prototype was developed. A study involving 30 participants varied the content type (text and symbols) and content color (white, red, green) in a repeated measures design. We investigated the influence of content type on content size to ensure legibility from a constant distance. We also analyzed the influence of content type and content color on the human detection range. The results show that, at a fixed distance, text has to be larger than symbols in order to maintain legibility. Moreover, symbols can be discerned from a greater distance than text. Color had no content overlapping effect on the human detection range. In order to ensure the maximum possible detection range among human road users, an AV should display symbols rather than text. Additionally, the symbols could be color-coded for better message comprehension without affecting the human detection range. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

15 pages, 1776 KiB  
Article
Sleep Inertia Countermeasures in Automated Driving: A Concept of Cognitive Stimulation
by Johanna Wörle, Ramona Kenntner-Mabiala, Barbara Metz, Samantha Fritzsch, Christian Purucker, Dennis Befelein and Andy Prill
Information 2020, 11(7), 342; https://0-doi-org.brum.beds.ac.uk/10.3390/info11070342 - 30 Jun 2020
Cited by 7 | Viewed by 3614
Abstract
When highly automated driving is realized, the role of the driver will change dramatically. Drivers will even be able to sleep during the drive. However, when awaking from sleep, drivers often experience sleep inertia, meaning they are feeling groggy and are impaired in [...] Read more.
When highly automated driving is realized, the role of the driver will change dramatically. Drivers will even be able to sleep during the drive. However, when awaking from sleep, drivers often experience sleep inertia, meaning they are feeling groggy and are impaired in their driving performance―which can be an issue with the concept of dual-mode vehicles that allow both manual and automated driving. Proactive methods to avoid sleep inertia like the widely applied ‘NASA nap’ are not immediately practicable in automated driving. Therefore, a reactive countermeasure, the sleep inertia counter-procedure for drivers (SICD), has been developed with the aim to activate and motivate the driver as well as to measure the driver’s alertness level. The SICD is evaluated in a study with N = 21 drivers in a level highly automation driving simulator. The SICD was able to activate the driver after sleep and was perceived as “assisting” by the drivers. It was not capable of measuring the driver’s alertness level. The interpretation of the findings is limited due to a lack of a comparative baseline condition. Future research is needed on direct comparisons of different countermeasures to sleep inertia that are effective and accepted by drivers. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

32 pages, 1747 KiB  
Article
Methodological Approach towards Evaluating the Effects of Non-Driving Related Tasks during Partially Automated Driving
by Cornelia Hollander, Nadine Rauh, Frederik Naujoks, Sebastian Hergeth, Josef F. Krems and Andreas Keinath
Information 2020, 11(7), 340; https://0-doi-org.brum.beds.ac.uk/10.3390/info11070340 - 30 Jun 2020
Cited by 4 | Viewed by 3221
Abstract
Partially automated driving (PAD, Society of Automotive Engineers (SAE) level 2) features provide steering and brake/acceleration support, while the driver must constantly supervise the support feature and intervene if needed to maintain safety. PAD could potentially increase comfort, road safety, and traffic efficiency. [...] Read more.
Partially automated driving (PAD, Society of Automotive Engineers (SAE) level 2) features provide steering and brake/acceleration support, while the driver must constantly supervise the support feature and intervene if needed to maintain safety. PAD could potentially increase comfort, road safety, and traffic efficiency. As during manual driving, users might engage in non-driving related tasks (NDRTs). However, studies systematically examining NDRT execution during PAD are rare and most importantly, no established methodologies to systematically evaluate driver distraction during PAD currently exist. The current project’s goal was to take the initial steps towards developing a test protocol for systematically evaluating NDRT’s effects during PAD. The methodologies used for manual driving were extended to PAD. Two generic take-over situations addressing system limits of a given PAD regarding longitudinal and lateral control were implemented to evaluate drivers’ supervisory and take-over capabilities while engaging in different NDRTs (e.g., manual radio tuning task). The test protocol was evaluated and refined across the three studies (two simulator and one test track). The results indicate that the methodology could sensitively detect differences between the NDRTs’ influences on drivers’ take-over and especially supervisory capabilities. Recommendations were formulated regarding the test protocol’s use in future studies examining the effects of NDRTs during PAD. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

24 pages, 4514 KiB  
Article
Comparison of Methods to Evaluate the Influence of an Automated Vehicle’s Driving Behavior on Pedestrians: Wizard of Oz, Virtual Reality, and Video
by Tanja Fuest, Elisabeth Schmidt and Klaus Bengler
Information 2020, 11(6), 291; https://0-doi-org.brum.beds.ac.uk/10.3390/info11060291 - 29 May 2020
Cited by 16 | Viewed by 3592
Abstract
Integrating automated vehicles into mixed traffic entails several challenges. Their driving behavior must be designed such that is understandable for all human road users, and that it ensures an efficient and safe traffic system. Previous studies investigated these issues, especially regarding the communication [...] Read more.
Integrating automated vehicles into mixed traffic entails several challenges. Their driving behavior must be designed such that is understandable for all human road users, and that it ensures an efficient and safe traffic system. Previous studies investigated these issues, especially regarding the communication between automated vehicles and pedestrians. These studies used different methods, e.g., videos, virtual reality, or Wizard of Oz vehicles. However, the extent of transferability between these studies is still unknown. Therefore, we replicated the same study design in four different settings: two video, one virtual reality, and one Wizard of Oz setup. In the first video setup, videos from the virtual reality setup were used, while in the second setup, we filmed the Wizard of Oz vehicle. In all studies, participants stood at the roadside in a shared space. An automated vehicle approached from the left, using different driving profiles characterized by changing speed to communicate its intention to let the pedestrians cross the road. Participants were asked to recognize the intention of the automated vehicle and to press a button as soon as they realized this intention. Results revealed differences in the intention recognition time between the four study setups, as well as in the correct intention rate. The results from vehicle–pedestrian interaction studies published in recent years that used different study settings can therefore only be compared to each other to a limited extent. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

14 pages, 4918 KiB  
Article
Effects of Marking Automated Vehicles on Human Drivers on Highways
by Tanja Fuest, Alexander Feierle, Elisabeth Schmidt and Klaus Bengler
Information 2020, 11(6), 286; https://0-doi-org.brum.beds.ac.uk/10.3390/info11060286 - 28 May 2020
Cited by 15 | Viewed by 3093
Abstract
Due to the short range of the sensor technology used in automated vehicles, we assume that the implemented driving strategies may initially differ from those of human drivers. Nevertheless, automated vehicles must be able to move safely through manual road traffic. Initially, they [...] Read more.
Due to the short range of the sensor technology used in automated vehicles, we assume that the implemented driving strategies may initially differ from those of human drivers. Nevertheless, automated vehicles must be able to move safely through manual road traffic. Initially, they will behave as carefully as human learners do. In the same way that driving-school vehicles tend to be marked in Germany, markings for automated vehicles could also prove advantageous. To this end, a simulation study with 40 participants was conducted. All participants experienced three different highway scenarios, each with and without a marked automated vehicle. One scenario was based around some roadworks, the next scenario was a traffic jam, and the last scenario involved a lane change. Common to all scenarios was that the automated vehicles strictly adhered to German highway regulations, and therefore moved in road traffic somewhat differently to human drivers. After each trial, we asked participants to rate how appropriate and disturbing the automated vehicle’s driving behavior was. We also measured objective data, such as the time of a lane change and the time headway. The results show no differences for the subjective and objective data regarding the marking of an automated vehicle. Reasons for this might be that the driving behavior itself is sufficiently informative for humans to recognize an automated vehicle. In addition, participants experienced the automated vehicle’s driving behavior for the first time, and it is reasonable to assume that an adjustment of the humans’ driving behavior would take place in the event of repeated encounters. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

13 pages, 398 KiB  
Article
Human–Vehicle Integration in the Code of Practice for Automated Driving
by Stefan Wolter, Giancarlo Caccia Dominioni, Sebastian Hergeth, Fabio Tango, Stuart Whitehouse and Frederik Naujoks
Information 2020, 11(6), 284; https://0-doi-org.brum.beds.ac.uk/10.3390/info11060284 - 27 May 2020
Cited by 5 | Viewed by 3287
Abstract
The advancement of SAE Level 3 automated driving systems requires best practices to guide the development process. In the past, the Code of Practice for the Design and Evaluation of ADAS served this role for SAE Level 1 and 2 systems. The challenges [...] Read more.
The advancement of SAE Level 3 automated driving systems requires best practices to guide the development process. In the past, the Code of Practice for the Design and Evaluation of ADAS served this role for SAE Level 1 and 2 systems. The challenges of Level 3 automation make it necessary to create a new Code of Practice for automated driving (CoP-AD) as part of the public-funded European project L3Pilot. It provides the developer with a comprehensive guideline on how to design and test automated driving functions, with a focus on highway driving and parking. A variety of areas such as Functional Safety, Cybersecurity, Ethics, and finally the Human–Vehicle Integration are part of it. This paper focuses on the latter, the Human Factors aspects addressed in the CoP-AD. The process of gathering the topics for this category is outlined in the body of the paper. Thorough literature reviews and workshops were part of it. A summary is given on the draft content of the CoP-AD Human–Vehicle Integration topics. This includes general Human Factors related guidelines as well as Mode Awareness, Trust, and Misuse. Driver Monitoring is highlighted as well, together with the topic of Controllability and the execution of Customer Clinics. Furthermore, the Training and Variability of Users is included. Finally, the application of the CoP-AD in the development process for Human-Vehicle Integration is illustrated. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

13 pages, 770 KiB  
Article
Mode Awareness and Automated Driving—What Is It and How Can It Be Measured?
by Christina Kurpiers, Bianca Biebl, Julia Mejia Hernandez and Florian Raisch
Information 2020, 11(5), 277; https://0-doi-org.brum.beds.ac.uk/10.3390/info11050277 - 21 May 2020
Cited by 20 | Viewed by 4266
Abstract
In SAE (Society of Automotive Engineers) Level 2, the driver has to monitor the traffic situation and system performance at all times, whereas the system assumes responsibility within a certain operational design domain in SAE Level 3. The different responsibility allocation in these [...] Read more.
In SAE (Society of Automotive Engineers) Level 2, the driver has to monitor the traffic situation and system performance at all times, whereas the system assumes responsibility within a certain operational design domain in SAE Level 3. The different responsibility allocation in these automation modes requires the driver to always be aware of the currently active system and its limits to ensure a safe drive. For that reason, current research focuses on identifying factors that might promote mode awareness. There is, however, no gold standard for measuring mode awareness and different approaches are used to assess this highly complex construct. This circumstance complicates the comparability and validity of study results. We thus propose a measurement method that combines the knowledge and the behavior pillar of mode awareness. The latter is represented by the relational attention ratio in manual, Level 2 and Level 3 driving as well as the controllability of a system limit in Level 2. The knowledge aspect of mode awareness is operationalized by a questionnaire on the mental model for the automation systems after an initial instruction as well as an extensive enquiry following the driving sequence. Further assessments of system trust, engagement in non-driving related tasks and subjective mode awareness are proposed. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

21 pages, 5582 KiB  
Article
Multi-Vehicle Simulation in Urban Automated Driving: Technical Implementation and Added Benefit
by Alexander Feierle, Michael Rettenmaier, Florian Zeitlmeir and Klaus Bengler
Information 2020, 11(5), 272; https://0-doi-org.brum.beds.ac.uk/10.3390/info11050272 - 19 May 2020
Cited by 12 | Viewed by 3683
Abstract
This article investigates the simultaneous interaction between an automated vehicle (AV) and its passenger, and between the same AV and a human driver of another vehicle. For this purpose, we have implemented a multi-vehicle simulation consisting of two driving simulators, one for the [...] Read more.
This article investigates the simultaneous interaction between an automated vehicle (AV) and its passenger, and between the same AV and a human driver of another vehicle. For this purpose, we have implemented a multi-vehicle simulation consisting of two driving simulators, one for the AV and one for the manual vehicle. The considered scenario is a road bottleneck with a double-parked vehicle either on one side of the road or on both sides of the road where an AV and a simultaneously oncoming human driver negotiate the right of way. The AV communicates to its passenger via the internal automation human–machine interface (HMI) and it concurrently displays the right of way to the human driver via an external HMI. In addition to the regular encounters, this paper analyzes the effect of an automation failure, where the AV first communicates to yield the right of way and then changes its strategy and passes through the bottleneck first despite oncoming traffic. The research questions the study aims to answer are what methods should be used for the implementation of multi-vehicle simulations with one AV, and if there is an added benefit of this multi-vehicle simulation compared to single-driver simulator studies. The results show an acceptable synchronicity for using traffic lights as basic synchronization and a distance control as the detail synchronization method. The participants had similar passing times in the multi-vehicle simulation compared to a previously conducted single-driver simulation. Moreover, there was a lower crash rate in the multi-vehicle simulation during the automation failure. Concluding the results, the proposed method seems to be an appropriate solution to implement multi-vehicle simulation with one AV. Additionally, multi-vehicle simulation offers a benefit if more than one human affects the interaction within a scenario. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

22 pages, 3070 KiB  
Article
Methodological Considerations Concerning Motion Sickness Investigations during Automated Driving
by Dominik Mühlbacher, Markus Tomzig, Katharina Reinmüller and Lena Rittger
Information 2020, 11(5), 265; https://0-doi-org.brum.beds.ac.uk/10.3390/info11050265 - 13 May 2020
Cited by 18 | Viewed by 5198
Abstract
Automated driving vehicles will allow all occupants to spend their time with various non-driving related tasks like relaxing, working, or reading during the journey. However, a significant percentage of people is susceptible to motion sickness, which limits the comfort of engaging in those [...] Read more.
Automated driving vehicles will allow all occupants to spend their time with various non-driving related tasks like relaxing, working, or reading during the journey. However, a significant percentage of people is susceptible to motion sickness, which limits the comfort of engaging in those tasks during automated driving. Therefore, it is necessary to investigate the phenomenon of motion sickness during automated driving and to develop countermeasures. As most existing studies concerning motion sickness are fundamental research studies, a methodology for driving studies is yet missing. This paper discusses methodological aspects for investigating motion sickness in the context of driving including measurement tools, test environments, sample, and ethical restrictions. Additionally, methodological considerations guided by different underlying research questions and hypotheses are provided. Selected results from own studies concerning motion sickness during automated driving which were conducted in a motion-based driving simulation and a real vehicle are used to support the discussion. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

14 pages, 2457 KiB  
Article
Engagement in Non-Driving Related Tasks as a Non-Intrusive Measure for Mode Awareness: A Simulator Study
by Yannick Forster, Viktoria Geisel, Sebastian Hergeth, Frederik Naujoks and Andreas Keinath
Information 2020, 11(5), 239; https://0-doi-org.brum.beds.ac.uk/10.3390/info11050239 - 28 Apr 2020
Cited by 12 | Viewed by 3060
Abstract
Research on the role of non-driving related tasks (NDRT) in the area of automated driving is indispensable. At the same time, the construct mode awareness has received considerable interest in regard to human–machine interface (HMI) evaluation. Based on the expectation that HMI design [...] Read more.
Research on the role of non-driving related tasks (NDRT) in the area of automated driving is indispensable. At the same time, the construct mode awareness has received considerable interest in regard to human–machine interface (HMI) evaluation. Based on the expectation that HMI design and practice with different levels of driving automation influence NDRT engagement, a driving simulator study was conducted. In a 2 × 5 (automation level x block) design, N = 49 participants completed several transitions of control. They were told that they could engage in an NDRT if they felt safe and comfortable to do so. The NDRT was the Surrogate Reference Task (SuRT) as a representative of a wide range of visual–manual NDRTs. Engagement (i.e., number of inputs on the NDRT interface) was assessed at the onset of a respective episode of automated driving (i.e., after transition) and during ongoing automation (i.e., before subsequent transition). Results revealed that over time, NDRT engagement increased during both L2 and L3 automation until stable engagement at the third block. This trend was observed for both onset and ongoing NDRT engagement. The overall engagement level and the increase in engagement are significantly stronger for L3 automation compared to L2 automation. These results outline the potential of NDRT engagement as an online non-intrusive measure for mode awareness. Moreover, repeated interaction is necessary until users are familiar with the automated system and its HMI to engage in NDRTs. These results provide researchers and practitioners with indications about users’ minimum degree of familiarity with driving automation and HMIs for mode awareness testing. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

22 pages, 2768 KiB  
Article
Supporting Drivers of Partially Automated Cars through an Adaptive Digital In-Car Tutor
by Anika Boelhouwer, Arie Paul van den Beukel, Mascha C. van der Voort, Willem B. Verwey and Marieke H. Martens
Information 2020, 11(4), 185; https://0-doi-org.brum.beds.ac.uk/10.3390/info11040185 - 30 Mar 2020
Cited by 14 | Viewed by 3994
Abstract
Drivers struggle to understand how, and when, to safely use their cars’ complex automated functions. Training is necessary but costly and time consuming. A Digital In-Car Tutor (DIT) is proposed to support drivers in learning about, and trying out, their car automation during [...] Read more.
Drivers struggle to understand how, and when, to safely use their cars’ complex automated functions. Training is necessary but costly and time consuming. A Digital In-Car Tutor (DIT) is proposed to support drivers in learning about, and trying out, their car automation during regular drives. During this driving simulator study, we investigated the effects of a DIT prototype on appropriate automation use and take-over quality. The study had three sessions, each containing multiple driving scenarios. Participants needed to use the automation when they thought that it was safe, and turn it off if it was not. The control group read an information brochure before driving, while the experiment group received the DIT during the first driving session. DIT users showed more correct automation use and a better take-over quality during the first driving session. The DIT especially reduced inappropriate reliance behaviour throughout all sessions. Users of the DIT did show some under-trust during the last driving session. Overall, the concept of a DIT shows potential as a low-cost and time-saving solution for safe guided learning in partially automated cars. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

19 pages, 1890 KiB  
Article
Standardized Test Procedure for External Human–Machine Interfaces of Automated Vehicles
by Christina Kaß, Stefanie Schoch, Frederik Naujoks, Sebastian Hergeth, Andreas Keinath and Alexandra Neukum
Information 2020, 11(3), 173; https://0-doi-org.brum.beds.ac.uk/10.3390/info11030173 - 24 Mar 2020
Cited by 29 | Viewed by 4703
Abstract
Research on external human–machine interfaces (eHMIs) has recently become a major area of interest in the field of human factors research on automated driving. The broad variety of methodological approaches renders the current state of research inconclusive and comparisons between interface designs impossible. [...] Read more.
Research on external human–machine interfaces (eHMIs) has recently become a major area of interest in the field of human factors research on automated driving. The broad variety of methodological approaches renders the current state of research inconclusive and comparisons between interface designs impossible. To date, there are no standardized test procedures to evaluate and compare different design variants of eHMIs with each other and with interactions without eHMIs. This article presents a standardized test procedure that enables the effective usability evaluation of eHMI design solutions. First, the test procedure provides a methodological approach to deduce relevant use cases for the evaluation of an eHMI. In addition, we define specific usability requirements that must be fulfilled by an eHMI to be effective, efficient, and satisfying. To prove whether an eHMI meets the defined requirements, we have developed a test protocol for the empirical evaluation of an eHMI with a participant study. The article elucidates underlying considerations and details of the test protocol that serves as framework to measure the behavior and subjective evaluations of non-automated road users when interacting with automated vehicles in an experimental setting. The standardized test procedure provides a useful framework for researchers and practitioners. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

12 pages, 518 KiB  
Article
The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios
by Marlene Susanne Lisa Scharfe, Kathrin Zeeb and Nele Russwinkel
Information 2020, 11(2), 115; https://0-doi-org.brum.beds.ac.uk/10.3390/info11020115 - 20 Feb 2020
Cited by 16 | Viewed by 3790
Abstract
In the development of highly automated driving systems (L3 and 4), much research has been done on the subject of driver takeover. Strong focus has been placed on the takeover quality. Previous research has shown that one of the main influencing factors is [...] Read more.
In the development of highly automated driving systems (L3 and 4), much research has been done on the subject of driver takeover. Strong focus has been placed on the takeover quality. Previous research has shown that one of the main influencing factors is the complexity of a traffic situation that has not been sufficiently addressed so far, as different approaches towards complexity exist. This paper differentiates between the objective complexity and the subjectively perceived complexity. In addition, the familiarity with a takeover situation is examined. Gold et al. show that repetition of takeover scenarios strongly influences the take-over performance. Yet, both complexity and familiarity have not been considered at the same time. Therefore, the aim of the present study is to examine the impact of objective complexity and familiarity on the subjectively perceived complexity and the resulting takeover quality. In a driving simulator study, participants are requested to take over vehicle control in an uncritical situation. Familiarity and objective complexity are varied by the number of surrounding vehicles and scenario repetitions. Subjective complexity is measured using the NASA-TLX; the takeover quality is gathered using the take-over controllability rating (TOC-Rating). The statistical evaluation results show that the parameters significantly influence the takeover quality. This is an important finding for the design of cognitive assistance systems for future highly automated and intelligent vehicles. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

20 pages, 2024 KiB  
Article
Repeated Usage of an L3 Motorway Chauffeur: Change of Evaluation and Usage
by Barbara Metz, Johanna Wörle, Michael Hanig, Marcus Schmitt and Aaron Lutz
Information 2020, 11(2), 114; https://0-doi-org.brum.beds.ac.uk/10.3390/info11020114 - 18 Feb 2020
Cited by 8 | Viewed by 3456
Abstract
Most studies on users’ perception of highly automated driving functions are focused on first contact/single usage. Nevertheless, it is expected that with repeated usage, acceptance and usage of automated driving functions might change this perception (behavioural adaptation). Changes can occur in drivers’ evaluation, [...] Read more.
Most studies on users’ perception of highly automated driving functions are focused on first contact/single usage. Nevertheless, it is expected that with repeated usage, acceptance and usage of automated driving functions might change this perception (behavioural adaptation). Changes can occur in drivers’ evaluation, in function usage and in drivers’ reactions to take-over situations. In a driving simulator study, N = 30 drivers used a level 3 (L3) automated driving function for motorways during six experimental sessions. They were free to activate/deactivate that system as they liked and to spend driving time on self-chosen side tasks. Results already show an increase of experienced trust and safety, together with an increase of time spent on side tasks between the first and fourth sessions. Furthermore, attention directed to the road decreases with growing experience with the system. The results are discussed with regard to the theory of behavioural adaptation. Results indicate that the adaptation of acceptance and usage of the highly automated driving function occurs rather quickly. At the same time, no behavioural adaptation for the reaction to take-over situations could be found. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

18 pages, 8148 KiB  
Article
External Human–Machine Interfaces: The Effect of Display Location on Crossing Intentions and Eye Movements
by Y. B. Eisma, S. van Bergen, S. M. ter Brake, M. T. T. Hensen, W. J. Tempelaar and J. C. F. de Winter
Information 2020, 11(1), 13; https://0-doi-org.brum.beds.ac.uk/10.3390/info11010013 - 24 Dec 2019
Cited by 63 | Viewed by 8124
Abstract
In the future, automated cars may feature external human–machine interfaces (eHMIs) to communicate relevant information to other road users. However, it is currently unknown where on the car the eHMI should be placed. In this study, 61 participants each viewed 36 animations of [...] Read more.
In the future, automated cars may feature external human–machine interfaces (eHMIs) to communicate relevant information to other road users. However, it is currently unknown where on the car the eHMI should be placed. In this study, 61 participants each viewed 36 animations of cars with eHMIs on either the roof, windscreen, grill, above the wheels, or a projection on the road. The eHMI showed ‘Waiting’ combined with a walking symbol 1.2 s before the car started to slow down, or ‘Driving’ while the car continued driving. Participants had to press and hold the spacebar when they felt it safe to cross. Results showed that, averaged over the period when the car approached and slowed down, the roof, windscreen, and grill eHMIs yielded the best performance (i.e., the highest spacebar press time). The projection and wheels eHMIs scored relatively poorly, yet still better than no eHMI. The wheels eHMI received a relatively high percentage of spacebar presses when the car appeared from a corner, a situation in which the roof, windscreen, and grill eHMIs were out of view. Eye-tracking analyses showed that the projection yielded dispersed eye movements, as participants scanned back and forth between the projection and the car. It is concluded that eHMIs should be presented on multiple sides of the car. A projection on the road is visually effortful for pedestrians, as it causes them to divide their attention between the projection and the car itself. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

18 pages, 2775 KiB  
Article
How Do eHMIs Affect Pedestrians’ Crossing Behavior? A Study Using a Head-Mounted Display Combined with a Motion Suit
by Lars Kooijman, Riender Happee and Joost C. F. de Winter
Information 2019, 10(12), 386; https://0-doi-org.brum.beds.ac.uk/10.3390/info10120386 - 06 Dec 2019
Cited by 40 | Viewed by 5612
Abstract
In future trac, automated vehicles may be equipped with external human-machine interfaces (eHMIs) that can communicate with pedestrians. Previous research suggests that, during first encounters, pedestrians regard text-based eHMIs as clearer than light-based eHMIs. However, in much of the previous research, pedestrians were [...] Read more.
In future trac, automated vehicles may be equipped with external human-machine interfaces (eHMIs) that can communicate with pedestrians. Previous research suggests that, during first encounters, pedestrians regard text-based eHMIs as clearer than light-based eHMIs. However, in much of the previous research, pedestrians were asked to imagine crossing the road, and unable or not allowed to do so. We investigated the e ects of eHMIs on participants’ crossing behavior. Twenty-four participants were immersed in a virtual urban environment using a head-mounted display coupled to a motion-tracking suit. We manipulated the approaching vehicles’ behavior (yielding, nonyielding) and eHMI type (None, Text, Front Brake Lights). Participants could cross the road whenever they felt safe enough to do so. The results showed that forward walking velocities, as recorded at the pelvis, were, on average, higher when an eHMI was present compared to no eHMI if the vehicle yielded. In nonyielding conditions, participants showed a slight forward motion and refrained from crossing. An analysis of participants’ thorax angle indicated rotation towards the approaching vehicles and subsequent rotation towards the crossing path. It is concluded that results obtained via a setup in which participants can cross the road are similar to results from survey studies, with eHMIs yielding a higher crossing intention compared to no eHMI. The motion suit allows investigating pedestrian behaviors related to bodily attention and hesitation. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

Review

Jump to: Editorial, Research, Other

15 pages, 431 KiB  
Review
Usability Evaluation—Advances in Experimental Design in the Context of Automated Driving Human–Machine Interfaces
by Deike Albers, Jonas Radlmayr, Alexandra Loew, Sebastian Hergeth, Frederik Naujoks, Andreas Keinath and Klaus Bengler
Information 2020, 11(5), 240; https://0-doi-org.brum.beds.ac.uk/10.3390/info11050240 - 28 Apr 2020
Cited by 14 | Viewed by 5110
Abstract
The projected introduction of conditional automated driving systems to the market has sparked multifaceted research on human–machine interfaces (HMIs) for such systems. By moderating the roles of the human driver and the driving automation system, the HMI is indispensable in avoiding side effects [...] Read more.
The projected introduction of conditional automated driving systems to the market has sparked multifaceted research on human–machine interfaces (HMIs) for such systems. By moderating the roles of the human driver and the driving automation system, the HMI is indispensable in avoiding side effects of automation such as mode confusion, misuse, and disuse. In addition to safety aspects, the usability of HMIs plays a vital role in improving the trust and acceptance of the automated driving system. This paper aggregates common research methods and findings based on an extensive literature review. Empirical studies, frameworks, and review articles are included. Findings and conclusions are presented with a focus on study characteristics such as test cases, dependent variables, testing environments, or participant samples. These methods and findings are discussed critically, taking into consideration requirements for usability assessments of HMIs in the context of conditional automated driving. The paper concludes with a derivation of recommended study characteristics framing best practice advice for the design of experiments. The advised selection of scenarios and metrics will be applied in a future validation study series comprising a driving simulator experiment and three real driving experiments on test tracks in Germany, the USA, and Japan. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

Other

16 pages, 3666 KiB  
Discussion
Checklist for Expert Evaluation of HMIs of Automated Vehicles—Discussions on Its Value and Adaptions of the Method within an Expert Workshop
by Nadja Schömig, Katharina Wiedemann, Sebastian Hergeth, Yannick Forster, Jeffrey Muttart, Alexander Eriksson, David Mitropoulos-Rundus, Kevin Grove, Josef Krems, Andreas Keinath, Alexandra Neukum and Frederik Naujoks
Information 2020, 11(4), 233; https://0-doi-org.brum.beds.ac.uk/10.3390/info11040233 - 24 Apr 2020
Cited by 11 | Viewed by 4508
Abstract
Within a workshop on evaluation methods for automated vehicles (AVs) at the Driving Assessment 2019 symposium in Santa Fe; New Mexico, a heuristic evaluation methodology that aims at supporting the development of human–machine interfaces (HMIs) for AVs was presented. The goal of the [...] Read more.
Within a workshop on evaluation methods for automated vehicles (AVs) at the Driving Assessment 2019 symposium in Santa Fe; New Mexico, a heuristic evaluation methodology that aims at supporting the development of human–machine interfaces (HMIs) for AVs was presented. The goal of the workshop was to bring together members of the human factors community to discuss the method and to further promote the development of HMI guidelines and assessment methods for the design of HMIs of automated driving systems (ADSs). The workshop included hands-on experience of rented series production partially automated vehicles, the application of the heuristic assessment method using a checklist, and intensive discussions about possible revisions of the checklist and the method itself. The aim of the paper is to summarize the results of the workshop, which will be used to further improve the checklist method and make the process available to the scientific community. The participants all had previous experience in HMI design of driver assistance systems, as well as development and evaluation methods. They brought valuable ideas into the discussion with regard to the overall value of the tool against the background of the intended application, concrete improvements of the checklist (e.g., categorization of items; checklist items that are currently perceived as missing or redundant in the checklist), when in the design process the tool should be applied, and improvements for the usability of the checklist. Full article
(This article belongs to the Special Issue Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-22
Back to TopTop