Multimodal Technol. Interact., Volume 5, Issue 4 (April 2021) – 7 articles

(1) Background: Primary driving tasks are increasingly being handled by vehicle automation so that support for non-driving related tasks (NDRTs) is becoming more and more important. In SAE L3 automation, vehicles can require the driver-passenger to take over driving controls, though. Interfaces for NDRTs must therefore guarantee safe operation and should also support productive work. (2) Method: We conducted a within-subjects driving simulator study ($N=53$) comparing Heads-Up Displays (HUDs) and Auditory Speech Displays (ASDs) for productive NDRT engagement. In this article, we assess the NDRT displays’ effectiveness by evaluating eye-tracking measures and setting them into relation to workload measures, self-ratings, and NDRT/take-over performance. (3) Results: Our data highlights substantially higher gaze dispersion but more extensive glances on the road center in the auditory condition than the HUD condition during automated driving. We further observed potentially safety-critical glance deviations from the road during take-overs after a HUD was used. These differences are reflected in self-ratings, workload indicators and take-over reaction times, but not in driving performance. (4) Conclusion: NDRT interfaces can influence visual attention even beyond their usage during automated driving. In particular, the HUD has resulted in safety-critical glances during manual driving after take-overs. We found this impacted workload and productivity but not driving performance. Full article

Notions of the smart city look to mobilize information technology to increase organizational efficiency, and more recently, to support new forms of community engagement and involvement in addressing municipal issues. As cities turn to civic enterprise technology platforms, we need to better understand how that class of system might be positioned and used to collaborate with informal community-born coalitions. Beginning in 2019, we undertook an embedded collaborative research project in Albany Georgia, a small rural city, to understand three primary research questions: (1) How do community organizing practices take shape around joint initiatives with local government? (2) What data, tools, and process are needed to support those initiatives? (3) How do the affordances of City-run enterprise platforms support such community-born initiatives? To develop insight into these questions, we deployed a mixed-methods study that interwove participant observation, qualitative fieldwork, and participatory workshops. From this, we point to several mismatches that arose between the assumptions of a managed enterprise environment and the complex needs of establishing and supporting a multiparty community coalition. Full article

We report on a conceptual framework for describing interactive sound installations from three complementary perspectives: artistic intention, interaction and system design. Its elaboration was informed by a systematic review of 181 peer-reviewed publications retrieved from the Scopus database, which describe 195 interactive sound installations. The resulting taxonomy is based on the comparison of the different facets of the installations reported in the literature and on existing frameworks, and it was used to characterize all publications. A visualization tool was developed to explore the different facets and identify trends and gaps in the literature. The main findings are presented in terms of bibliometric analysis, and from the three perspectives considered. Various trends were derived from the database, among which we found that interactive sound installations are of prominent interest in the field of computer science. Furthermore, most installations described in the corpus consist of prototypes or belong to exhibitions, output two sensory modalities and include three or more sound sources. Beyond the trends, this review highlights a wide range of practices and a great variety of approaches to the design of interactive sound installations. Full article

This paper describes a virtual field trip application as a new type of immersive, multimodal, interactive, data visualization of a virtual arboretum. Deployed in a game engine, it is a large, open-world simulation, representing 100 hectares and ideal for use when free choice in navigation and high fidelity are required. Although the computer graphics are photorealistic, it is different and unique from other applications that use game art or 2D 360-degree video, because it reflects high information fidelity as a result of the domain expert review, and the integration of geographic information system (GIS) data with drone images. Combined in-game as a data visualization, it is ideal for generating past or future worlds, in addition to representations of the present. Fusing information from many data sources—terrain data, waterbody data, plant inventory, population density data, accurate plant models, bioacoustics, and drone images—its design process and methods could be repeated and used in a wide range of augmented reality (AR) and virtual reality (VR) applications and devices. Results on presence, embodiment, emotions, engagement, and learning are summarized from prior pilot studies for context on use, and are relevant to schools, museums, arboretums, and botanical gardens interested in developing immersive informal learning applications. Full article

This paper aims to explore the impact of a collective immersion on learners’ engagement and performance. Building on Bandura’s social learning theory and the theory on the sense of presence, we hypothesise that collective immersion has a positive impact on performance as well as cognitive, emotional and behavioural engagement. Ninety-three participants distributed in four conditions took part in the experiment. The four conditions manipulated the collective and individual dimensions of the learning environment as well as the high and low immersion of the learning material. The two conditions that offered a high immersion setting used two types of the novel immersive dome: a large one for collective immersion and a small one for individual use. All participants were presented with the same stimuli, an 8-min-long video of a virtual neighbourhood visit in Paris in the 18th century. The participants’ reactions were measured during and after the task. The learning outcome, as well as the cognitive, emotional and behavioural engagement, were measured. Final results showed that collective immersion learning outcomes are not significantly different, but we find that collective immersion impacts the cognitive, emotional and behavioural engagement of learners. Full article

Context-adaptive functions are not new in the driving context, but even so, investigations into these functions concerning the automation human–machine interface (aHMI) have yet to be carried out. This study presents research into context-adaptive availability notifications for an SAE Level 3 automation in scenarios where participants were surprised by either availability or non-availability. For this purpose, participants (N = 30) took part in a driving simulator study, experiencing a baseline HMI concept as a comparison, and a context-adaptive HMI concept that provided context-adaptive availability notifications with the aim of improving acceptance and usability, while decreasing frustration (due to unexpected non-availability) and gaze deviation from the road when driving manually. Furthermore, it was hypothesized that participants, when experiencing the context-adaptive HMI, would activate the automated driving function more quickly when facing unexpected availability. None of the hypotheses could be statistically confirmed; indeed, where gaze behavior was concerned, the opposite effects were found, indicating increased distraction induced by the context-adaptive HMI. However, the trend in respect to the activation time was towards shorter times with the context-adaptive notifications. These results led to the conclusion that context-adaptive availability notifications might not always be beneficial for users, while more salient availability notifications in the case of an unexpected availability could be advantageous. Full article

Ubiquitous technology lets us work in flexible and decentralised ways. Passengers can already use travel time to be productive, and we envision even better performance and experience in vehicles with emerging technologies, such as virtual reality (VR) headsets. However, the confined physical space constrains interactions while the virtual space may be conceptually borderless. We therefore conducted a VR study (N = 33) to examine the influence of physical restraints and virtual working environments on performance, presence, and the feeling of safety. Our findings show that virtual borders make passengers touch the car interior less, while performance and presence are comparable across conditions. Although passengers prefer a secluded and unlimited virtual environment (nature), they are more productive in a shared and limited one (office). We further discuss choices for virtual borders and environments, social experience, and safety responsiveness. Our work highlights opportunities and challenges for future research and design of rear-seat VR interaction. Full article