Multimodal Technol. Interact., Volume 5, Issue 10 (October 2021) – 8 articles

This paper describes an overview of olfactory displays (human–computer interfaces that generate and diffuse an odor to a user to stimulate their sense of smell) that have been proposed and researched for supporting education and training. Past research has shown that olfaction (the sense of smell) can support memorization of information, stimulate information recall, and help immerse learners and trainees into educational virtual environments, as well as complement and/or supplement other human sensory channels for learning. This paper begins with an introduction to olfaction and olfactory displays, and a review of techniques for storing, generating and diffusing odors at the computer interface. The paper proceeds with a discussion on educational theories that support olfactory displays for education and training, and a literature review on olfactory displays that support learning and training. Finally, the paper summarizes the advantages and challenges regarding the development and application of olfactory displays for education and training. Full article

The increasing phenomenon of “cyberhate” is concerning because of the potential social implications of this form of verbal violence, which is aimed at already-stigmatized social groups. According to information collected by the Ministry of the Interior of Spain, the category of sexual orientation and gender identity is subject to the third-highest number of registered hate crimes, ranking behind racism/xenophobia and ideology. However, most of the existing computational approaches to online hate detection simultaneously attempt to address all types of discrimination, leading to weaker prototype performances. These approaches focus on other reasons for hate—primarily racism and xenophobia—and usually focus on English messages. Furthermore, few detection models have used manually generated databases as a training corpus. Using supervised machine learning techniques, the present research sought to overcome these limitations by developing and evaluating an automatic detector of hate speech motivated by gender and sexual orientation. The focus was Spanish-language posts on Twitter. For this purpose, eight predictive models were developed from an ad hoc generated training corpus, using shallow modeling and deep learning. The evaluation metrics showed that the deep learning algorithm performed significantly better than the shallow modeling algorithms, and logistic regression yielded the best performance of the shallow algorithms. Full article

(1) COVID-19 pandemic restrictions caused a dramatic shift in research activities, forcing the adoption of remote practices and methods. Despite the known benefits of remote testing, there is limited knowledge on how to prepare and conduct such studies in the industrial context where the target users are experts and company employees. (2) In this article, we detail how we organized VR user tests in five industrial cases during the pandemic, focusing on practicalities and procedures. We cover both on-site testing, including disinfecting and other safety protocols, as well as remote and hybrid setups where both remote and on-site participants were involved. Subject matter experts from eight countries were involved in a total of 22 tests. (3) We share insights for VR user test arrangements relevant to the pandemic, remote and hybrid setups, and an industrial context, among others. (4) Our work confirms that with careful planning it is possible to organize user tests remotely. There are also some limitations in remote user testing, such as reduced visibility and interaction with participants. Most importantly, we list practical recommendations for organizing hybrid user tests with safety and disinfecting procedures for on-site VR use. Full article

As a precursor to future public transportation, automated shuttle buses can already be experienced in some test regions, but the general public still has reservations and may not yet be ready for this change. For example, the fact that such vehicles might operate independently (without a human driver) creates a barrier of uncertainty and mistrust among people. In this work, we aim to identify and classify the prevailing reservations and propose solutions. We followed the User Centered Design (UCD) process to design concepts that are specifically tailored to the needs of future public transport users. After related work analysis, on-site research, and pre-studies, two main studies were conducted specifically to address communication in the exterior (n = 24) and interior/service design (n = 21). For both studies, we applied a mixed-methods approach combining quantitative and qualitative measures. Our results indicate that, in general, existing ways of communication in the exterior are insufficient to meet future needs. The two visualization concepts for external communication developed in this work were rated (significantly) better in most dimensions of the User Experience Questionnaire (UEQ), when compared to the baseline condition with no additional visualization. Furthermore, preferences among the study participants towards simple, highly visible, and well-known lighting concepts could be observed. As for the interior, the results show that participants rated attractiveness highly for the two design concepts (closer, further in the future) as compared to current, state-of-the-art solutions (automated buses currently in operation). For the “near future” concept, the pragmatic quality dominated, while in the other (the “far future”) concept the hedonic quality was in the foreground. From the results, design recommendations in different categories were derived, which reflect the general openness of the public towards new technologies and interior approaches, but also point out the importance for privacy and designated personal spaces inside an (automated) shuttle bus. Some of the results do not strictly apply to automated shuttle buses, and can serve as valuable suggestions for improving conventional shuttle buses. Full article

Augmenting online interpersonal communication with biosignals, often in the form of heart rate sharing, has shown promise in increasing affiliation, feelings of closeness, and intimacy. Increasing empathetic awareness in the professional domain and in the customer interface could benefit both customer and employee satisfaction, but heart rate sharing in this context needs to consider issues around physiological monitoring of employees, appropriate level of intimacy, as well as the productivity outlook. In this study, we explore heart rate sharing at the workplace and study its effects on task performance. Altogether, 124 participants completed a collaborative visual guidance task using a chat box with heart rate visualization. Participants’ feedback about heart rate sharing reveal themes such as a stronger sense of human contact and increased self-reflection, but also raise concerns around unnecessity, intimacy, privacy and negative interpretations. Live heart rate was always measured, but to investigate the effect of heart rate sharing on task performance, half of the customers were told that they were seeing a recording, and half were told that they were seeing the advisor’s live heart beat. We found a negative link between awareness and task performance. We also found that higher ratings of usefulness of the heart rate visualization were associated with increased feelings of closeness. These results reveal that intimacy and privacy issues are particularly important for heart rate sharing in professional contexts, that preference modulates the effects of heart rate sharing on social closeness, and that heart rate sharing may have a negative effect on performance. Full article

Understanding the behavioral dynamics that underline human-robot interactions in groups remains one of the core challenges in social robotics research. However, despite a growing interest in this topic, there is still a lack of established and validated measures that allow researchers to analyze human-robot interactions in group scenarios; and very few that have been developed and tested specifically for research conducted in-the-wild. This is a problem because it hinders the development of general models of human-robot interaction, and makes the comprehension of the inner workings of the relational dynamics between humans and robots, in group contexts, significantly more difficult. In this paper, we aim to provide a reflection on the current state of research on human-robot interaction in small groups, as well as to outline directions for future research with an emphasis on methodological and transversal issues. Full article

In emergency contexts such as the COVID-19 pandemic, mental health risk factors affect children and may affect behavioral and emotional problems including anxiety, self-blame, emotional disturbance, and stress. Preventive measures are crucial to address these health risks. Research highlights strength-based factors, such as socio-emotional skills and prosocial behaviors, as important for childhood development and socio-emotional wellbeing. Yet, the initial evidence base shows mixed effectiveness and insufficient behavior change theory application into socio-emotional and prosocial digital game interventions. Additionally, few interventions are designed with a clear process to convert theory into practical game solutions and very limited guidance on the digital game development process exists. This study describes the Intervention Mapping Protocol (IMP) and theory application in the design of a digital game intervention which aims to strengthen social-emotional skills development and promote prosocial behavior in 8–11-year-old children. The method systematically describes the steps of the IMP process in detail to guide future theory-based game design. The results explain the final digital game prototype that was co-designed with continuous input and insights provided by stakeholders and feedback from children. This paper contributes to our understanding of theory application in the design of digital game interventions focused on health and behavior change and provides much needed guidance on how theory and stakeholder input can be incorporated into a final game design. Full article

Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind^® robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel^® RealSense™ 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills. Full article