Multimodal Technol. Interact., Volume 5, Issue 9 (September 2021) – 8 articles

Conversational agents offer promise to provide an alternative to costly and scarce access to human health providers. Particularly in the context of adherence to treatment advice and health behavior change, they can provide an ongoing coaching role to motivate and keep the health consumer on track. Due to the recognized importance of face-to-face communication and establishment of a therapist-patient working alliance as the biggest single predictor of adherence, our review focuses on embodied conversational agents (ECAs) and their use in health and well-being interventions. The article also introduces ECAs who provide explanations of their recommendations, known as explainable agents (XAs), as a way to build trust and enhance the working alliance towards improved behavior change. Of particular promise, is work in which XAs are able to engage in conversation to learn about their user and personalize their recommendations based on their knowledge of the user and then tailor their explanations to the beliefs and goals of the user to increase relevancy and motivation and address possible barriers to increase intention to perform the healthy behavior. Full article

Exercise is a prevailing topic in modern society as more people are pursuing a healthy lifestyle. Physical activities provide significant benefits to human well-being from the inside out. Human pose estimation, action recognition and repetitive counting fields developed rapidly in the past several years. However, few works combined them together to assist people in exercise. In this paper, we propose a multitask system covering the three domains. Different from existing methods, heatmaps, which are the byproducts of 2D human pose estimation models, are adopted for exercise recognition and counting. Recent heatmap processing methods have been proven effective in extracting dynamic body pose information. Inspired by this, we propose a deep-learning multitask model of exercise recognition and repetition counting. To the best of our knowledge, this approach is attempted for the first time. To meet the needs of the multitask model, we create a new dataset Rep-Penn with action, counting and speed labels. Our multitask system can estimate human pose, identify physical activities and count repeated motions. We achieved 95.69% accuracy in exercise recognition on the Rep-Penn dataset. The multitask model also performed well in repetitive counting with 0.004 Mean Average Error (MAE) and 0.997 Off-By-One (OBO) accuracy on the Rep-Penn dataset. Compared with existing frameworks, our method obtained state-of-the-art results. Full article

We introduce the first study of the automatic detoxification of Russian texts to combat offensive language. This kind of textual style transfer can be used for processing toxic content on social media or for eliminating toxicity in automatically generated texts. While much work has been done for the English language in this field, there are no works on detoxification for the Russian language. We suggest two types of models—an approach based on BERT architecture that performs local corrections and a supervised approach based on a pretrained GPT-2 language model. We compare these methods with several baselines. In addition, we provide the training datasets and describe the evaluation setup and metrics for automatic and manual evaluation. The results show that the tested approaches can be successfully used for detoxification, although there is room for improvement. Full article

Multidisciplinary topics in education pose a major challenge for traditional learning and teaching methods. Such topics can deter students from selecting particular courses or hinder their study progress. This study focused on the subject of medicinal chemistry, which is a discipline combining medicine and chemistry. This combination of applied and basic science creates a complex field of education that is challenging to both teach and learn. Chemical and pharmacological principles are typically presented in 2D molecular structures and, recently, 3D molecular models have been utilized to improve the visualization of chemical compounds and their chemical interactions. Contemporary studies have presented Virtual Reality (VR) as an alternative method for improving the learning and teaching of multidisciplinary specialties such as this. However, current educational efforts employing VR offer limited interactivity and a traditional teaching method previously presented in 2D. This reduces students’ interest and concentration in the taught subjects. This paper presents the development rationale of a novel VR educational application based on the evaluation of the user requirements by 405 pharmacy undergraduate students. The results informed the development and preliminary evaluation of a proposed VR serious game application, which was deployed in a real-life class environment and evaluated in contrast to traditional teaching methods by 15 students. The derived results confirmed the advantages of VR technology as a learning and teaching tool, in addition to the end-users’ willingness to adopt VR systems as a learning aid. Full article

Life expectancy has increased in recent years. Physical activity has been postulated as a key element in active aging strategies. However, adherence to physical exercise programs has traditionally been low among the elderly, and the current situation of the COVID-19 pandemic has added extra impediments. Immersive virtual reality (IVR) devices could motivate this population to practice exercise. This study aimed to analyse the use of IVR exergames as a tool to facilitate physical exercise in older people. Four healthy older adults (males, 65–77 years) participated in the study. They carried out two exergaming sessions with HTC Vive Pro^TM. Outcomes were evaluated using the Simulator Sickness Questionnaire (SSQ), System Usability Scale (SUS), Game Experience Questionnaire (GEQ post-game module), an ad hoc satisfaction questionnaire, and perceived effort. All participants completed the sessions without adverse effects, with no SSQ symptoms reported. SUS scores were high in both sessions (SUS > 85/100). Post-game GEQ scores were 3.08–3.41/4 (positive experiences) and 0.08–0.16/4 (negative experiences). Opinions showed high levels of satisfaction with the experience. Exergaming programs, based on commercial head-mounted displays, are a feasible alternative to traditional senior exercise, and could be a solution to the current situation that has arisen from the impact of the COVID-19 pandemic. Full article

Automated vehicles do not yet have clearly defined signaling methods towards other road users, which could complement natural communication practices with human drivers, such as eye contact or hand gestures. In order to establish trust, external human–machine interfaces (eHMIs) have been proposed, but so far, these have not been widely evaluated in natural traffic contexts. This paper presents a user study where 30 participants interacted with a functional display-based visual eHMI for an automated shuttle in mixed urban traffic. Two distinct features were investigated: the communication of (1) its awareness of different obstacles on the road ahead and (2) of its intention to start or to brake. The results indicate that the majority of participants in general regarded eHMIs as necessary for automated vehicles. When reflecting their experience with the eHMIs, about half of the participants experienced an increased comprehension and safety. The combined presentation of obstacle awareness and vehicle intentions helped more participants to understand the shuttle’s behavior than the presentation of obstacle awareness only, but fewer participants regarded this combination of awareness and intent to be safe. The strength of the found effects on subjective responses varied with regard to age and gender. Full article

This paper presents a practical human-computer interaction system for wheelchair motion through eye tracking and eye blink detection. In this system, the pupil in the eye image has been extracted after binarization, and the center of the pupil was localized to capture the trajectory of eye movement and determine the direction of eye gaze. Meanwhile, convolutional neural networks for feature extraction and classification of open-eye and closed-eye images have been built, and machine learning was performed by extracting features from multiple individual images of open-eye and closed-eye states for input to the system. As an application of this human-computer interaction control system, experimental validation was carried out on a modified wheelchair and the proposed method proved to be effective and reliable based on the experimental results. Full article

HRI designers are faced with the task of creating robots that are easy and pleasant to use for the users. The growing body of research in human–robot interaction (HRI) is still mainly focused on technical aspects of the interaction. It lacks defined guidelines that describe how behavioral expressions for social robots need to be designed to promote high usability and positive user experience. To achieve this goal, we propose to apply the concept of design patterns to HRI. We present a design process that provides step-by-step guidance and methods for HRI designers to generate high quality behavioral patterns for social robots that can be used for different robots and use cases. To document the resulting patterns, we developed a documentation format that provides a clear, standardized structure to note down all relevant aspects of a pattern so that others can understand its design recommendations and apply them to their own robot and use cases. In the present paper, we demonstrate our pattern approach based on an example and describe how we arrived at a pattern language of 40 behavioral patterns that found the basis for future social robot design and related research activities. Full article