Multimodal Technol. Interact., Volume 5, Issue 3 (March 2021) – 8 articles

This paper presents a research-through-design project to develop and interpret speculative smart home technologies for cohousing communities—Cohousing IoT. Fieldwork at multiple sites coupled to a constructive design research process led to three prototypes designed for cohousing communities: Cohousing Radio, Physical RSVP, and Participation Scales. These were brought back to the communities that inspired them as a form of evaluation, but also to generate new understandings of designing for cohousing. In discussing how they understand these prototypes, this paper offers an account of how research though design generates knowledge that is specific to the conditions and issues that matter to communities. This contributes to design research more broadly in two ways. First, it demonstrates how contemporary ideas of smart home technology are or could be made relevant to broader ways of living in the future. Second, it provides an example of how a design research process can serve to uncover community values, issues, and goals. Full article

The role of affect has been long studied in human–computer interactions. Unlike previous studies that focused on seven basic emotions, an avatar named Diana was introduced who expresses a higher level of emotional intelligence. To adapt to the users various affects during interaction, Diana simulates emotions with dynamic facial expressions. When two people collaborated to build blocks, their affects were recognized and labeled using the Affdex SDK and a descriptive analysis was provided. When participants turned to collaborate with Diana, their subjective responses were collected and the length of completion was recorded. Three modes of Diana were involved: a flat-faced Diana, a Diana that used mimicry facial expressions, and a Diana that used emotionally responsive facial expressions. Twenty-one responses were collected through a five-point Likert scale questionnaire and the NASA TLX. Results from questionnaires were not statistically different. However, the emotionally responsive Diana obtained more positive responses, and people spent the longest time with the mimicry Diana. In post-study comments, most participants perceived facial expressions on Diana’s face as natural, four mentioned uncomfortable feelings caused by the Uncanny Valley effect. Full article

By performing motor-imagery tasks, for example, imagining hand movements, Motor-Imagery based Brain-Computer Interfaces (MI-BCIs) users can control digital technologies, for example, neuroprosthesis, using their brain activity only. MI-BCI users need to train, usually using a unimodal visual feedback, to produce brain activity patterns that are recognizable by the system. The literature indicates that multimodal vibrotactile and visual feedback is more effective than unimodal visual feedback, at least for short term training. However, the multi-session influence of such multimodal feedback on MI-BCI user training remained unknown, so did the influence of the order of presentation of the feedback modalities. In our experiment, 16 participants trained to control a MI-BCI during five sessions with a realistic visual feedback and five others with both a realistic visual feedback and a vibrotactile one. training benefits from a multimodal feedback, in terms of performances and self-reported mindfulness. There is also a significant influence of the order presentation of the modality. Participants who started training with a visual feedback had higher performances than those who started training with a multimodal feedback. We recommend taking into account the order of presentation for future experiments assessing the influence of several modalities of feedback. Full article

Autism affects how people perceive and make sense of the world around them. Autism is a spectrum condition which impacts people in different ways. Also referred to as Autism Spectrum Disorder (ASD), it is characterized by challenges in the domains of social, cognitive and motor functioning, which differ in severity. Previous research suggests that music can have cognitive, psychosocial, behavioural, and motor benefits in this population. We systematically review the use of technology in Music-therapy and related sound-based activities to improve the motor and social skills of children. In May 2020 we conducted a systematic search on Music-therapy and musical activities for autistic children in research databases including Science Direct, APA PsycNet, Cochrane, IEE and Web of Science, to collect relevant studies. We initially collected 5179 papers of which only 27 studies were identified as suitable for the scope of this review. In the paper, we analyse and describe key characteristics of each project. We then highlight the commonalities, strengths and limitations of existing work, and identify implications for future interaction design. Full article

Medicinal chemistry (MC) is an indispensable component of the pharmacy curriculum. The pharmacists’ unique knowledge of a medicine’s chemistry enhances their understanding of the pharmacological activity, manufacturing, storage, use, supply, and handling of drugs. However, chemistry is a challenging subject for both teaching and learning. These challenges are typically caused by the inability of students to construct a mental image of the three-dimensional (3D) structure of a drug molecule from its two-dimensional presentations. This study explores a prototype virtual reality (VR) gamification option, as an educational tool developed to aid the learning process and to improve the delivery of the MC subject to students. The developed system is evaluated by a cohort of 41 students. The analysis of the results was encouraging and provided invaluable feedback for the future development of the proposed system. Full article

The deployment of augmented reality (AR) has attracted educators’ interest and introduced new opportunities in education. Additionally, the advancement of artificial intelligence has enabled educational researchers to apply innovative methods and techniques for the monitoring and evaluation of the teaching and learning process. The so-called learning analytics (LA) discipline emerged with the promise to revolutionize traditional instructional practices by introducing systematic and multidimensional ways to improve the effectiveness of the instructional process. However, the implementation of LA methods is usually associated with web-based platforms, which offer direct access to learners’ data with minimal effort or adjustments. On the other hand, the complex nature of immersive technologies and the diverse instructional approaches which are utilized in different scientific domains have limited the opportunities for research and development in this direction. Within these research contexts, we present a conceptual framework that describes the elements of an LA process tailored to the information that can be gathered from the use of educational applications, and further provide an indicative case study for AR-supported educational interventions. The current work contributes by elucidating and concretizing the design elements of AR-supported applications and provides researchers and designers with guidelines on how to apply instructional strategies in (augmented) real-world projects. Full article

Serious Games (SGs) could be enriched with collaborative learning techniques, an approach that has the potential to increase the players’ gaming and learning experience. However, the design of Multiplayer Serious Games (MSGs) with science content needs to be facilitated with methods and tools that provide coherent designs. This research proposes a methodology that employs the design technique of personas to effectively describe the multiplayer design patterns and the game narrative according to the content of a science school book and, at the same time, balancing the educational and the entertaining character of a SG. We evaluated the suggested methodology with two groups of professionals who were assigned with the task to design a MSG, integrating the same educational content according to the suggested methodology. The designs were qualitatively examined, confirming the potential of the Multiplayer Serious Game Methodology (MSGM) to facilitate the science content integration. We expect that the MSGM could assist groups of professionals, such as teachers and developers to create coherent MSG designs. Full article

Many studies have attempted to understand which individual differences may be related to the symptoms of discomfort during the virtual experience (simulator sickness) and the generally considered positive sense of being inside the simulated scene (sense of presence). Nevertheless, a very limited number of studies have employed modern consumer-oriented head-mounted displays (HMDs). These systems aim to produce a high the sense of the presence of the user, remove stimuli from the external environment, and provide high definition, photo-realistic, three-dimensional images. Our results showed that motion sickness susceptibility and simulator sickness are related, and neuroticism may be associated and predict simulator sickness. Furthermore, the results showed that people who are more used to playing videogames are less susceptible to simulator sickness; female participants reported more simulator sickness compared to males (but only for nausea-related symptoms). Female participants also experienced a higher sense of presence compared to males. We suggest that published findings on simulator sickness and the sense of presence in virtual reality environments need to be replicated with the use of modern HMDs. Full article