Multimodal Technol. Interact., Volume 6, Issue 5 (May 2022) – 11 articles

For many living in rural areas, the loss of traditional community assets and increased social fragmentation are a common feature of everyday life. The empty village church is a poignant symbol of these challenges; yet, these are sites that hold considerable potential for new placemaking solutions that respond to the needs of communities today. This means looking beyond “the traditional village church” to recognise a longer history of church adaptation and resilience within the lives of communities. In this paper we ask: how can co-design, projected through a Wicked problems and Clumsy solutions lens, help imagine new futures for communities and their historic churches today? Clumsy solutions consider a plurality of different perspectives on the nature of problems and their resolution to deliver more effective solutions with broad appeal. In the search for clumsiness, we turn to ‘long history’ and ‘slow technology’ for inspiration, uncovering deeper resonance with historical communities of place and anchoring that continuity within church sites themselves. Our paper demonstrates how Wicked/Clumsy thinking can account for the challenges faced by rural communities today, bootstrap co-design activities in the development of clumsy solutions, and uncover clumsiness in long history and slow technology dimensions—together laying the foundation for new placemaking strategies. Full article

We examined the influence of a parent on robot-assisted activities for a child with Autism Spectrum Disorder. We observed the interactions between a robot and the child wearing a wearable device during free play sessions. The child participated in four sessions with the parent and interacted willingly with the robot, therapist, and parent. The parent intervened when the child did not interact with the robot, considered “disengagement with the robot”. The number and method of intervention were decided solely by the parent. This study adopted video recording for behavioral observations and specifically observed the situations before the disengagement with the robot, the child’s behaviors during disengagement, and the parent’s intervention. The results showed that mostly the child abruptly discontinued the interactions with the robot without being stimulated by the surrounding environment. The second most common reason was being distracted by various devices in the play sessions, such as the wearable device, a video camera, and a laptop. Once he was disengaged with the robot, he primarily exhibited inappropriate and repetitive behaviors accentuating the symptoms of autism spectrum disorder. The child could re-initiate the interaction with the robot with an 80% chance through the parent’s intervention. This suggests that engagement with a robot may differ depending on the parent’s participation. Moreover, we must consider types of parental feedback to re-initiate engagement with a robot to benefit from the therapy adequately. In addition, environmental distractions must be considered, especially when using multiple devices for therapy. Full article

This paper presents quantitative research on the perception of the didactic use of virtual reality by university professors in Colombia, with special attention to the differences according to their area of knowledge, as the main variable, and gender and digital generation, as secondary variables. The study involved 204 professors from different Colombian universities. As an instrument, a survey designed for this purpose was used with four scales that were used to measure, on a Likert scale, different dimensions involving the participants’ perception of the use of virtual reality in the classroom. The answers were analyzed statistically and the differences in the perceptions have been identified by means of parametric statistical tests according to the following: (i) area of knowledge, (ii) gender, (iii) digital generation of the participants. The results showed that the participants expressed high valuations of virtual reality, despite having intermediate or low levels of digital competence. Gaps were identified in terms of area of knowledge, gender, and digital generation (digital natives or immigrants) with respect to opinions of virtual reality and digital competence. The highest valuations of virtual reality are given by professors of Humanities, and by digital natives. It is suggested that Colombian universities implement training plans on digital competence for professors and that these plans be aimed at strengthening knowledge of virtual reality. Full article

A web-based authoring platform for the representation of traditional crafts is proposed. This platform is rooted in a systematic method for craft representation, the adoption, knowledge, and representation standards of the cultural heritage (CH) domain, and the integration of outcomes from advanced digitization techniques. In this paper, we present the implementation of this method by an online, collaborative documentation platform where digital assets are curated into digitally preservable craft representations. The approach is demonstrated through the representation of three traditional crafts as use cases, and the lessons learned from this endeavor are presented. Full article

The design of location-based games (LBGs) for cultural heritage should ensure the active participation and contribution of local communities and heritage professionals to achieve contextual relevance, importance, and content validity. This paper presents an approach and methods of the participatory and co-design of LBGs that promote awareness and learning about the intangible cultural heritage of craftsmanship and artisanal technology throughout a long-term project from sensitization to implementation. Following the design thinking process, we outline the participatory methods (and reflect on results and lessons learnt) of involving cultural heritage professionals, local communities, and visitors (users) of museums and cultural settlements, mainly: field visits, design workshops, field playtesting, and field studies. We discuss issues of participatory design that we experienced throughout the project such as participant centrality and representativeness, producing tangible output from meetings, co-creation of content via playtesting, and implications from the pandemic. This work contributes a case of participatory and co-design of LBGs for cultural heritage that is characterized by longevity and engagement throughout the design process for three LBGs of a museum network in different cultural sites. Full article

Music elicits strong emotional reactions in people, regardless of their gender, age or cultural background. Understanding the effects of music on brain activity can enhance existing music therapy techniques and lead to improvements in various medical and affective computing research. We explore the effects of three different music genres on people’s cerebral hemodynamic responses. Functional near-infrared spectroscopy (fNIRS) signals were collected from 27 participants while they listened to 12 different pieces of music. The signals were pre-processed to reflect oxyhemoglobin (HbO2) and deoxyhemoglobin (HbR) concentrations in the brain. K-nearest neighbor (KNN), random forest (RF) and a one-dimensional (1D) convolutional neural network (CNN) were used to classify the signals using music genre and subjective responses provided by the participants as labels. Results from this study show that the highest accuracy in distinguishing three music genres was achieved by deep learning models (73.4% accuracy in music genre classification and 80.5% accuracy when predicting participants’ subjective rating of emotional content of music). This study validates a strong motivation for using fNIRS signals to detect people’s emotional state while listening to music. It could also be beneficial in giving personalised music recommendations based on people’s brain activity to improve their emotional well-being. Full article

The challenges to implement digital technologies in community-based projects are exposed in a case study co-designing an indigenous Community Museum, situated in the Kelabit Highlands of Borneo, Malaysia. Over a five-year period, this co-design project consisted of field trips, community engagements, and creating a documentary film and an inaugural exhibition in the newly constructed Kelabit Museum. This article highlights the limitations of digital technologies in museum contexts. Co-designing with stakeholders resulted in the decision to take a non-digital approach to the museum development to encourage greater community agency and prevent disengagement, as it incorporated heritage values in local community developments and cultural tourism plans. The findings demonstrate that community self-determination conflicted with preconceived outcomes, resulting in a need to re-evaluate the goals of the project. Instead, the ambition of cultural heritage preservation that maintained community participation emerged as the central goal. Removing the focus on a digital solution expanded community participation, which is a finding that should be used to frame other community cultural developments. Full article

Over the last decades, a shift towards participatory approaches could be observed in cultural heritage institutions. In co-curation processes, museums collaborate with public audiences to identify, select, prepare, and interpret cultural materials. This article focuses on the question how to engage and motivate local communities or individuals in rethinking dominant discourses or expert narratives regarding cultural heritage and bringing in their own experiences and knowledge. Based on four case studies of cultural co-curation, we delineate two basic challenges for this process: (1) Authority—even though museums strive to involve the public, there is still an imbalance in participation due to the museums’ authoritative status. (2) Motivation—participation in co-curation processes requires high levels of motivation, which are difficult to achieve. Based on the media synchronicity theory, we discuss which characteristics of new media technologies can be helpful to overcome these challenges. Media can increase awareness on counternarratives and blind spots in cultural collections. They can provide a setting where the participants can easily contribute, feel competent to do so, are empowered to rethink dominant discourses, develop a sense of relatedness with other contributors, and maintain autonomy in how and to which degree they engage in the discourse. Full article

In this paper we apply our understanding of the radical enactivist agenda to the classic AI-hard problem of Natural Language Understanding. When Turing devised his famous test the assumption was that a computer could use language and the challenge would be to mimic human intelligence. It turned out playing chess and formal logic were easy compared to understanding what people say. The techniques of good old-fashioned AI (GOFAI) assume symbolic representation is the core of reasoning and by that paradigm human communication consists of transferring representations from one mind to another. However, one finds that representations appear in another’s mind without appearing in the intermediary language. People communicate by mind reading it seems. Systems with speech interfaces such as Alexa and Siri are of course common, but they are limited. Rather than adding mind reading skills, we introduced a “cheat” that enabled our systems to fake it. The cheat is simple and only slightly interesting to computer scientists and not at all interesting to philosophers. However, reading about the enactivist idea that we “directly perceive” the intentions of others, our cheat took on a new light and in this paper we look again at how natural language understanding might actually work between humans. Full article

Following the advances in modern head-mounted displays, research exploring the human experience of virtual environments has seen a surge in interest. Researchers have examined how to promote individuals’ sense of presence, i.e., their experience of “being” in the VE, as well as to diminish the negative side effects of cybersickness. Studies investigating the relationship between sense of presence and cybersickness have reported heterogeneous results. Authors that found a positive relation have argued that the phenomena have shared cognitive underpinnings. However, recent literature has reported that positive associations can be explained by the confounding factor of immersion. The current study aims to investigate how cybersickness and sense of presence are associated and develop over time. During the experiment, participants were exposed to a virtual roller coaster and presented orally with questions aimed to quantify their perceived sense of presence and cybersickness. The results of the experiment indicate that cybersickness and sense of presence are both modulated by the time spent in the virtual setting. The utilized short measures for sense of presence and cybersickness were found to be reliable alternatives to multi-item questionnaires. Full article

Orchestrating collaborative learning activities is a challenge, even with the support of technology. Tangibles as orchestration tools represent an ambient and embodied approach to sharing information about the learning content and flow of the activity, thus facilitating both collaboration and its orchestration. Therefore, we propose tangibles as a solution to orchestrate productive collaborative learning. Concretely, this paper makes three contributions toward this end: First, we analyze the design space for tangibles as an orchestration tool to support collaborative learning and identify twelve essential dimensions. Second, we present five tangible tools for collaborative learning activities in face-to-face and online classrooms. Third, we present principles and challenges to designing tangibles for orchestrating collaborative learning based on our findings from the evaluation of ten educational experts who evaluated these tools using a usability scale and open questions. The key findings were: (1) they had good usability; (2) their main advantages are ease of use and support for collaborative learning; (3) their main disadvantages are limited functions and the difficulty to scale them to more users. We conclude by providing reflections and recommendations for the future design of tangibles for orchestration. Full article