Multimodal Technol. Interact., Volume 6, Issue 4 (April 2022) – 8 articles

Social robots hold potential for supporting children’s well-being in classrooms. However, it is unclear which robot features add to a trustworthy relationship between a child and a robot and whether social robots are just as able to reduce stress as traditional interventions, such as listening to classical music. We set up two experiments wherein children interacted with a robot in a real-life school environment. Our main results show that regardless of the robotic features tested (intonation, male/female voice, and humor) most children tend to trust a robot during their first interaction. Adding humor to the robots’ dialogue seems to have a negative impact on children’s trust, especially for girls and children without prior experience with robots. In comparing a classical music session with a social robot interaction, we found no significant differences. Both interventions were able to lower the stress levels of children, however, not significantly. Our results show the potential for robots to build trustworthy interactions with children and to lower children’s stress levels. Considering these results, we believe that social robots provide a new tool for children to make their feelings explicit, thereby enabling children to share negative experiences (such as bullying) which would otherwise stay unnoticed. Full article

Emotion classification is a research area in which there has been very intensive literature production concerning natural language processing, multimedia data, semantic knowledge discovery, social network mining, and text and multimedia data mining. This paper addresses the issue of emotion classification and proposes a method for classifying the emotions expressed in multimodal data extracted from videos. The proposed method models multimodal data as a sequence of features extracted from facial expressions, speech, gestures, and text, using a linguistic approach. Each sequence of multimodal data is correctly associated with the emotion by a method that models each emotion using a hidden Markov model. The trained model is evaluated on samples of multimodal sentences associated with seven basic emotions. The experimental results demonstrate a good classification rate for emotions. Full article

The data management process is characterised by a set of tasks where data quality management (DQM) is one of the core components. Data quality, however, is a multidimensional concept, where the nature of the data quality issues is very diverse. One of the most widely anticipated data quality challenges, which becomes particularly vital when data come from multiple data sources which is a typical situation in the current data-driven world, is duplicates or non-uniqueness. Even more, duplicates were recognised to be one of the key domain-specific data quality dimensions in the context of the Internet of Things (IoT) application domains, where smart grids and health dominate most. Duplicate data lead to inaccurate analyses, leading to wrong decisions, negatively affect data-driven and/or data processing activities such as the development of models, forecasts, simulations, have a negative impact on customer service, risk and crisis management, service personalisation in terms of both their accuracy and trustworthiness, decrease user adoption and satisfaction, etc. The process of determination and elimination of duplicates is known as deduplication, while the process of finding duplicates in one or more databases that refer to the same entities is known as Record Linkage. To find the duplicates, the data sets are compared with each other using similarity functions that are usually used to compare two input strings to find similarities between them, which requires quadratic time complexity. To defuse the quadratic complexity of the problem, especially in large data sources, record linkage methods, such as blocking and sorted neighbourhood, are used. In this paper, we propose a six-step record linkage deduplication framework. The operation of the framework is demonstrated on a simplified example of research data artifacts, such as publications, research projects and others of the real-world research institution representing Research Information Systems (RIS) domain. To make the proposed framework usable we integrated it into a tool that is already used in practice, by developing a prototype of an extension for the well-known DataCleaner. The framework detects and visualises duplicates thereby identifying and providing the user with identified redundancies in a user-friendly manner allowing their further elimination. By removing the redundancies, the quality of the data is improved therefore improving analyses and decision-making. This study makes a call for other researchers to take a step towards the “golden record” that can be achieved when all data quality issues are recognised and resolved, thus moving towards absolute data quality. Full article

Videogame research needs to identify how game features impact learning outcomes. This study explored the impact of the game feature–human interaction on training outcomes (i.e., affective states and declarative knowledge), and examined possible mechanisms (i.e., perceived value and active learning) that mediate this relationship. Participants included 385 undergraduate students: 122 trained alone and 263 trained with a team. All participants completed a computer-based training with four learning objectives (i.e., accessing the game, using the main controls, playing the game scenarios, knowing the game stations) prior to playing the game. After accounting for the indirect effects in the model, human interaction (i.e., playing with a team) had a significant direct effect on affective states, but not declarative knowledge. Learners who trained with a team reported greater positive affective states (indicated by psychological meaning, perceived enjoyment, motivation, and emotional engagement), but no difference in declarative knowledge (i.e., participants knowledge of team roles and responsibilities). Further analyses showed game-based training with a team impacted the affective states of learners through mechanisms of perceived value and active learning, while only active learning mediated the relationship between human interaction and declarative knowledge. Full article

This article takes a somatic dance standpoint to investigate bodily sensorial encounters with virtual reality technology (VR). Using somatic dance practices as sensory tuning methods, this article investigates the virtuality and corporeality of human and nonhuman bodies across virtual and physical environments and expanded sensory modes of seeing and feeling, using a multiperson interactive VR framework. Predicated on a dance-based sensibility and mode of engagement with VR, workshops crafted for groups of dancer and nondancer participants offer methods for the design and facilitation of VR encounters that expand ideas and expectations of both body and technology. Using autoethnographic writing and participant testimonies, alongside conceptual thinking around participation in VR, I present this methodological approach and demonstrate its potential to address new questions about VR experiences that are centred on the sensorial body. Full article

Humanity is living through a crisis that sees our way of life exhausting the resources of the earth and ourselves. The fashion sector shows the negative impacts of conspicuous consumption on our socioenvironmental wellbeing. Despite citizens’ growing awareness of their responsibility within consumption cycles, they reveal concerns about their lack of understanding and the support required for them to become agents of responsible consumption. The Circular Economy flourishes as a conceptual approach to help society transition to a more sustainable existence. This paper explores how emerging creative technology and interaction design might support a shift in the role of citizens in the Circular Economy. We performed a design inquiry that investigated the moment of acquisition via configuration of products, storytelling, and multimodal interaction techniques for the creation of experiences that could catalyse citizen-consumers to become custodians of materials. We developed a retail-based concept tool—The Compositor Tool—with which we ran a user study to investigate new experiential ways that consumers can participate in materials’ circularity. The study highlighted how experience design and new interaction techniques can introduce circularity as part of consumer experience by forging deeper connections between people and products/materials and enabling consumers to have more creative and informative material engagement. Full article

In many popular songs, lyrics are an important element. The act of collaborative lyric writing by multiple people may produce richer ideas than creative acts by individuals. However, if all members of a songwriting group do not accurately share the elements which are considered to be important in songwriting (i.e., story, character, viewpoint, and line of sight), then it would be difficult for ideas to diverge and converge, which is important in a creative act. In the present paper, we propose a collaborative lyric conception support method, which consists of lyric divergence support using a lyric association map (LAM) based on radial thinking in mind mapping and lyric convergence support using an enumeration of lyric candidate sentences created from the conceived lyric candidates. The visualization of the process of conception in a group facilitates the association, sharing, examination, and consideration of lyrics. Full article

This paper critically discusses the combination of creative and social research methods to generate a novel approach to explore the multimodal technoscape. This paper draws on an interdisciplinary exploratory case study on interactive skin—an emergent technology that augments and/or interacts with the skin. This paper shows how concepts from skin studies and the HCI literature can be used to draw on creative methods to think about and with the body. We describe the use of an online probe pack, a speculative research workshop and sensory research interviews using ‘proxy feelers’ to agitate the design space of interactive skin futures. We show how combining these methods provoked and expanded the scope of interactive skin from the technological to the sensory and the social. We discuss the opportunities and challenges of the research dialogues that this approach facilitated, make the case for creative methodological improvisation and exploration of emergent technologies and show how creative and social research methods can be combined to explore the interconnection between technology, society and design. Full article