Next Article in Journal
RSA-CP-IDABE: A Secure Framework for Multi-User and Multi-Owner Cloud Environment
Previous Article in Journal
Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multimodal Interaction: Correlates of Learners’ Metacognitive Skill Training Negotiation Experience

1
Department of Informatics and Telecommunications, University of the Peloponnese, 22100 Tripolis, Greece
2
Department of Informatics and Telecommunications, University of Athens, 15784 Athens, Greece
*
Author to whom correspondence should be addressed.
Submission received: 16 May 2020 / Revised: 24 July 2020 / Accepted: 26 July 2020 / Published: 29 July 2020

Abstract

:
Metacognitive training reflects knowledge, consideration and control over decision-making and task performance evident in any social and learning context. Interest in understanding the best account of effective (win-win) negotiation emerges in different social and cultural interactions worldwide. The research presented in this paper explores an extended study of metacognitive training system during negotiation using an embodied conversational agent. It elaborates on the findings from the usability evaluation employing 40 adult learners pre- and postinteraction with the system, reporting on the usability and metacognitive, individual- and community-level related attributes. Empirical evidence indicates (a) higher levels of self-efficacy, individual readiness to change and civic action after user-system experience, (b) significant and positive direct associations between self-efficacy, self-regulation, interpersonal and problem-solving skills, individual readiness to change, mastery goal orientation and civic action pre- and postinteraction and (c) gender differences in the perceptions of system usability performance according to country of origin. Theoretical and practical implications in tandem with future research avenues are discussed in light of embodied conversational agent metacognitive training in negotiation.

1. Introduction

Negotiation is a social influence process of interaction by two or more parties (conversational agents) making decisions, allocating resources, or resolving disagreement in a mutually interesting, and possibly cross-cultural, context. It reflects differing preferences and decision-making between partners and engenders mutual effects on one another with a view to work together to achieve a satisfactory agreed outcome, that is, an effective negotiation [1]. Negotiation may take place between human agents as well as between human and virtual agents. The negotiation process can be manifested in intelligent interaction, such as serious games, intelligent tutoring systems, chatbot interaction and human-robot interaction systems. Embodied conversational agents (ECAs) are used across diverse health (e.g., cognitive disability), academic (e.g., STEM education), business and training settings [2,3,4,5]. In human-to-human negotiation, issues regarding interaction may be addressed and resolved either with more integrative, collaborative-associated (i.e., win-win) or more competitive-oriented (i.e., win-lose) approaches [6]. In ECAs, in particular, the latter are programmed not only to facilitate negotiation by enabling positive climate, aligned consensus, favourable attitudes and learning experience [7,8,9]. They can also be used to set the negotiation procedure in a rather more equal basis of reference, given that status and power differentials rather pertinent in human-to-human interaction, for example one partner may dominate over the other, risking trust between negotiating parties [10], tend to become more easily absent in human-to-agent interaction [11]. As such, the negotiation process may seem to fall short of being an easy-managed, fully predictable and consistently successful social interaction procedure [12].
The social nature of negotiation between humans, as an interaction process, maps into the need for metacognitive skill training [13]. This is apparent in any given social, educational and business exchange (interaction) which requires knowledge (awareness), consideration (reflection) and control (regulation) over ones’ objectives in a way that directly relates to successful task performance [14] and decision-making improvement [15]. In cases when humans do perform well, they (a) are more likely to indicate higher levels of self-efficacy (i.e., the positive motivational attitude of accomplishing challenging tasks), (b) differentiate and self-regulate their learning approach for subsequent activity and vice versa (i.e., self-regulation) [16,17], (c) seem to practice their skill to use logical reasoning for problem-solving and communicate successfully with others to evaluate the relevant context (i.e., interpersonal and problem-solving skills), (d) are oriented toward adopting mastery-as-a-goal in their choices (i.e., mastery goal motivational disposition) and choose assignments that adhere to inspiring and advocate additional training and learning ones [18]. In taking an active social presence role when relating to others [5], facilitating rapport and fostering cooperation and coordination [19], they are expected to benefit from activities to engage with the community by proactively assisting others (i.e., civic action) and exhibit positive reactions towards the deployment of changes (i.e., readiness to change) [20]. Similar human-to-human interaction social presence and intelligence attributes are replicated in ECA setups (human-to-virtual agent), as the latter are likewise modelled to be socially intelligent multimodal systems that demonstrate human-matching physical appearance and behavioural characteristics, treated by their human counterparts like real humans [21]. As the employment of ECAs tends to increase, enabling (a) successful interaction between humans and ECAs, in terms of their negotiation (training) functional capabilities by gaining the users’ perceptions regarding natural human–computer interaction (macrolevel; multimodal system performance) and (b) assess the contribution of metacognitive-, individual- and community-associated skills and attitudes (microlevel; users’ dispositions) as an integral element for mastering successful user learning experience exercised within ECAs across diverse cultural context. An advanced research interface of such an ECA system [22,23,24] is the one that employs multimodal interaction for instructing metacognitive knowledge and skills of both application and users [25] by modelling human negotiation behaviour and being evaluated across diverse cultural context [26]. Such multimodal metacognitive training system allows (a) extended cooperation between negotiating agents (human and virtual) in order to reach a designated consensus over negotiation issues, (b) natural language interaction by both interacting agents (human and virtual) and (c) demonstration of expressive quality of verbal and nonverbal characteristics generated by the conversational agent.
Based on the above reasoning, therefore, the research illustrated in the current paper pursues: (a) to extend previous evidence regarding ECA functional interaction (negotiation training) abilities [27] (i.e., macrolevel; dialogue system performance), (b) to further explore users’ metacognitive-individual and community-related attitudes and skills embedded within ECA negotiation training, for the first time (i.e., microlevel; users’ attitudes and skills) and (c) to integrate ECA functionality (macrolevel) with users’ metacognitive-motivational and behavioural indicators (microlevel), also for the first time, as an essential element of carefully designed actions in human to virtual agent immersive learning and negotiation training environment, where social, affective and functional properties of agents, as well as user attitudes and behaviour, tend to serve as critical attributes in unravelling human–artificial interactions successfully [28]. In accordance with the aforementioned rationale, therefore, the research questions that reflect the scope of the current research are the following:
  • [RQ1] Do self-efficacy, self-regulation, interpersonal and problem-solving skills, individual readiness to change, mastery goal orientation and civic action attitudes and skills associate significantly with negotiation training?
  • [RQ2] Does the interaction with the ECA training system lead to improved metacognitive skill attributes (as listed above) when measuring them before and after the interactive training?
  • [RQ3] Does a prominent metacognitive attitude change (pre- and postinteraction) exist and how do the other examined attitudes affect it?
  • [RQ4] Are there any differences in the perceptions of users regarding the ECA usability for evaluation based on gender and country of origin?
The paper is divided as follows. Section 2 presents the related work in negotiation-related, metacognitive, motivational and behavioural attitudes and skills within the intelligent virtual agent context. Section 3 presents the research design. Section 4 reports the results on metacognitive skill training, while Section 5 elaborates on the user study results, focusing on user gender and origin. Section 6 summarizes the results and illustrates their theoretical and practical implications. Section 7 concludes the paper and presents future research ideas on ECA negotiation-associated learning and training.

2. Related Work

Advances in intelligent interaction have been developed by using either narrower or richer modalities, (multimodal) designs pending on their context of relevance to support real-life human–computer interactions. Bickmore and Cassell claim that “as computers begin to resemble humans, the bar of user expectations is raised” [29]. An enormous progress has been propelled. However, building intelligent virtual agents with advanced verbal and nonverbal skills altogether to fully address consistent, coherent, realistic and desirable functional and behavioural responses interpreted by the human users as humanlike and supporting long-term engagement in the endorsed behaviour posits a considerable ongoing challenge for both academic and commercial artificial intelligence designers to navigate [30]. Given the broad range of conditions in which intelligent agents or ECAs may operate, exploring the way that certain design options might influence usability, interaction and learning experience outcomes across diverse situations and cultural context, is a considerable research issue to investigate [31]. An emerging need is the extension of modelling of intelligent agent-to-agent negotiations (within multiagent environments) toward human-like negotiation behaviour as embedded in the training context [24].
Existing research has stressed the importance of investigating the antecedents and outcomes of metacognitive and individual and community-associated attitudes and skills across human-to-human (face-to-face) and human-to-virtual (human-to-agent) interactions during instruction and learning [32]. Therefore, the challenge of designing context-specific intelligent virtual systems and ECAs to support learner motivation, involvement, knowledge, attitudes and skills remains an explicitly open demanding perspective aimed to design approaches that are capable of gaining user rapport, establishing a long-term interaction in a personalized and immersive way guided by intelligence (i.e., cognitive abilities) to build positive learning achievement, learning attitudes and training efficiency outcomes (i.e., knowledge, problem-solving, logical thinking), as related work indicates below.
Teaching and learning within sophisticated interaction applications illustrates that intelligent systems appear to rely on positive outcomes for motivational and problem-resolution intentions and capabilities [33], accompanied by facilitating student accessibility of instructional (educational) content [34] and supporting integrative potential during interaction including satisfaction and connectedness with the virtual agent during training interaction [7]. Inducing motivation and user self-confidence [35] and increased e-learning performance [36], aims to result in effective interaction and user engagement with the (training) virtual agents and task fulfilment by elderly individuals including users with diagnosed cognitive impairments [2]. Enhancing learners’ positive judgments regarding their ability to organize and accomplish the required courses of action to achieve successful learning performance after training (i.e., self-efficacy [37,38]) positively affects the inclination to participate in the learning activity at hand [4,39].
The knowledge acquisition of the trainees is focused on exercising critical thinking [40] and increasing the instructional and engaging power of the training environment [41,42]. Although the majority of the above reported results seems to indicate rather favourable metacognitive and individual-level related effects for users in intelligent systems training situations, in terms of building effective learning performance indicators, relative evidence tends to be rather mixed. Holmes reports improvement in learning performance indicators for learners after interaction with intelligent tutoring system [43]. Mayer and DaPra [44] improved knowledge transfer when interacting with intelligent virtual agents, while van der Meij [38] reports lowered learning gains, that is decreased task retention, for vocational education training students interacting with agents, in contrast to human interaction partners. Similar inconclusive findings are found in regard to gender differences for users interacting with ECAs. Guadagno et al. reported that both male and female users indicated similar attitudes and higher attitude change when interacting with an ECA of the same gender as theirs [45]. This effect was noticeable for female participants, potentially attributed to more powerful identification (matching) with their own gender in comparison to male counterparts (i.e., Self-Categorization and Social Identity Theories [46,47,48]). In other cases, male users exhibited more positive behavioural change after interacting with an ECA [49]. Within human–robot interaction, in particular, female users did not exhibit any attitude change in response to the robot’s gender identification during interaction, while their male peers demonstrated different attitudes based on matching to their own gender [50]. Along with the mixed evidence findings, therefore, further exploration of learners’ metacognitive-individual and community-related attitudes and skills within natural-language human–agent negotiation training of diverse cultural and gender-specific interaction, is required.

3. Research Methodology

This section elaborates on the research design employed, a brief description of the metacognitive training ECA system and the empirical evidence analysis obtained.

3.1. Multimodal ECA

For this work we used the Metalogue ECA [25]. The ECA basic built utilised cognition, learning, in-action feedback and spoken dialogue interaction. These were represented by the several modalities embedded such as speech recognition and natural language processing, facial expression, body movement and biosensor metrics, eye gaze and face tracking, detection of static and flexible face and body manifestations, natural language processing, body and face states [51,52,53].
The ECA uses a virtual agent that negotiates issues of interest with human interaction partners. The agent and the human have options to agree, disagree, propose and counterpropose solutions to reach an agreement. The outcome can be an agreement or not. Each issue that is negotiated has arguments and positive and negative points for each subtopic of the issue. The two negotiation partners have different goals to pursue. The interaction is achieved via natural voice communication as well as visual representation of the status of the negotiation and the user goals (Figure 1). The user goals are visualised as options under the four aspects of the negotiation. The ones in blue are the positive outcomes (darker blue is more positive) and in red are the negative (dark red is more negative). White coloured options are neutral. The visualised goals represent the user agenda (i.e., point of view), the virtual agent has a similar agenda that is unknown to the user. The user and the agent negotiate and bargain positions and issues to achieve an outcome that is positive for both, that is an agreement. Additionally, real-time feedback on the user posture, voice changes (volume and speed) helps the user learn to communicate in a timely, sophisticated, attentive and nonaggressive manner.
The negotiation training was the scenario of choice that aimed to train the human metacognitive skills. Through negotiation, the user learns how to use argumentation, how to be empathic to the needs of others, how to reach agreements, and so on. In our experiments, the verbal and nonverbal reactions such as bending, using direct eye contact, smiling and moving hands, have been utilised to facilitate learners feeling trust, connectedness and immersion [54,55,56] potentially fostered by the social presence and intelligence of the agent, as perceived by the participants [34,57].

3.2. Research Design

The research strategy followed the before-after assessment method [5] in order to assess the effects of the metacognitive multimodal training on the learning negotiation (win-win) experience of 40 adult participants characterized by diverse demographic and cultural origin. The mean age of users was 20 years, 60% male and 40% female. Before the usability evaluation sessions, a pilot study was performed with five users interacting with the system in order to evaluate successive and converging visual signals and estimate the capacity of information offered, aimed to achieve appropriate instruction during the actual time system condition [58].
The participants were all fluent English speakers. First, they were introduced to the ECA functionalities, watched two demo interactions with the system executed by the facilitators and were provided with ample time for inquiries. All participants completed the informed consent forms and filled in the before-interaction questionnaire. Each user negotiated three random multi-issue (win-win) scenarios with the ECA for an average time of 15 min for all three sessions, as in previous similar studies [59]. Finally, they completed the after-interaction user experience questionnaire and 23 usability evaluation questions supplemented by required demographic information on five Likert scale items.
The user experience evaluation accompanied the main metacognitive training assessment of six scales, as follows: general self-efficacy [60] (10 items), self-regulation [61] (10 questions), interpersonal and problem-solving skills, civic action (adapted from Civic Attitudes and Skills Questionnaire; 12 and eight items, accordingly) [18], individual readiness to change (modified version of individual readiness to change scale; six questions) [62] and mastery goal orientation (eight items) [18,63] scales. Higher scores on all scales reflect greater intentions towards (a) fulfilling challenging tasks (self-efficacy), (b) maintaining and controlling own attention (self-regulation), (c) analytically thinking in solving problems and placing oneself in the position of others (interpersonal and problem-solving), (d) actively assisting others in volunteering, community and environmental issues (civic action), (e) adopting and sustaining change initiatives (individual readiness to change) and (f) encompassing mastery as a goal in performing triggering tasks or activities (mastery goal orientation). In all cases, adequate reliabilities were obtained (see Table 1) except for self-regulation before-interaction, for which there was no correlation found and consequently was excluded from subsequent analyses. Items that were worded negatively for presentation were reverse coded before the analyses were conducted. 40 participants fully completed the interaction and provided feedback.

4. Metacognitive Skill Training

The following subsections present the findings regarding the research questions 1–3, set in the Introduction.

4.1. Metacognitive Skill Attribute Correlations

Table 1 presents the means, standard deviations, α consistency reliabilities and correlations for the study variables for Greek user (participants) (N = 30). At the bivariate level, most of the variables before-interaction correlated significantly and positively with those after-interaction, the strongest correlation being that between civic action before and after (r = 0.92, p < 0.01) and the weakest between self-efficacy before and civic action after (r = 0.01, p = ns).

4.2. Learner Skill Change before and after Interaction

To study the change in the learner skill training according to the variables of the study, paired samples t-tests were executed for the Greek adult learners (N = 30). The results indicate that there is a significant difference between self-efficacy before and after the interaction, lending support to users’ higher sense of intrinsic motivation, competence and mastery after training and corroborating to them having a stronger belief of executing tasks effectively after the skill training session (Table 2).
The significant difference reported between civic action pre- and postinteraction, further indicates that the participants are likely to get involved in current and future civic and community service post-training, mirroring their favourable attitudes towards humanitarianism. Finally, significant difference is found between individual readiness pre- and postinteraction, indicating that the users are more likely to advocate and be confident in achieving to establish helpful change initiatives post-training, mapping into their positive sense of confidence in accomplishing beneficial change efforts. The visual representation of self-efficacy, civic action and individual readiness to change before-after scales is indicated in Figure 2.

4.3. Self-Efficacy Prediction Pre- and Post-Training

In order to test for the prediction of self-efficacy pre- and post-training hierarchical regression analyses were executed for all participants. Before the analysis, we ensured that all prerequisite testing conditions related to the analysis, such as deviations from normality, lack of multicollinearity and influential cases, were met.
The control variable was gender, while the independent variables tested for prediction pre- and postinteraction were the self-regulation, interpersonal and problem-solving skills, civic action, individual readiness to change and mastery goal orientation sets of responses, accordingly. The results of the relationships that were computed between the prospective variables are presented in Table 3 and Table 4.
Significant relationships were indicated between interpersonal and problem-solving skills before, civic action before and self-efficacy before (β = 0.52, p < 0.01 and β = −0.39, p < 0.05, respectively), signifying interpersonal and problem-solving skills before as the best predictor, with the final model explaining an additional 10% (F(4, 36) = 3.10, p < 0.05) of the variance in self-efficacy before scores (Table 3). A significant relationship was indicated between interpersonal and problem-solving skills after and self-efficacy after (β = 0.59, p < 0.001), explaining an additional 29% (F(3, 36) = 13.31, p < 0.001) of the variance in self-efficacy after the interaction (Table 4).

5. Usability Evaluation

The distribution (%) of the total sample (N = 40), Greek (N = 30) and German (N = 10) users’ responses regarding the usability evaluation questions postinteraction by gender and origin, are reported below. The full data can be found in the Appendix A.
The participants originated from Greece (75.0%) and Germany (25.0%), representing 60.0% male and 40.0% female learners, respectively. Table 5 indicates the percentage distribution of the user perceptions regarding metacognitive and negotiation training, scope of the system and suggestions for future improvement.
The higher percentage of participants (45.0%) indicated that they did not really know what metacognitive skills were, followed by those who claimed lack of familiarity (37.5%) and awareness of metacognitive skill training (17.5%), respectively. 47.5% of users indicated that the system scope was negotiation skill training, followed by 40% of those reporting that the system was about artificial intelligence skill training, 10.0% metacognitive skill training and 2.5% responded they did not know what the system was about. Regarding the user perception on the “current” functionalities and suggestions for future versions, 67.5% of them reported features add-ons and 32.5% learning progress feedback for the ECA.
There are specific differences regarding the perception of the interaction between the two genders (see Table A1 of the Appendix A). For example, female participants reported that it was moderately easy to interact with the system, while the male participants indicated a perceived slightly artificial interaction. The male participants considered the approach as quite promising to become a fully-fledged skill training application. On the other hand, the female participants were positive but more reserved. In sum, both genders overall seemed to indicate moderate to higher satisfactory perceptions regarding timely communication flow, useful provision of information, self-confidence in their knowledge of the training application functionalities, interesting idea to pursue, easy setup to understand, flexible to use a simplified version and pre- and postinteraction feedback as moderately valuable to facilitate learning performance awareness.
Greek users tended to exhibit moderate to higher favourable perceptions regarding accurate communication flow, helpful delivery of information, self-confidence in their understanding of its’ functionalities, useful idea to adopt, straightforward setup to comprehend, adaptable enough to exercise a simplified version and pre-, during and postinteraction feedback as moderately valuable to facilitate learning performance (see Table A2 of the Appendix A). However, female Greek participants indicated rather moderate to neutral perceptions concerning task fulfilment during interaction with the system, in relation to their male Greek counterparts and the total sample’s related attitudes of both genders, accordingly.
The German participants exhibited rather mixed responses in relation to the ones received by both genders of the total sample and their Greek counterparts, respectively (see Table A2 of the Appendix A). Although the majority of female German users reported absolute satisfaction regarding the utility of the information provided and a moderate extent of agreement with the fast speed of interaction. They reported experiencing rather less accurate actions of the system, greater difficulty in multimodal interaction and fairly moderate perceptions as to the overall usefulness of the interactive system’s role.

6. Discussion

The prototype, implementation and evaluation of our negotiation training multimodal system aimed to seize interactive learning and training of metacognitive-related essentials for both system and participants, demonstrated significant positive findings in relation to the favourable attitudes and skills that learners improved after they interacted with the ECA during our assessment workshop. Our usability evaluation (macrolevel) corroborated by integration and additional exploration of metacognitive and individual- and community-level attitudes and skills embedded in multi-issue ECA negotiation learning context (microlevel) revealed certain user perception as the main results reported in this paper. ECA training and usability assessment incorporated gender differences and, for the first time, with the conceptual and practical exploration of metacognitive, individual- and community-linked attitudes and skills, such as self-efficacy, self-regulation, individual readiness to change, mastery goal orientation and facets of civic attitudes and skills like civic action and interpersonal and problem-solving skills they convey and mirror.
Such theoretical whilst also evidence-based joint examination of the associations between the aforementioned concepts is explored (a) within multimodal natural language negotiation environment, (b) extends relevant results from service, computer-supported learning, business and intelligent tutoring systems alike [18,62,64,65] and (c) signals further noteworthy contributions and designates beneficial prospect for both comprehension and improvement of such positive attitudinal constructs and civic-associated attitudes and skills for future activities aimed to foster learners to delve into and expand those attitudes within artificial intelligence negotiation learning and training context, to a greater extent (RQ1).
The significant positive correlations obtained pre- and postinteraction seem to capture the exploration of favourable metacognitive, individual- and community-associated attitudes and skills not only prior but also post-negotiation context, thus, relating and transferring beneficial learning experience outcomes to computer-human interaction context mapped into the ECA negotiation environment. In addition, the higher levels of self-efficacy, civic action and individual readiness to change that Greek users exhibited postinteraction (t-tests), tend to indicate that our training approach appears to fuel such metacognitive, individual- and community-based interaction learning attitudes and skills in ECA negotiation context, per se. Thus, lending support to previous related findings in both human-to-human and intelligent settings alike [37,66], validates the extension of prior favourable virtual learning and training experience [42] to integrate additional positive civic and change-related attitudes and skills outcomes as transferred into the ECA as a natural negotiation learning experience (RQ2).
The study participants who had a high self-reported logical reasoning and analytical thinking, listening to other people’s opinions, planning to do volunteering work and become involved in community (regressions), were more positive in facilitating and being more competent and confident in accomplishing their goals and handling whatever comes their way pre-interaction. Postinteraction, similar findings were obtained only for those bearing stronger ability to cooperate with others, thinking logically in solving problems and listening to others’ opinions. This may have to do with the capability of the ECA to detect this interaction, lending support to previous findings which postulate that virtual intelligent systems seem to facilitate learners in logical reasoning, decision making and interpersonal and problem-solving tasks [40,67,68]. Based on the significant contributions indicated above, therefore, the proposed multimodal training approach appeared to be advantageous in (a) fuelling, mastering and associating both metacognitive, individual- and community-related interaction-based learning attitudes and skills and (b) expanding prior related findings from human-to-human [69] into agent-to-human in intelligent virtual negotiation training settings [27] to intelligent virtual natural negotiation learning context. Stressing further, the helpful learning experience that ECAs may bear in challenging learners and users to be proactive in advancing their reflection, lead to understanding and mastery of their current and future learning performance within intelligent virtual natural language negotiation per se. Self-efficacy was found to be the metacognitive skill attitude that most prominently described the change of the users’ skills after the training (RQ3).
Following the usability evaluation responses postinteraction based on gender (distribution of users’ responses based on gender), it seems that, overall and across Greek and German users, both male and female participants did perceive the system with moderately clear role, use, easiness to complete tasks and understanding its’ functionalities, timely communication, helpful information provided and an interesting idea for future training development. The female Greek users, in particular, seemed to indicate rather moderate to neutral perceptions concerning task fulfilment during interaction in comparison to their male Greek counterparts and the overall sample’s related attitudes of both genders, respectively (RQ4). Similar mixed results were also reported by German genders. Although the majority of female participants seemed to be absolutely pleased with the usage of the information offered and at a moderate degree of agreement with the fast pace of interaction perceived, they reported rather less correct actions of the system and lower self-confidence in their knowledge of using the system on their own, increased difficulty in multimodal interaction and moderate perceptions as regards the overall helpfulness of the system (RQ4). The aforementioned gender differences concerning interaction performance indicators, especially within Greek and German participants, (a) tend to corroborate previous findings, illustrating more favourable attitudes for male users after interacting with intelligent virtual agents [49], and (b) contrast those reported by [30,70], who demonstrated no gender differences between participants in their interaction with the intelligent virtual character (agent) at hand. This contradiction perhaps reinforces the significance of the findings especially within the diverse cultural background that characterizes our users. This poses an appealing question for future exploration and is additionally discussed later.
It could be also noted, as shown by the research findings, that the majority of users perceived the system as that of negotiation skills training and suggested further features add-on and learning progress feedback for future system improvement. In that sense, therefore, and in conjunction with their moderately favourable usability experience attitudes postinteraction already illustrated, it could be argued that our approach appeared to be also promising in expanding negotiation and aligned with metacognitive, individual- and community-related skills training beyond traditional and intelligent tutoring learning context, as introduced, exercised and assessed within an ECA environment.
A longitudinal research design and intervention over longer cohorts and follow-up evaluation across different real-life learning environments (e.g., industry, education) or academic context (e.g., win-win negotiation, metacognitive, individual and civics-related attitudes and skills), field (e.g., leadership, project management, organizational behaviour, civics) and (or) diverse user groups based on their generational (i.e., millennials vs. older populations) and cultural background differentials, might permit greater generalizability of our current findings. Such longitudinal replication not only would clarify and promote (a) the positive effects of metacognitive, individual and community-related attitudes and skills within intelligent virtual natural language agents’ long-term relatedness further, but would also (b) explore the persistency of similar favourable user perceptions gained, as useful feedback obtained for the future improvement of our training system. The latter might also further be facilitated by an additional potential exploration of direct personalization features embedded, following corresponding research [71]. Nevertheless, the hands-on issues and built-in difficulties associated with the extremely demanding complexity of the ECA design, execution and assessment, need to be taken into account.

7. Conclusions

Sophisticated interactive systems that are capable of negotiating through natural language with humans are among society’s ongoing advanced technological challenges and their application appears to be significant for critical educational, business and individual life improvement invested in a long-term approach. In the present paper, we sized up empirical research evidence that aimed, for the first time, to jointly explore the effectiveness of a multimodal system (i.e., macrolevel) deployed to instruct metacognitive, individual- and community-related attitudes and skills (i.e., microlevel) to mature cross-cultural learners in a natural multi-issue negotiation space setting. The results indicated moderate to higher satisfaction levels overall for participants negotiating with the dialogue system, slight gender cross-cultural differences regarding the overall user experience, positive overall learner feedback concerning negotiation training and favourable metacognitive, individual and community attitudes and skills outcomes after interaction with the ECA trainer. Looking further forward, avenues for future research include the following: (a) in transferring the ongoing research calls for facilitating the best account of successful integrative negotiation continuum from human face-to-face to human-to-virtual interactions, by investigating the integrative (win-win) negotiation patterns emerging during ECA interaction across diverse interpersonal, intergroup and international context and generations (millennials vs. prior generations) and (b) whether the aforementioned win-win negotiation strategies tend to be rather static or change over time based on, for instance, ECA and user personality traits, demographics and nonstandard conditions endowed. Analogous to inherent uncertainty surrounding human face-to-face negotiation outcomes in terms of coordinating the interdependency existed between partners to foster favourable decision-making and agreements [72], it may be claimed that it is equally or even more fundamental to further understand how embodied conversational and human agents naturally interact in an autonomous negotiation environment (modelling humanlike realism) fuelled by programmed better agreements and associated time, cost and cognitive load reduction gains invested [73].

Author Contributions

Conceptualization, D.S., E.M., C.V. and D.M.; methodology, D.S., E.M., C.V. and D.M.; software, D.S., E.M., C.V. and D.M.; validation, D.S., E.M., C.V. and D.M.; formal analysis, D.S., E.M., C.V. and D.M.; investigation, D.S., E.M., C.V. and D.M.; resources, D.S., E.M., C.V. and D.M.; data curation, D.S., E.M., C.V. and D.M.; writing—original draft preparation, D.S., E.M., C.V. and D.M.; writing—review and editing, D.S., E.M., C.V. and D.M.; visualization, D.S., E.M., C.V. and D.M.; supervision, D.S., E.M., C.V. and D.M.; project administration, D.S., E.M., C.V. and D.M.; funding acquisition, D.S., E.M., C.V. and D.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

The authors would like to thank the participants and all the personnel that made this study possible.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Distribution (%) of users’ responses in usability evaluation postinteraction based on gender (N = 40).
Table A1. Distribution (%) of users’ responses in usability evaluation postinteraction based on gender (N = 40).
Q1: Do You Think the Actions of the System Were Correct?
No, not reallySlightlySomewhatModeratelyYes, they were spot on
Male 4.237.5.745.812.5
Female6.312.52543.712.5
Q2: Did the Interaction with the System Made Sense to You?
Slightly didSomewhat didModerately didYes, the system role and use are clear
Male8.38.341.741.7
Female18.818.843.718.8
Q3: Did the System Communicate Enough Information to you?
Slightly didSomewhat didModerately didYes, the system communicated enough information
Male4.229.15016.7
Female12.5255012.5
Q4: Did the System Communicate Too Much Information to You?
Yes, the system overloaded me with informationSlightly didSomewhat didModerately didThe information provision was just fine
Male12.520.820.829.220.8
Female31.118.818.812.518.8
Q5: Was the Information Provided by the System to You Useful?
No, I was expecting information I could somehow useSlightly didSomewhat didModerately didYes, it was very useful
Male 4.22541.729.1
Female 6.318.85025
Q6: Was the System Communication to You Timely?
No, it was out of contextSlightlySomewhatModeratelyYes, it was timed correctly and in context
Male 12.541.729.216.6
Female6.325252518.8
Q7: Was It Easy to Complete Tasks in Your Interaction?
No, it was very hardHardNeutralEasyYes, very easy
Male 12.537.537.512.5
Female 12.556.3256.3
Q8: Was the Pace of Interaction Fast Enough to Feel Right?
No, it was too slowSlightly slowNeutralModerateYes, it was just right
Male4.216.72537.516.7
Female 6.331.343.818.8
Q9: Was the Pace of Interaction Slow Enough to Feel Right?
No, it was too fastSlightly fastNeutralModerate Yes, it was just right
Male 8.333.329.229.2
Female6.312.531.2550
Q10: Did You Know What You Could Say at Each Point of the Dialogue?
NeverRarelySometimesOftenAlways
Male 16.72533.325
Female6.318.837.52512.5
Q11: Would You Say That Your Interaction with the System Was Natural?
No, it was very artificialSlightly artificialNeutralModerately naturalYes, it was quite natural
Male4.233.3252512.5
Female 5012.52512.5
Q12: Are You Confident You Know Enough about the Functionalities and the Information Found in the System so You Would Be Able to Use It on Your Own?
Yes, very confidentYes, but there are notions I did not understandSo and soNot muchIt is quite complex, I suggest you develop a training course
Male33.337.520.84.24.2
Female252543.76.3
Q13: How Easy Was to Interact with the ECA?
Very hardHardNeutralModerately easyPretty easy
Male 20.833.32520.8
Female6.36.331.3506.3
Q14: How Natural Was to Interact with ECA?
No, it was very artificialSlightly artificialNeutralModerately naturalYes, it was very natural
Male4.229.22516.725
Female6.312.562.518.8
Q15: Do You Think the Concept Is an Interesting Idea?
No, not muchSomewhatModeratelyYes, a lot
Male4.24.212.579.1
Female 18.881.2
Q16: Do You Find the Setup of the System Intimidating?
Yes, it is quite hard to understand/useHardNeutralModerately easyNo, it is quite easy to understand/use
Male4.24.28.337.545.8
Female 6.3252543.8
Q17: Would You Use the System again if It Was an Integral Part of Your Training Routine?
No, I hated itSlightly hated itNeutralModerately liked itYes, I quite liked it
Male 4.212.516.766.7
Female 252550
Q18: Do You Think the System Has the Potential to Become a Great Skills Training Application?
No, it is uselessSlightly uselessNeutralSlightly promisingYes, it is quite promising
Male 4.24.22566.7
Female 2543.831.3
Q19: Would You Use a Simplified Version of the System with Only the Content or Functionality You Find It Interesting?
No, no waySlightlySomewhatModeratelyYes, sure
Male4.28.38.329.250
Female6.36.36.337.543.79
Q20: Was the Feedback Provided “during” the Interaction Valuable to You?
No, not valuableSlightly valuableSomewhat valuableModerately valuableYes, very valuable
Male 41.637.520.8
Female 6.331.343.818.8
Q21: Was the Feedback Provided “after” the Interaction Valuable to you?
No, not valuableSlightly valuableSomewhat valuableModerately valuableYes, very valuable
Male8.3 29.241.620.8
Female 18.86.35025
Q22: Did the Feedback That Was Provided “during” the Interaction Help You to Become More Aware of Your Performance?
No, not at allSlightlySomewhatModeratelyYes, very much
Male4.233.316.72520.8
Female6.312.518.843.718.8
Q23: Did the Feedback That Was Provided “after” the Interaction Help You to Become More Aware of Your Performance?
No, not at allSlightlySomewhatModeratelyYes, very much
Male8.38.329.237.516.7
Female6.312.512.55018.8
Table A2. Distribution (%) of Greek (N = 30, 17 male, 13 female) and German (N = 10, 7 male, 3 female) users’ responses in the usability evaluation questions postinteraction per gender. The origin of the participants is depicted using the abbreviations GR for Greek and DE for German users.
Table A2. Distribution (%) of Greek (N = 30, 17 male, 13 female) and German (N = 10, 7 male, 3 female) users’ responses in the usability evaluation questions postinteraction per gender. The origin of the participants is depicted using the abbreviations GR for Greek and DE for German users.
Q1: Do You Think the Actions of the System Were Correct?
No, not reallySlightlySomewhatModeratelyYes, they were spot on
GRMale 35.352.911.8
Female 7.730.846.115.4
DEMale 14.357.114.314.3
Female33.333.3 33.3
Q2: Did the Interaction with the System Made Sense to You?
Slightly didSomewhat didModerately didYes, the system role and use is clear
GRMale 5.941.252.9
Female17.617.647.217.6
DEMale28.614.342.914.3
Female33.333.3 33.3
Q3: Did the System Communicate Enough Information to You?
Slightly didSomewhat didModerately didYes, the system communicated enough information
GRMale5.923.552.917.6
Female7.723.153.815.4
DEMale 28.657.114.3
Female33.333.333.3
Q4: Did the system communicate too much information to you?
Yes, the system overloaded me with informationSlightly didSomewhat didModerately didThe information provision was just fine
GRMale11.817.623.529.417.6
Female23.130.715.415.415.4
DEMale 28.614.328.628.6
Female33.3 33.3 33.3
Q5: Was the Information Provided by the System to You Useful?
No, I was expecting information I could somehow useSlightly didSomewhat didModerately didYes, it was very useful
GRMale 5.917.641.235.3
Female 7.730.830.830.8
DEMale 42.957.1
Female 100
Q6: Was the System Communication to You Timely?
No, it was out of contextSlightlySomewhatModeratelyYes, it was timed correctly and in context
GRMale 11.835.329.423.5
Female 15.438.430.815.4
DEMale 14.357.128.6
Female33.366.7
Q7: Was It Easy to Complete Tasks in Your Interaction?
No, it was very hardHardNeutralEasyYes, very easy
GRMale 5.935.347.111.8
Female 7.761.523.17.7
DEMale14.328.642.814.3
Female 33.333.333.3
Q8: Was the Pace of Interaction Fast Enough to Feel Right?
No, it was too slowSlightly slowNeutralModerateYes, it was just right
GRMale 5.929.441.223.5
Female 7.730.846.115.4
DEMale14.357.114.314.3
Female 33.366.7
Q9: Was the Pace of Interaction Slow Enough to Feel Right?
No, it was too fastSlightly fastNeutralModerateYes, it was just right
GRMale 5.935.329.429.4
Female7.77.730.853.8
DEMale 14.328.628.628.6
Female 33.333.333.3
Q10: Did You Know What You Could Say at Each Point of the Dialogue?
NeverRarelySometimesOftenAlways
GRMale 17.617.629.435.3
Female 23.138.538.57.69
DEMale 28.628.642.9
Female33.333.333.3
Q11: Would You Say That Your Interaction with the System Was Natural?
No, it was very artificialSlightly artificialNeutralModerately naturalYes, it was quite natural
GRMale5.923.529.429.411.8
Female 46.115.423.115.4
DEMale14.371.414.3
Female 66.7 33.3
Q12: Are You Confident You Know Enough about the Functionalities and the Information Found in the System so You Would Be Able to Use It on Your Own?
Yes, very confidentYes, but there are notions I did not understandSo and soNot muchIt is quite complex, I suggest you develop a training course
GRMale47.141.211.8
Female30.823.138.57.7
DEMale 57.128.614.3
Female 33.366.7
Q13: How Easy Was to Interact with the ECA?
Very hardHardNeutralModerately easyPretty easy
GRMale 17.635.329.417.6
Female 30.861.57.7
DEMale 28.657.114.3
Female33.333.333.3
Q14: How Natural Was to Interact with ECA?
No, it was very artificialSlightly artificialNeutralModerately naturalYes, it was very natural
GRMale5.923.529.45.935.3
Female 15.453.823.17.7
DEMale14.342.914.328.6
Female33.3 66.7
Q15: Do You Think the Concept Is an Interesting Idea?
No, not muchSomewhatModeratelyYes, a lot
GRMale5.9 5.988.2
Female 23.176.9
DEMale 14.328.657.1
Female 100
Q16: Do You Find the Setup of the System Intimidating?
Yes, it is quite hard to understand/useHardNeutralModerately easyNo, it is quite easy to understand/use
GRMale5.9 5.929.458.8
Female 7.730.815.446.1
DEMale 14.314.371.4
Female 100
Q17: Would You Use the System again if It Was an Integral Part of Your Training Routine?
No, I hated itSlightly hated itNeutralModerately liked itYes, I quite liked it
GRMale 5.95.911.876.5
Female 15.430.853.8
DEMale 42.857.2
Female 100.0
Q18: Do you Think the System Has the Potential to Become a Great Skills Training Application?
No, it is uselessSlightly uselessNeutralSlightly promisingYes, it is quite promising
GRMale 35.364.7
Female 15.438.546.1
DEMale 57.242.8
Female 66.733.3
Q19: Would You Use a Simplified Version of the System with Only the Content or Functionality You Find It Interesting?
No, no waySlightlySomewhatModeratelyYes, sure
GRMale5.911.811.829.441.2
Female7.753.838.5
DEMale 28.6 71.4
Female 33.366.7
Q20: Was the Feedback Provided “during” the Interaction Valuable to You?
No, not valuableSlightly valuableSomewhat valuableModerately valuableYes, very valuable
GRMale5.9 35.347.111.8
Female 7.723.146.123.1
DEMale 100
Female 66.733.3
Q21: Was the Feedback Provided “after” the Interaction Valuable to You?
No, not valuableSlightly valuableSomewhat valuableModerately valuableYes, very valuable
GRMale11.8 23.547.117.6
Female 15.4 53.830.8
DEMale 85.714.3
Female 33.333.333.3
Q22: Did the Feedback That Was Provided “during” the Interaction Help you to Become More Aware of Your Performance?
No, not at allSlightlySomewhatModeratelyYes, very much
GRMale5.929.417.623.523.5
Female7.7 15.453.823.1
DEMale 57.2 42.8
Female 66.733.3
Q23: Did the Feedback That Was Provided “after” the Interaction Help You to Become More Aware of Your Performance?
No, not at allSlightlySomewhatModeratelyYes, very much
GRMale11.85.935.329.417.6
Female7.7 7.761.523.1
DEMale14.314.3 71.4
Female 66.733.3

References

  1. Brett, J.M. Negotiating Globally: How to Negotiate Deals, Resolve Disputes, and Make Decisions Across Cultural Boundaries; John Wiley & Sons: Hoboken, NJ, USA, 2014; ISBN 9781118602614. (cloth/website). [Google Scholar]
  2. Yaghoubzadeh, R.; Pitsch, K.; Kopp, S. Adaptive Grounding and Dialogue Management for Autonomous Conversational Assistants for Elderly Users. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2015; Volume 9238, pp. 28–38. [Google Scholar]
  3. Muntean, C.H.; Andrews, J.; Muntean, G.-M. Final Frontier: An Educational Game on Solar System Concepts Acquisition for Primary Schools. In Proceedings of the 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), Timisoara, Romania, 3–7 July 2017; pp. 335–337. [Google Scholar]
  4. Zhao, D.; Chis, A.E.; Choudhary, N.; Makri, E.G.; Muntean, G.-M.; Muntean, C.H. Improving Learning Outcome using the NEWTON Loop Game: A Serious Game Targeting Iteration in Java Programming Course. In Proceedings of the EDULEARN19, Technologies Palma, Spain, 1–3 July 2019; pp. 1362–1369. [Google Scholar]
  5. Kaptein, M.; Markopoulos, P.; de Ruyter, B.; Aarts, E. Two acts of social intelligence: The effects of mimicry and social praise on the evaluation of an artificial agent. AI Soc. 2011, 26, 261–273. [Google Scholar] [CrossRef] [Green Version]
  6. Brett, J.; Thompson, L. Negotiation. Organ. Behav. Hum. Decis. Process. 2016, 136, 68–79. [Google Scholar] [CrossRef]
  7. Gratch, J.; DeVault, D.; Lucas, G.M.; Marsella, S. Negotiation as a Challenge Problem for Virtual Humans. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2015; Volume 9238, pp. 201–215. [Google Scholar]
  8. Cassell, J. Embodied conversational agents: Representation and intelligence in user interfaces. AI Mag. 2001, 22, 67. [Google Scholar]
  9. Zhou, M.X.; Mark, G.; Li, J.; Yang, H. Trusting Virtual Agents: The effect of personality. ACM Trans. Interact. Intell. Syst. 2019, 9, 1–36. [Google Scholar] [CrossRef]
  10. Thompson, L.; Nadler, J. Negotiating via Information Technology: Theory and Application. J. Soc. Issues 2002, 58, 109–124. [Google Scholar] [CrossRef]
  11. Low, P.K.; Ang, S. Information Communication Technology (ICT) for Negotiations. J. Res. Int. Bus. Manag. 2011, 1, 183–196. [Google Scholar]
  12. Lin, R.; Kraus, S. Can automated agents proficiently negotiate with humans? Commun. ACM 2010, 53, 78–88. [Google Scholar] [CrossRef]
  13. Chiu, M.; Kuo, S.W. From metacognition to social metacognition: Similarities, differences, and learning. J. Educ. Res. 2009, 3, 1–19. [Google Scholar]
  14. Ellis, R. Task-based language teaching: Sorting out the misunderstandings. Int. J. Appl. Linguist. 2009, 19, 221–246. [Google Scholar] [CrossRef]
  15. Aquilar, F.; Galluccio, M. Psychological Processes in International Negotiations; Springer: New York, NY, USA, 2008; ISBN 978-0-387-71378-6. [Google Scholar]
  16. Imel, S. Metacognitive Skills for Adult Learning. Trends and Issues Alert; No. 39; ERIC Clearinghouse on Adult Career and Vocational Education, ERIC Publications: Columbus, OH, USA, 2002. Available online: https://eric.ed.gov/?id=ED469264 (accessed on 29 July 2020).
  17. Hartman, H.J. Metacognition in Learning and Instruction; Neuropsychology and Cognition; Springer: Dordrecht, The Netherlands, 2001; Volume 19, ISBN 978-90-481-5661-0. [Google Scholar]
  18. Moely, B.E.; Mercer, S.H.; Ilustre, V.; Miron, D.; Mcfarland, M. Psychometric properties and correlates of the Civic Attitudes and Skills Questionnaire (CASQ): A measure of students’ attitudes related to service-learning. Mich. J. Community Serv. Learn. 2002, 8, 15–26. [Google Scholar]
  19. Drolet, A.L.; Morris, M.W. Rapport in Conflict Resolution: Accounting for How Face-to-Face Contact Fosters Mutual Cooperation in Mixed-Motive Conflicts. J. Exp. Soc. Psychol. 2000, 36, 26–50. [Google Scholar] [CrossRef] [Green Version]
  20. Armenakis, A.A.; Harris, S.G.; Mossholder, K.W. Creating Readiness for Organizational Change. Hum. Relat. 1993, 46, 681–703. [Google Scholar] [CrossRef]
  21. Reeves, B.; Nass, C. The Media Equation: How People Treat Computers, Television, and New Media; Cambridge University Press: Cambridge, UK, 1996; ISBN 157586052X. [Google Scholar]
  22. McRorie, M.; Sneddon, I.; McKeown, G.; Bevacqua, E.; de Sevin, E.; Pelachaud, C. Evaluation of Four Designed Virtual Agent Personalities. IEEE Trans. Affect. Comput. 2012, 3, 311–322. [Google Scholar] [CrossRef] [Green Version]
  23. Callejas, Z.; Ravenet, B.; Ochs, M.; Pelachaud, C. A Computational Model of Social Attitudes for a Virtual Recruiter. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, Paris, France, 5–9 May 2014; International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, USA, 2014; pp. 93–100. [Google Scholar]
  24. Core, M.; Traum, D.; Lane, H.C.; Swartout, W.; Gratch, J.; van Lent, M.; Marsella, S. Teaching Negotiation Skills through Practice and Reflection with Virtual Humans. Simulation 2006, 82, 685–701. [Google Scholar] [CrossRef]
  25. Alexandersson, J.; Aretoulaki, M.; Campbell, N.; Gardner, M.; Girenko, A.; Klakow, D.; Koryzis, D.; Petukhova, V.; Specht, M.; Spiliotopoulos, D.; et al. Metalogue: A Multiperspective Multimodal Dialogue System with Metacognitive Abilities for Highly Adaptive and Flexible Dialogue Management. In Proceedings of the 2014 International Conference on Intelligent Environments, Shanghai, China, 30 June–4 July 2014; pp. 365–368. [Google Scholar]
  26. Makri, E.; Spiliotopoulos, D.; Vassilakis, C.; Margaris, D. Human behaviour in multimodal interaction: Main effects of civic action and interpersonal and problem-solving skills. J. Ambient Intell. Humaniz. Comput. 2020. [Google Scholar] [CrossRef]
  27. Ding, D.; Burger, F.; Brinkman, W.-P.; Neerincx, M.A. Virtual Reality Negotiation Training System with Virtual Cognitions. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2017; Volume 10498, pp. 119–128. [Google Scholar]
  28. Chattaraman, V.; Kwon, W.-S.; Gilbert, J.E.; Ross, K. Should AI-Based, conversational digital assistants employ social- or task-oriented interaction style? A task-competency and reciprocity perspective for older adults. Comput. Hum. Behav. 2019, 90, 315–330. [Google Scholar] [CrossRef]
  29. Bickmore, T.; Cassell, J. Social Dialongue with Embodied Conversational Agents. In Advances in Natural Multimodal Dialogue Systems. Text, Speech and Language Technology; van Kuppevelt, J.C.J., Dybkjær, L., Bernsen, N.O., Eds.; Springer: Berlin/Heidelberg, Germany, 2005; Volume 30, pp. 23–54. [Google Scholar]
  30. Rosenthal-von der Pütten, A.M.; Straßmann, C.; Yaghoubzadeh, R.; Kopp, S.; Krämer, N.C. Dominant and submissive nonverbal behavior of virtual agents and its effects on evaluation and negotiation outcome in different age groups. Comput. Hum. Behav. 2019, 90, 397–409. [Google Scholar] [CrossRef]
  31. Kouroupetroglou, G.; Spiliotopoulos, D. Usability methodologies for real-life voice user interfaces. Int. J. Inf. Technol. Web Eng. 2009, 4, 78–94. [Google Scholar] [CrossRef] [Green Version]
  32. Veletsianos, G.; Russell, G.S. What Do Learners and Pedagogical Agents Discuss When Given Opportunities for Open-Ended Dialogue? J. Educ. Comput. Res. 2013, 48, 381–401. [Google Scholar] [CrossRef]
  33. Dunsworth, Q.; Atkinson, R.K. Fostering multimedia learning of science: Exploring the role of an animated agent’s image. Comput. Educ. 2007, 49, 677–690. [Google Scholar] [CrossRef]
  34. Li, K.C.; Lee, L.Y.-K.; Wong, S.-L.; Yau, I.S.-Y.; Wong, B.T.-M. Effects of mobile apps for nursing students: Learning motivation, social interaction and study performance. Open Learn. J. Open Distance e-Learn. 2018, 33, 99–114. [Google Scholar] [CrossRef]
  35. Kim, Y.; Baylor, A.L. Pedagogical Agents as Learning Companions: The Role of Agent Competency and Type of Interaction. Educ. Technol. Res. Dev. 2006, 54, 223–243. [Google Scholar] [CrossRef]
  36. Johnson, W.; Rickel, J.; Lester, J. Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments. Int. J. Artif. Intell. Educ. 2000, 11, 47–78. [Google Scholar]
  37. Henderson, M.; Huang, H.; Grant, S.; Henderson, L. Language acquisition in Second Life: Improving self-efficacy beliefs. In Proceedings of the ASCILITE 2009—The Australasian Society for Computers in Learning in Tertiary Education, Auckland, New Zealand, 6–9 December 2009; pp. 464–474. [Google Scholar]
  38. van der Meij, H. Do Pedagogical Agents Enhance Software Training? Hum. Comput. Interact. 2013, 28, 518–547. [Google Scholar] [CrossRef]
  39. Chae, S.W.; Lee, K.C.; Seo, Y.W. Exploring the Effect of Avatar Trust on Learners’ Perceived Participation Intentions in an e-Learning Environment. Int. J. Hum. Comput. Interact. 2016, 32, 373–393. [Google Scholar]
  40. Schwienhorst, K. Why Virtual, Why Environments? Implementing Virtual Reality Concepts in Computer-Assisted Language Learning. Simul. Gaming 2002, 33, 196–209. [Google Scholar] [CrossRef] [Green Version]
  41. Yung, H.I.; Paas, F. Effects of cueing by a pedagogical agent in an instructional animation: A cognitive load approach. Educ. Technol. Soc. 2015, 18, 153–160. [Google Scholar]
  42. Lester, J.C.; Converse, S.A.; Kahler, S.E.; Barlow, S.T.; Stone, B.A.; Bhogal, R.S. The persona effect: Affective impact of animated pedagogical agents. In Proceedings of the SIGCHI conference on Human factors in computing systems–CHI ’97, Atlanta, Georgia, 22–27 March 1997; ACM Press: New York, NY, USA, 1997; pp. 359–366. [Google Scholar]
  43. Holmes, J. Designing agents to support learning by explaining. Comput. Educ. 2007, 48, 523–547. [Google Scholar] [CrossRef]
  44. Mayer, R.; DaPra, C. An Embodiment Effect in Computer-Based Learning With Animated Pedagogical Agents. J. Exp. Psychol. Appl. 2012, 18, 239–252. [Google Scholar] [CrossRef] [PubMed]
  45. Guadagno, R.; Blascovich, J.; Bailenson, J.; McCall, C. Virtual Humans and Persuasion: The Effects of Agency and Behavioral Realism. Media Psychol. 2007, 10, 1–22. [Google Scholar]
  46. Tajfel, H.; Turner, J.C. An integrative theory of intergroup conflict. In Rediscovering Social Identity; Postmes, T., Branscombe, N.R., Eds.; Key readings in social psychology; Psychology Press: New York, NY, USA, 2010; pp. 173–190. ISBN1 978-1-84169-492-4. (Paperback); ISBN2 978-1-84169-491-7. (Hardcover). [Google Scholar]
  47. Turner, J.C. Towards a cognitive redefinition of the social group. Cah. Psychol. Cogn. Psychol. Cogn. 1981, 1, 93–118. [Google Scholar]
  48. Cameron, J.E.; Lalonde, R.N. Social identification and gender-related ideology in women and men. Br. J. Soc. Psychol. 2001, 40, 59–77. [Google Scholar] [CrossRef]
  49. Krämer, N.C.; Hoffmann, L.; Kopp, S. Know Your Users! Empirical Results for Tailoring an Agent’s Nonverbal Behavior to Different User Groups. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2010; Volume 6356, pp. 468–474. [Google Scholar]
  50. Crowelly, C.R.; Villanoy, M.; Scheutzz, M.; Schermerhornz, P. Gendered voice and robot entities: Perceptions and reactions of male and female subjects. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, 11–15 October 2009; pp. 3735–3741. [Google Scholar]
  51. Risse, T.; Demidova, E.; Dietze, S.; Peters, W.; Papailiou, N.; Doka, K.; Stavrakas, Y.; Plachouras, V.; Senellart, P.; Carpentier, F.; et al. The ARCOMEM Architecture for Social- and Semantic-Driven Web Archiving. Future Internet 2014, 6, 688–716. [Google Scholar] [CrossRef] [Green Version]
  52. Androutsopoulos, I.; Spiliotopoulos, D. Symbolic authoring for multilingual natural language generation. Methods Appl. Artif. Intell. 2002, 2308, 131–142. [Google Scholar]
  53. Spiliotopoulos, D.; Stavropoulou, P.; Kouroupetroglou, G. Acoustic rendering of data tables using earcons and prosody for document accessibility. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin/Heidelberg, Germany, 2009; Volume 5616 LNCS, pp. 587–596. [Google Scholar]
  54. Gefen, D.; Straub, D.W. Consumer trust in B2C e-Commerce and the importance of social presence: Experiments in e-Products and e-Services. Omega 2004, 32, 407–424. [Google Scholar] [CrossRef]
  55. Nowak, K.L.; Biocca, F. The Effect of the Agency and Anthropomorphism on Users’ Sense of Telepresence, Copresence, and Social Presence in Virtual Environments. Presence Teleoperators Virtual Environ. 2003, 12, 481–494. [Google Scholar] [CrossRef]
  56. Cerekovic, A.; Aran, O.; Gatica-Perez, D. Rapport with Virtual Agents: What Do Human Social Cues and Personality Explain? IEEE Trans. Affect. Comput. 2017, 8, 382–395. [Google Scholar] [CrossRef]
  57. Qiu, L.; Benbasat, I. Evaluating Anthropomorphic Product Recommendation Agents: A Social Relationship Perspective to Designing Information Systems. J. Manag. Inf. Syst. 2009, 25, 145–182. [Google Scholar] [CrossRef]
  58. Koryzis, D.; Samaras, C.V.; Makri, E.; Svolopoulos, V.; Spiliotopoulos, D. Visual Cue Streams for Multimodal Dialogue Interaction. In Advances in Human Factors, Business Management, Training and Education; Springer: Cham, Germany, 2017; Volume 498, ISBN 9783319420691. [Google Scholar]
  59. Gama, S.; Barata, G.; Gonçalves, D.; Prada, R.; Paiva, A. SARA: Social Affective Relational Agent: A Study on the Role of Empathy in Artificial Social Agents. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2011; Volume 6974, pp. 507–516. [Google Scholar]
  60. Schwarzer, R.; Jerusalem, M. Generalized Self-Efficacy scale. In Measures in Health Psychology: A User’s Portfolio. Causal and Control Beliefs; Weinman, J., Wright, S., Johnston, M., Eds.; NFER-NELSON: Windsor, UK, 1995; pp. 35–37. [Google Scholar]
  61. Schwarzer, R.; Diehl, M.; Schmitz, G.S. Self-Regulation. Available online: http://userpage.fu-berlin.de/~health/selfreg_e.htm (accessed on 1 May 2020).
  62. Vakola, M. What’s in there for me? Individual readiness to change and the perceived impact of organizational change. Leadersh. Organ. Dev. J. 2014, 35, 195–209. [Google Scholar] [CrossRef]
  63. Pennequin, V.; Questel, F.; Delaville, E.; Delugre, M.; Maintenant, C. Metacognition and emotional regulation in children from 8 to 12 years old. Br. J. Educ. Psychol. 2019, 90, 1–16. [Google Scholar] [CrossRef]
  64. Wilson, K.; Narayan, A. Relationships among individual task self-efficacy, self-regulated learning strategy use and academic performance in a computer-supported collaborative learning environment. Educ. Psychol. 2016, 36, 236–253. [Google Scholar]
  65. McQuiggan, S.W.; Lester, J.C. Diagnosing Self-efficacy in Intelligent Tutoring Systems: An Empirical Study. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2006; Volume 4053, pp. 565–574. [Google Scholar]
  66. Roll, I.; Aleven, V.; McLaren, B.M.; Koedinger, K.R. Metacognitive Practice Makes Perfect: Improving Students’ Self-Assessment Skills with an Intelligent Tutoring System. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2011; Volume 6738, pp. 288–295. [Google Scholar]
  67. Kraus, S. Automated Negotiation and Decision Making in Multiagent Environments. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2001; Volume 2086, pp. 150–172. [Google Scholar]
  68. Liew, T.W.; Tan, S.-M.; Jayothisa, C. The Effects of Peer-Like and Expert-Like Pedagogical Agents on Learners’ Agent Perceptions, Task-Related Attitudes, and Learning Achievement. J. Educ. Technol. Soc. 2013, 16, 275–286. [Google Scholar]
  69. Diehl, M.; Semegon, A.B.; Schwarzer, R. Assessing Attention Control in Goal Pursuit: A Component of Dispositional Self-Regulation. J. Pers. Assess. 2006, 86, 306–317. [Google Scholar] [CrossRef] [PubMed]
  70. von der Pütten, A.M.; Krämer, N.C.; Gratch, J. How Our Personality Shapes Our Interactions with Virtual Characters - Implications for Research and Development. In Lecture Notes in Computer Science; Springer Science+Business Media: Berlin/Heidelberg, Germany, 2010; Volume 6356, pp. 208–221. [Google Scholar]
  71. Kocaballi, A.B.; Berkovsky, S.; Quiroz, J.C.; Laranjo, L.; Tong, H.L.; Rezazadegan, D.; Briatore, A.; Coiera, E. The Personalization of Conversational Agents in Health Care: Systematic Review. J. Med. Internet Res. 2019, 21, e15360. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  72. Rahim, M.A.; Katz, J.P. Forty years of conflict: The effects of gender and generation on conflict-management strategies. Int. J. Confl. Manag. 2019, 31, 1–16. [Google Scholar] [CrossRef]
  73. Baarslag, T.; Kaisers, M.; Gerding, E.H.; Jonker, C.M.; Gratch, J. When Will Negotiation Agents Be Able to Represent Us? The Challenges and Opportunities for Autonomous Negotiators. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia, 19–25 August 2017; pp. 4684–4690. [Google Scholar]
Figure 1. The multimodal embodied conversational agents (ECA; negotiator) and the multi-issue negotiation environment [26].
Figure 1. The multimodal embodied conversational agents (ECA; negotiator) and the multi-issue negotiation environment [26].
Information 11 00381 g001
Figure 2. Self-efficacy, civic action and individual readiness to change before and after interaction.
Figure 2. Self-efficacy, civic action and individual readiness to change before and after interaction.
Information 11 00381 g002
Table 1. Aggregated metacognitive skill training variable correlations.
Table 1. Aggregated metacognitive skill training variable correlations.
MeanSD1234567891011
1. Self-efficacy before3.090.39(0.79)
2. Self-efficacy after3.300.450.82 **(0.86)
3. Self-regulation after2.910.420.53 **0.60 **(0.67)
4. Interpersonal and Problem-solving Skills before4.130.480.39 *0.52 **0.41 *(0.79)
5. Interpersonal and Problem-solving Skills after4.170.530.65 **0.78 **0.52 **0.76 **(0.86)
6. Civic Action before3.450.90−0.090.090.350.290.11(0.92)
7. Civic Action after3.591.040.010.260.45 *0.320.260.92 **(0.94)
8. Individual Readiness to Change before4.920.700.36 *0.53 **0.62 **0.340.45 *0.350.38 *(0.74)
9. Individual Readiness to Change after5.120.780.290.54 **0.69 **0.38 *0.58 **0.48 **0.58 **0.77 **(0.80)
10. Mastery Goal Orientation before4.310.550.36 *0.60 **0.310.57 **0.67 **0.200.350.42 *0.37 *(0.85)
11. Mastery Goal Orientation after4.340.620.48 **0.69 **0.47 **0.66 **0.72 **0.110.290.43 *0.39 *0.91 **(0.89)
Notes: * p < 0.05, ** p < 0.01 alpha coefficients are presented on the diagonal (before-after interaction with the system), N = 30 (Greek users).
Table 2. Paired samples t tests.
Table 2. Paired samples t tests.
Sig. (2-Tailed)
1. Self-Efficacy before–Self-Efficacy aftert (28) = −4.60, p < 0.001
2. Civic Action before–Civic Action aftert (28) = −1.75, p < 0.01
3. Individual Readiness to Change before–Individual Readiness to Change aftert (28) = −1.78, p < 0.01
Table 3. Hierarchical regression analysis predicting self-efficacy before interaction.
Table 3. Hierarchical regression analysis predicting self-efficacy before interaction.
Self-Efficacy Before
βR2ΔR2
Step 1: Control variables
Gender−0.18
Step 2: Main effects
Self-regulation before0.050.25 *0.10 *
Interpersonal and problem-solving skills before0.52 **
Civic Action before−0.39 *
Notes: * p < 0.05, ** p < 0.01 (one-tailed), N = 40.
Table 4. Hierarchical regression analyses predicting self-efficacy after interaction.
Table 4. Hierarchical regression analyses predicting self-efficacy after interaction.
Self-Efficacy After
βR2ΔR2
Step 1: Control variables
Gender−0.06
Step 2: Main effects
Self-regulation after0.210.52 ***0.29 ***
Interpersonal and problem-solving skills after0.59 ***
Notes: *** p < 0.001 (one-tailed), N = 40.
Table 5. Distribution (%) of user perceptions regarding metacognitive training, system scope and suggestions for future improvement (N = 40).
Table 5. Distribution (%) of user perceptions regarding metacognitive training, system scope and suggestions for future improvement (N = 40).
1Are you familiar with metacognitive skills training?YesNoI do not really know what metacognitive skills are
17.537.545.0
2What would you say the system is about?Metacognitive skills trainingNegotiation skills trainingAI skills trainingDo not know
10.047.5402.5
3What is your view on the current functionalities and suggestions for future versions?Features add-onLearning Progress Feedback
67.532.5

Share and Cite

MDPI and ACS Style

Spiliotopoulos, D.; Makri, E.; Vassilakis, C.; Margaris, D. Multimodal Interaction: Correlates of Learners’ Metacognitive Skill Training Negotiation Experience. Information 2020, 11, 381. https://0-doi-org.brum.beds.ac.uk/10.3390/info11080381

AMA Style

Spiliotopoulos D, Makri E, Vassilakis C, Margaris D. Multimodal Interaction: Correlates of Learners’ Metacognitive Skill Training Negotiation Experience. Information. 2020; 11(8):381. https://0-doi-org.brum.beds.ac.uk/10.3390/info11080381

Chicago/Turabian Style

Spiliotopoulos, Dimitris, Eleni Makri, Costas Vassilakis, and Dionisis Margaris. 2020. "Multimodal Interaction: Correlates of Learners’ Metacognitive Skill Training Negotiation Experience" Information 11, no. 8: 381. https://0-doi-org.brum.beds.ac.uk/10.3390/info11080381

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop