An Exploratory Study on the Validation of THUNDERS: A Process to Achieve Shared Understanding in Problem-Solving Activities

Abstract

The complexity in collaborative work is mainly related to the difficulty in social interaction, which generates low levels of understanding among participants about what they should do and about the problem to be solved, resulting in problems in the motivation to generate true collaboration. Therefore, in the search to improve collaborative work and encourage this collaboration, it is necessary to implement strategies that promote the construction of shared understanding and obtain better group results. However, building it becomes a challenge due to the factors that influence it and how little is known about its construction. In this sense, to improve collaborative work, as a result of a research process, the THUNDERS process is proposed, which provides a set of elements to build shared understanding in problem-solving activities and with heterogeneous group formation. Specifically, this paper presents the results of the statistical validation of THUNDERS through the Student’s t-test, which was used in an exploratory study in the educational field in two Colombian universities, where learning styles were considered for the formation of groups; having groups that used the process and other control groups that did not use it, the collaborative activity consisted of determining the scope of a process line simulating a software development company. According to the results obtained in the context of this study, it can be considered that THUNDERS encourages and improves shared understanding when people in a group work collaboratively to solve a problem. In addition, elements for improvement were identified that should be incorporated in further stages of this research so that the process allows for an easy and guided construction of shared understanding in any application context.

1. Introduction

The importance of teamwork arises from the consideration that the greater the number of participants committed to carrying out an activity, the greater the quantity and quality of the obtained results [1], alongside considering that working with other people can achieve better results than working individually since it is gathering and complementing a group of people with different capabilities, abilities, ideas, and skills regardless of the context in which the collaborative activity is performed [2]. That is why today the concept is related to group work dynamics in different contexts: work, study, and family, thus obtaining the benefits of achieving common goals, whether corporate, learning, or simply as an activity of social interaction [3].

Achieving collaboration improves the way a team works together and solves a problem, leading to greater innovation, efficient processes, more success, and better communication [4]. In order to collaborate, group members have permanent interactions with shared goals, a high degree of negotiation, interactivity, and interdependence, to define and assign tasks, and exchange ideas or points of view [5]. Therefore, it is necessary to establish social rules to organize a collaborative activity [6]. However, while the idea of collaboration seems easy enough, in reality, it can be quite complex working with others. Each person on a team has strengths and weaknesses, communication preferences, and personal goals [7]. When not handled correctly, these can fundamentally affect the way of working, generating communication difficulties between people and, indeed, causing the group to fail to agree on the interpretation of the tasks they will carry out, what they are working on, how they will work together, and the topic of the activity—in other words, failures in shared understanding, which consequently generate low levels of collaboration [8]. In this sense, the scant understanding among group participants about what they should do in the activity and about the problem to solve generates little motivation to collaborate, causing tasks to be performed without the necessary coordination [9]. Shared understanding can be defined as the level at which the members of a group who work on a particular issue agree on the perspectives and interpretations on that issue, reaching mutual agreements, and in this way, are able to act in a coordinated manner [10]. For this reason, since a group that works in a collaborative activity must achieve the objective and solve the proposed problem, it is necessary to build within the group a shared understanding [11] of the content, the procedure, and the use of communication technologies that will be used between them. This is done in order that all the participants know where they are going and in this way act in coordination towards the same goal [12]. For this, it is necessary to find the application of techniques, strategies, activities, and processes that support the construction of a shared understanding in specifically heterogeneous groups, where it is expected that these groups achieve efficiency and effectiveness in their work, producing better group results [13]. The importance of having a shared understanding in this type of organization of groups, specifically heterogeneous, is that shared understanding improves their performance because everyone knows what they should do and speaks the same language. However, its construction is a challenge, especially in the early stages of defining the problem [14], and more, in this type of group [13,15], because due to their varied skills, knowledge, experiences, personalities, or educational levels, the group members perhaps use the same words for different concepts or different words for the same concepts without realizing it [16]. Differences in interpretations and meanings assigned to key concepts can interfere with the productivity of collaborative work if these are not clarified early on [17,18,19], in addition to the fact that very little is known about what leads to its construction [13]. Hence there is a need to have a shared understanding among the members of the group and to know how to build it. If it is sought to create a shared understanding in a collaborative activity, this must operate on how a person works alone and how the group works to solve the activity [10]. It cannot simply bring people together in a room, hand them the need to solve something, and wait for them to create a shared understanding without support. For this, they must instead have guided work processes that unite people continuously, to share their understanding of what is happening, and the activity being carried out, in addition to generating a debate to show disagreements, different points of view, improve uneven understandings, and finally reach a common point, where everyone participates and agrees [20].

According to the main problem of collaborative work presented above, which is summarized in the difficulty of obtaining collaboration by all participants to achieve the common goal, the objective of this research is to define a process that improves collaborative work through the construction, monitoring, and assistance of shared understanding, which will improve communication among participants and as a consequence will result in greater collaboration. Therefore, in the search to build a process that meets these needs, this paper presents the results of the statistical validation, using the Student’s t-test, in the execution of the process known as THUNDERS (CollaboraTive work through sHared UNDErstanding in pRoblem-solving activitieS) as a proposal for the construction of shared understanding in this type of activity and for heterogeneous groups. The validation was conducted by carrying out an exploratory study with groups of students from two universities, who had to carry out a collaborative activity to solve a problem associated with defining the scope in software process lines. For the design, execution, and analysis of the results obtained from the collaborative activity, THUNDERS was used. THUNDERS is a second version of the process presented in [21], which has been improved and complemented thanks to the results obtained from validation of the initial version. The process presented here is based on guidelines with which to guide the entire collaborative process in such a way that it can be ensured that from the design of the activity a shared understanding is built into its execution which thus allows the resolution of the problem. It is a process that is based on having elements of individual construction, collaborative construction of meaning, collaborative construction, and constructive conflict resolution, applying a collaborative engineering approach. In addition, this second version includes the complete detail of each phase, activity, tasks, steps, and necessary work products, including monitoring elements. Specifically, in this paper, the results obtained from the validation of THUNDERS are presented in more detail. In the context of validation of this exploratory study, it can be considered that THUNDERS, the proposed process, encourages and improves shared understanding where a group of people work on the resolution of collaborative activity. In addition, aspects for improvement were identified that should be incorporated in later stages of this research so that the process allows easy construction and measurement of shared understanding in heterogeneous groups. This is because it is still a complex process to apply in collaborative activity. It is necessary to include more support with guides and different elements that support its easier execution.

The main importance of the research and validation presented here is to show the need to analyze and investigate cognitive factors that influence collaborative work and require assistance and guidance for its construction and measurement during the execution of the collaborative activity, in order to take action in time to improve communication among participants and consequently achieve greater collaboration to solve the problem and achieve the objectives set. In addition to this, the results obtained here are empirical evidence of the use of the process in a real academic context that served as a basis for its validation and subsequent improvement.

This document is structured as follows: Section 2 describes the theorical framework with some important concepts and related work; Section 3 briefly defines the THUNDERS process with each of its elements; Section 4 contains the planning, execution of the exploratory study, the results obtained, and their discussion; Section 5 contains the threats from this work; and finally, Section 6 contains the conclusions, limitations, and future work.

2. Theoretical Framework

2.1. Concepts

2.1.1. Collaborative Work

This refers to the participation of a group of people who contribute their ideas and knowledge in order to achieve a common goal [22]. It also involves direct interaction between individuals to produce a product and involves negotiations, discussions, and reaching consensus on the perspectives or ideas of others to obtain the expected results [23].

2.1.2. Shared Understanding

Christiane Bittner et al. [15] defined shared understanding as “The ability to coordinate behaviors towards common goals or objectives (“meaning-in-use” or action perspective) of multiple agents within a group (group level) based on mutual knowledge, beliefs and assumptions (content and structure) about the task, the group, the process or the tools and technologies used (object scope/perspective) that may change throughout the group work process due to various influencing factors and impacts the processes and outcomes of group work”.

2.1.3. Heterogeneous Group

Heterogeneous grouping is a type of membership distribution in which there is a relatively even distribution of people with different characteristics, such as intellectual abilities, different emotional needs, varying ages, educational levels, interests, and special needs [24]. It is necessary to bring mixed ability groups together systematically to ensure a truly heterogeneous composition [25]. In this sense, the idea of collaborative heterogeneous group work is to design group work that requires the multiple skills of group members to solve, manage, and complete a complex task, which has the uncertainty of a challenge [26] and needs the contributions of all group members [27].

To form this type of group, a single characteristic of the participants can be chosen, and each group must have different values of that characteristic, or several characteristics can be chosen, and groups can be formed with the required variety [24].

2.1.4. Problem-Solving Activities

The specific type of activities that THUNDERS is expected to perform are problem-solving activities. In this sense, problem solving is the act of defining a problem, determining its cause, identifying, prioritizing, and selecting alternatives for a solution, and implementing it [28]. Collaborative problem solving (CPS) is defined as “activities in which a problem must be solved involving collaboration among a group of people” [29].

A problem can be of several types [28]:

• Theoretical: The purpose is to reflect on a topic or knowledge.
• Practical: With objectives aimed at progress and innovation.
• Theoretical–practical: To obtain unknown information in the solution of practical problems.

In THUNDERS collaborative problem-solving activities can specify the problem in any context and on any topic, as long as the support of a group of people is needed to identify the causes of the problem, look for possible solutions, choose one of them and implement it, and finally verify that the solution does solve the problem, all this with the collaboration of the group members. The problem can be as simple or complex as desired. For example, if we talk about a simple problem, we have information about the routes and train schedules between two points; the problem is to find out which is the fastest route to make a particular trip, where group members must help each other to find this faster route.

2.2. Related Work

2.2.1. Works Related to Shared Understanding

The objective of [30] was to find out how software developers and designers achieve shared understanding. For this a case study was conducted by interviewing employees of a Finnish ERP product development company to answer the following questions: How do developers and designers achieve shared understanding of the UX of a software system under development? and What are the artifacts used in their collaboration? Similarly, in [31], study, construction, mutual support, cohesion, and constructive conflict were major factors underlying shared understanding in emerging technology (ET) firms; furthermore, we explicitly examined the relationship between shared understanding, shared knowledge quality, and team effectiveness. A mixed methods study was conducted with a valid sample collected from 52 ET firms in co-working spaces located in Taiwan and Vietnam. Moreover, [32] discussed what shared understanding is, its importance, and the positive implications of having it in team sports such as soccer. As a result, practical suggestions were offered on how coaches can achieve shared understanding in teams. In another context [33], in a study involving interprofessional emergency medical teams (emergency medicine residents, nurses, and medical students), resuscitation simulations were carried out, recorded, and coded independently. This study made it possible to measure the team’s perception of the shared understanding according to the information provided and a measurement of the effectiveness of the team leader. Finally, Bates et al. [34] developed and validated a perception questionnaire as a tool to evaluate shared clinical understanding. A questionnaire was validated among physicians of pediatric cardiology patients that determined whether or not according to the critical and objective evaluation of the physicians there was a shared understanding among these physicians on providing patient information.

2.2.2. Works Related to Shared Understanding in the Context of Education

Faculty in health-related academic programs run the risk of misunderstanding when terms and concepts are not clearly defined, which hinders progress in the field. In [35] a methodology was defined based on a template of terms that emerged from a research process where the varied use of terms in biomedical and health profession educational literature contexts was identified and described. The methodology aimed to facilitate clarity of definitions and create a terminology and approach to describing terms to provide shared understanding. In turn [36] used empirical data obtained in a Master’s level business course to show how a responsible mindset can be facilitated among students in business programs by creating a shared understanding of the achievement of the SDGs. Specifically, the work inquires how to facilitate the creation of a shared understanding for a responsible mindset in higher business education. Moreover, in the design area, one of the challenges for students is to learn to work with interactive and interaction technology, since they have to assimilate knowledge about interactive prototypes, to work in teams, as well as many other things, and in a limited time. In this sense, [37] presented a technique called “Acting Machines” that allows a team to collaboratively develop an interactive prototype. The technique is based on the Acting Machine diagram, a visual representation of programmed interactions, which supports shared understanding that links conceptual stages with those of prototype implementation. The work described in [38] explored the shared understanding of how to create a sense of community in large class sizes, where techniques, recommendations, and suggestions for actively creating a sense of community in classrooms of more than 200 students are shared.

2.2.3. Works Related to Shared Understanding in Empirical Studies

In [39] a case study was conducted in three small organizations using continuous software engineering (CSE) to understand and identify the factors that prevent shared understanding of non-functional requirements (NFR) and their relationship to rework. Using a mixed methods approach, the following factors were identified: lack of domain knowledge, rapid pace of change, and communication problems. Some strategies were also identified to mitigate the lack of shared understanding through more effective management of requirements knowledge in such organizations. In the same vein Werner in [40] presents a dissertation on the basis of a theory of the complex and intricate relationship between shared understanding of NFRs and CSE. In [41], how the distribution of the team influences the success of projects by using a shared understanding approach was addressed. An empirical study was carried out in a software product development company in which a quantitative survey design was used, complemented by seven semi-structured interviews. It was found that the level of distribution of the team does not significantly influence the shared understanding of the success of the project. Dossick et. al. in [42] showed the first findings of an empirical study that sought to explore the use of photo elicitation techniques in combination with ethnography to assess the amount of shared understanding in multidisciplinary teams working on a building design project. The study was conducted with students in an interdisciplinary project-based class in which the interactions among students of architecture, construction science, and construction management were studied, and the visualizations these students created and used in order to learn and develop integrated skills. Results from two studies are shown in [43]: an experiment with students and a pilot field study with practitioners using a content validity survey instrument to measure shared understanding in businesses and IT professionals that aimed to monitor the relationships between businesses and the IT units of their organizations.

The first part of this section describes those works in which shared understanding is studied in a general way in different application contexts, the second part focuses on showing related works in the academic context, and the final part of the section describes some empirical studies where shared understanding is used, but it is striking that none of these studies used or defined a process that establishes, step by step, what to do and how to build shared understanding, as is the case of THUNDERS, which is intended to serve as a guide to support the entire collaborative process, from the design of the collaborative activity to the analysis of the results obtained, including elements to achieve a shared understanding specifically in problem-solving activities with heterogeneous groups. It is also important to clarify that this paper focuses mainly on statistical analysis as a result of the validation of THUNDERS in an exploratory study, to determine whether it encourages and improves shared understanding in the context used.

3. THUNDERS in a Nutshell

3.1. THUNDERS Construction

Research into building a process for the construction of shared understanding in collaborative problem-solving activities, specifically with the formation of heterogeneous groups, is research that has been carried out previously. Initially, consider the work defined by Collazos et al. in [30], who divided the computer-supported collaborative learning process into three phases, pre-process, process, and post-process, with some activities and tasks, but for the specific context of learning.

A first version of the process [21] was defined, based on this work, for collaborative work in a variety of contexts and not only in education. For this purpose, the pre-process phase and some activities of the process were updated and adapted, and some elements for the construction of shared understanding were incorporated. These elements were incorporated from the analysis of the information previously obtained in a systematic review of the literature, which sought to characterize and identify the definitions, approaches, and importance of the construction of the existing and proposed shared understanding in the literature, considering the activities of planning, execution of the review, and analysis of the information, as established by Kitchenham et al. [44]. After the execution of the review, a total of 273 papers were obtained, which were subjected to a review and filtering process, considering the research questions, and the inclusion and exclusion criteria previously defined, leaving 21 papers, which were defined as primary papers.

Considering the work that served as the basis and presented in [45] and from these primary papers (mainly from [12,13,15,46,47,48,49]), elements were taken for the construction of the shared understanding, in addition to defining and detailing some elements that were identified as necessary for the specification of the defined phases; the heterogeneous formation of the groups was considered as well as the need to consider that the collaborative activities to be executed should be problem-solving activities; and with this, the components that would make up the process in its initial version were established.

With this initial version, an experiment was conducted that identified some elements to improve and correct in the process. With these results of the experiment, a second version was created, which we called the THUNDERS process, which incorporates the definition and detail of all the phases of the process, monitoring elements, measurement of shared understanding, and measurement of the performance of group collaboration.

Figure 1 below shows the differences between the two versions, with the elements that were defined for each.

In light of this, it is important to make it clear that for this work, a process is a sequence of steps arranged with some type of logic that focuses on achieving a specific result [50]. In this work, the process that we will obtain will be a process defined at the conceptual level. Specifically, THUNDERS is a process that contains phases, activities, tasks, steps, guidelines, work products, roles, and workflows to support the execution of collaborative activities in problem solving and, at the same time, supports the achievement of a shared understanding.

3.2. THUNDERS Phases

As indicated, THUNDERS features three phases: pre-process, process, and post-process. The importance of dividing THUNDERS into three phases is that each phase has a milestone that defines its end and prepares the start of the next. For the pre-process phase, the milestone is the complete design of the collaborative activity; for the process phase, it is the solution of the problem, and finally, for the post-process phase, it is the execution of the mechanism of individual and group performance evaluation of the participants in the activity.

The first phase of pre-process aims to design and define the collaborative activity, with each of its necessary elements, such as activities for the definition of the topic, definition of the problem to be solved, definition of the characteristics of the heterogeneous training, design of roles, complete design of the collaborative activity, definition of the objectives of the activity, definition of the rules, success criteria, design of the necessary material, and design of the methods of verification of compliance and shared understanding (See Figure 2). Each of these activities has tasks, steps, inputs, outputs, and work products that allows each specified activity to be fulfilled.

The objective of each of the activities of the pre-process phase is detailed below (

Table 1. Objective of each of the pre-process activities.

Activity	Objective
Define population	Define the characteristics of the groups that will carry out the collaborative activity. Determine the procedures that will make it possible to identify these characteristics in each participant. Group the characteristics heterogeneously.
Design roles	Design the role that each participant must fulfil, with the respective responsibilities and duties.
Define pre-conditions	Determine the knowledge, skills, and abilities that the participants will need to execute the collaborative activity according to the topic and the problem to be solved.
Decisions on grouping of members	Organize the groups that will carry out the proposed activity, defining their size, the distribution, and selection of the participants that will form them, considering the characteristics defined previously for forming heterogeneous groups. In this task, it is defined to which group each participant belongs.
Design problem-solving activity	Guide and structure the design of the collaborative activity, determining each of the tasks that must be carried out to solve the problem among the participants.
Define objectives	Clearly define the objectives to be achieved with the problem-solving activity and with the participation of each member of the group, in addition to determining the expected results or products.
Define success criteria	Define criteria, conditions, or requirements that will make it possible to determine when the problem-solving activity will end successfully.
Specify rules of activity	Define the rules, restrictions, norms, and conditions that will allow controlling behaviors, communication, debates, dates, start and end of each task, and elements of the activity to promote the orderly and controlled development of the activity.
Design materials	Select existing materials or design new ones necessary to support participants in carrying out the activity and to encourage them to work collaboratively. Define the allocation strategy for these materials.
Design verification methods	Define the strategy, criteria, and mechanisms that will be used to verify compliance with the problem, the construction of shared understanding, and the performance of the participants.

), in order to learn more about what was actioned in this phase to achieve the design and specification of the collaborative activity.

The process phase aims to execute the collaborative activity and solve the problem. Initially, the design of the activity defined in pre-process is socialized, with its respective problem to be solved, rules, success criteria, objectives, and verification methods. This phase also encompasses creation of the groups, with the assignment of their respective roles and responsibilities, and assignment of the material to the groups. Following socialization, it is determined if shared understanding was constructed and, whenever this does not happen, the necessary actions are taken to guide the group and clearly explain what they must do and must solve. With this information clear, the activity is carried out. With the actions to solve the problem, debate is generated and finally, the solution to the problem posed is obtained. Each of the activities in this phase are shown in Figure 3.

The objective of each of the activities of the process phase is detailed in

Table 2. The objective of each of the process activities.

Activity	Objective
Organization	Execute the organization of the groups, assignment of roles, socialization of the activity with its information and assignment of material, in order to organize the necessary elements to begin carrying out the collaborative activity.
Shared understanding	Ensure group members agree and understand the activity to be carried out, through individual understanding and socialization, after a debate, for a group understanding that will allow coordinated work.
Collaborative activity	Execute the tasks of the collaborative activity to obtain the solution to the problem, with the support of all the members of the group.
Collaborative knowledge building	Generate a product (document, map, or graph) with the support of all members of the group, which will allow formalizing the implementation of the solution to the problem.

to understand what was actioned in this phase to execute the activity designed in the previous phase and achieve the solution of the problem and the goal set in the activity.

The post-process phase aims to verify the resolution of the collaborative activity problem, where it is initially verified that the success criteria were met and subsequently the performance of the participants and the result obtained by the group are quantitatively defined. Each of the activities in this phase are shown in Figure 4.

The objective of each of the activities of the post-process phase is detailed in

Table 3. The objective of each of the post-process activities.

Activity	Objective
Review success criteria	Execute and apply the strategy, criteria, and mechanisms to verify compliance with the problem, evaluating the achievement of the defined objectives and verifying that the problem was solved.
Summative evaluation	Execute and apply the strategy and mechanisms to verify the individual and group performance of the participants to achieve the objectives of the activity and the solution of the problem within each group.

, to understand what was actioned in this phase to verify that after the execution of the activity, the problem and the objectives of the collaborative activity have been solved, in addition to obtaining a summative evaluation that allowed obtaining quantitative results on the results obtained.

THUNDERS can be considered an extension and adaptation of the model proposed by Collazos et al. [44], which defines a guide for the design of collaborative learning activities: it is an extension because new activities are included and new tasks, new roles, and work products are defined to seek to build shared understanding through the execution of the process; and it is an adaptation because the work of Collazos et al. [44] is focused solely on carrying out collaborative learning activities while THUNDERS seeks to be a process to carry out problem-solving activities in collaborative work.

3.3. THUNDERS Roles

Each phase contains a definition of roles, which shows who is in charge of executing the THUNDERS tasks and who is responsible for obtaining the respective work products. A role can be executed by one or several people depending on the complexity of the collaborative activity. For the pre-process phase, there is the Activity Designer who is in charge of defining the outline and details of the collaborative activity to meet the objectives and solve the problem. This role must also ensure that the design is consistent with the participants, problem, and objectives chosen for its subsequent implementation. For the process phase, the Activity Leader is in charge of executing, managing, monitoring, and coordinating the activities, tasks, and steps of the process so that they are carried out within the stipulated time and design. They are also in charge of analyzing the information obtained in order to make the pertinent decisions required. In this phase, there are also the Participants, who are in charge of executing the collaborative activity and solving the problem. Finally, in the post-process phase, there is the Coordinator of the activity, who is in charge of verifying at the end of the collaborative activity that the previously defined success criteria were met and of defining the performance of the participants.

4. Exploratory Study

4.1. Exploratory Study in a Nutshell

Initially, it is necessary to determine that exploratory studies are normally carried out when the objective is to examine a topic or research problem that has been little studied or that has not been addressed before. That is, when the literature review reveals that there are only unresolved guidelines and ideas vaguely related to the problem. This study type serves to increase the degree of familiarity with relatively unknown phenomena, to obtain information on the possibility of carrying out further complete research about a particular context [51]. That is why it was decided to carry out a study of this type, to have more familiarity with the context and to find flaws and opportunities for improvement in the defined process in its second version. This is because similar processes have not been found that will determine what to do, how to do it, and how to guide a collaborative activity from its design in order to build shared understanding and therefore achieve the necessary collaboration and obtain the results expected.

4.2. Context of the Exploratory Study

THUNDERS aims to be a process for collaborative work that can be used in any context. In this case, for the exploratory study, the context in which it was conducted refers to a university environment. This is due to the ease of access to participants and the time they can devote to the execution of the different tasks, without neglecting that the activity to be developed is one that simulates a collaborative activity in a software development company.

In this study, 75 students from the Universidad del Cauca (UC) in Colombia participated; all of them were systems engineering students with 37 on the software engineering I course in the fifth semester (7 women and 30 men) and 38 students on the software engineering II course in the sixth semester (10 women and 28 men). In addition, 75 students from the Corporación Universitaria Comfacauca, Unicomfacauca (UF) in Colombia participated; all of them were systems engineering students with 27 on the software engineering I course in the fifth semester (10 women and 17 men) and 48 students on the software engineering II course in the sixth semester (15 women and 33 men).

With these students, groups of five participants were formed, resulting in 15 groups in each university. For this study, the learning styles characteristic was selected, which refers to the different ways that individuals have of using their intelligence, or, how they prefer to use their abilities to learn some concept or experience [52]. With this, groups were formed in each university using a software tool called Collab [53] that analyzes learning styles and organizes the group heterogeneously using a genetic algorithm described in [54]. The characteristic of learning styles ensures that heterogeneous groups of students are formed, allowing them to complement each other in each group and thus obtain better results. To identify the predominant style of each participant, each participant filled out a questionnaire, and subsequently, based on these results, Collab formed the groups heterogeneously.

4.3. Design of the Exploratory Study

For the execution of the exploratory study, in UC the THUNDERS process was used to define, execute, and verify the collaborative activity, while UF was used as a control group, where each of the groups formed carried out the same collaborative problem-solving activity but without employing THUNDERS. In UF, only the formation of heterogeneous groups was carried out. From there it was left to both teacher and participants to define, execute, and analyze the results of the collaborative activity according to their knowledge and experience.

The problem-solving activity, which the groups from both universities had to solve, specifically in the field of software engineering, consisted of each group assuming that they were part of the process engineering team of a software development company, where they had to establish the development processes that best adapt and support the projects in the company, thus defining the scope of a process line.

It was to be considered that a new project was coming to the company in the domain of banking applications, characterized as follows: a project which needed to be developed entirely in a single location of the company and should have international cooperation; it should be in accordance with international quality standards since it was considered as a large product in which the client would not be involved as part of the development team. With this information, each group had to follow an execution guide called SpeTion-SPrl, following a set of templates and formats that they had to develop collaboratively, in order to solve the problem of the activity, which was mainly to determine the scope of a software process line so that it could adapt and support the projects that the company was currently handling.

Previously, a software tool called MEPAC was developed internally, which served as support to follow the THUNDERS process at UC [55]. Through forms, MEPAC provides the step-by-step to follow each of the activities, tasks, and necessary steps of the process.

The aim of the exploratory study was to inquire about the construction of shared understanding using THUNDERS in a problem-solving activity. The research question of the study was determined as follows: Does the THUNDERS process promote and enhance shared understanding in a problem-solving activity? Considering the exploratory study goal and the research question, the following hypotheses were evaluated:

• The THUNDERS process encourages the construction of shared understanding in a problem-solving activity.
• The THUNDERS process improves the construction of shared understanding in a problem-solving activity.

In order to validate these hypotheses, the following variables were considered:

• Improvement in the group descriptions: this is the level of improvement found in the descriptions after the execution of the process. This variable represents the statistically significant difference between the notes that were given to the individual and group descriptions.
• Improvement in the UC and UF descriptions: this is the level of improvement found in group descriptions after the execution of the process. This variable represents the statistically significant difference in the notes given to the group descriptions between UC and UF groups.
• Understanding other descriptions: this comprises the level of understanding that a participant has given the descriptions of what they should do from the activity, from other group participants. This variable represents a perceptual judgment of understanding of other descriptions.
• The opinion of other descriptions: this is the level of opinion that a participant has with the descriptions of other group participants, of what they should do with the activity. This variable represents a perceptual judgment of opinion of other descriptions.
• Improvement in homogeneous understanding: this is the level of improvement in the homogeneous understanding of the group of the activity to be undertaken, after use of the proposed process. This variable represents the statistically significant difference between the perceptual judgment of homogeneous understanding before and after the use of the proposed process.
• Improvement in the discrepancy: the level of improvement in the discrepancy by each participant compared to other participants about the activity, after the use of the proposed process. This variable represents the statistically significant difference between the perceptual judgment of discrepancy before and after the use of the proposed process.
• Improvement in homogeneous understanding in UC and UF: this is the level of improvement in the homogeneous understanding of the group of the activity to be developed, after the use of the process. This variable represents the statistically significant difference of the perceptual judgment of homogeneous understanding between the UC and UF groups.
• Improvement in the discrepancy in UC and UF: this is the level of improvement in the discrepancy by each participant compared to other participants about the activity, after the use of the process. This variable represents the statistically significant difference of the perceptual judgment of discrepancy between the UC and UF groups.
• Improvement in the construction activity: this is the level of improvement in the construction activity results, following use of the process. This variable represents the statistically significant difference of the perceptual judgment of construction activity results between the UC and UF groups.
• Improvement in the co-construction activity: this is the level of improvement in the co-construction activity results, after the use of the process. This variable represents the statistically significant difference of the perceptual judgment of co-construction activity results between the UC and UF groups.
• Improvement in the constructive conflict activity: the level of improvement in the constructive conflict activity results, after the use of the process. This variable represents the statistically significant difference of the perceptual judgment of constructive conflict activity results between the UC and UF groups.
• Improvement in the quality of the results: this is the level of improvement in the quality of the final results obtained in performing the problem-solving activity, after the use of the process. This variable represents the statistically significant given difference of scores to the final results between the UC and UF groups.
• Improvement in perception about the achievement of the objectives: this is the level of improvement in the perception of the participants, about the achievement of the objectives, after the use of the process. This variable represents the statistically significant difference in the perceptual judgment of objective achievement between the UC and UF groups.
• Improvement in perception about the satisfaction with the process elements: this is the level of improvement in the perception of the participants, about the satisfaction with the process elements, after making use of the process. This variable represents the statistically significant difference in the perceptual judgment of satisfaction with the process elements between the UC and UF groups.
• Improvement in perception about the satisfaction with the activity outcome: this comprises the level of improvement in the perception of the participants, about the activity outcome, after the use of the process. This variable represents the statistically significant difference in the perceptual judgment of satisfaction with the outcome of the activity between the UC and UF groups.

To validate the hypothesis on “The THUNDERS process encourages the construction of shared understanding in a problem-solving activity,” variables 3, 4, 9, 10, 11, 12, 13, 14, and 15 were specifically considered.

To validate the hypothesis about “The THUNDERS process improves the construction of shared understanding in a problem-solving activity,” the variables 1, 2, 5, 6, 7, and 8 were specifically considered.

Design of Activities, Time, and Instruments

For the execution of the exploratory study, a set of activities and their respective approximate duration and necessary instruments were designed. For UC who used THUNDERS, a duration of 1 h 25 min was planned for the pre-process phase, 2 h 45 min for the process phase, and finally, 1 h for the post-process phase. Similarly, for UF who did not use THUNDERS, it was planned that it would take 6 h from the planning of the activity to the analysis of the results obtained. For both universities, an additional 20 min were planned so that at the end of the analysis of the results, both participants and teacher completed a survey designed to obtain the necessary information.

For UC the following support instruments were used: Collab [53], MEPAC [55]; presentation of the introduction to the experiment and conceptual elements; activity document and result templates; form for individual writing; form for writing questions about descriptions of other understandings; form for classifying the clarity and opinion about descriptions of understandings (taken from [13]); form for determining the shared knowledge and/or differences in knowledge with their groupmates (taken from [13]); form for writing a group description; and templates to be filled out to solve the problem.

For UF, support instruments used comprised: Collab [53]; PowerPoint presentation of the introduction to the exploratory study and conceptual elements; activity document and results templates; and templates to be filled out in to solve the problem.

Finally, for the participants from both universities, it was planned to use an individual survey format with questions that they were required to answer about their perception of the performance of their group when working collaboratively, about the achievement of objectives within their group, about satisfaction with the elements of the proposed process, and finally, about satisfaction with the results obtained from the activity carried out (taken from [10]).

4.4. Execution of the Exploratory Study

At UC, all the phases, activities, tasks, and steps specified in the THUNDERS process were applied, making use of the tools planned for their support. MEPAC was used to guide each of the phases, and Collab for the formation of groups in the process and the different forms designed to execute the collaborative activity on previously designed process lines, and the necessary forms to collect the information that allowed the analysis of the data provided. UF meanwhile carried out the same designed collaborative activity. They also formed their groups using Collab. The teacher designed the necessary elements for the activity without a specific guide, and simply between the groups, they came up with a solution to that activity without following the proposed process. The time used to apply the THUNDERS process and complete the survey in UC was 4 h and 40 min and for UF it was 3 h and 30 min, from the design of the activity to the analysis of the results and the respective survey.

4.5. Results

From the execution of the exploratory study, results were generated that were statistically analyzed, bearing in mind that a control group (UF) was used that did not employ the THUNDERS process and another group (UC) that did. At first glance, the results appear evident, but it was necessary to ensure that the differences between these were statistically significant. For this, the Student’s t-distribution [56] was used to validate the hypotheses defined in this study. The Student’s t-test is used to determine whether there is a significant difference between means in normally distributed populations of one or two groups by hypothesis testing. A t-test can be used to determine whether a single group differs from a known value (a one-sample t-test), whether two groups differ from each other (independent samples t-test), or whether there is a significant difference in paired measures (a dependent or paired samples t-test) [57].

As previously mentioned, the t-test features three types of tests, which were considered for this study: (a) t-test for means of two paired samples, this means data that come from the same people, that is, the comparison of the experimental group BEFORE and AFTER a stimulus (in this case the stimulus is the application of THUNDERS). The next two types of t-tests use data that come from two different groups, where one received the stimulus, and the other did not (for this case, a control group (UF) that did not use the THUNDERS process and another group (UC) that did use it); (b) t-test for two samples with equal variances; and (c) t-test for two samples with unequal variances.

The calculations of the different Student’s t-tests applied in this study were carried out using the function offered by the Excel Office suite for calculating this test. The values employed to make this calculation are shown in

Table 4. Values used for t-tests.

	t-Tests Type (a)	Type (b) and (c) t-Tests
Reliability level	95%	95%
Significance level	5%	5%
Observations or cases	15	15 (UC), 15 (UF)
Critical value in	Two tailed	Two tailed
Degrees of freedom	14	28

.

Initially, for the type (b) and (c) t-tests it was necessary to determine if the variances of the values were equal or unequal; this is why it uses the Fisher test (the function provided by Excel Office suite was also used for the F-test to calculate this test) [58], where we defined two hypotheses.

• H₀ = the variances are equal.
• H_a = the variances are different.

Additionally, in the three test types, the acceptance or rejection of the null hypothesis were considered:

• If p-value or F-Value ≤ significance level, the null hypothesis is rejected.
• If p-value or F-Value > significance level, the null hypothesis is accepted.

Statistical Analysis

Once the groups were organized with their respective roles and responsibilities, the necessary information of the activity was shared, and then, each participant was given the specification of the activity to be carried out. Each was assigned a time to read and understand it.

Variable 1: Improvement in the group descriptions

Having completed this individual analysis, the UC groups were asked to write a description of what they individually understood about what they had to do in the activity and about the problem to be solved. After the interaction with the group, and resolving any doubts and generating the necessary debates, they were required to make a description in the group, with the participation of all, about what they understood, in such a way that all agreed with what was specified.

To analyze the shared understanding of the activity carried out, it was necessary to determine what the participants understood both individually and as a group. This is because one premise of shared understanding is that there must be an individual, complete, and correct understanding in order to later interact with the group, improve this individual understanding, and be able to have a common, equally complete, and correct understanding. For this, the statistical difference between the individual and group understanding of the UC groups was first analyzed. To do this, each understanding was scored by the activity coordinator (from 0 to 5, where 0 is the understanding furthest from what they should have done about the activity and 5 is the most correct understanding), obtaining the following values (

Table 5. Individual and group scores.

	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
Individual score	2	2.8	2	2	2.5	3	2.8	2.3	3	2.5	3	2	2.3	3	2.5	2.51
Group score	3.1	3.2	2	3	2.5	5	3.8	3	5	3.1	4	2.5	2.8	3.5	3	3.51

):

It is apparent that judging from the results, the group description generates better scores, according to the individual qualifications. To ensure that these differences are not only apparent but statistically significant, the type (a) t-test was applied.

To validate “Improvement in the group descriptions” variable, consideration was made of:

• Null hypothesis: There is no statistically significant difference in the mean of scores between the individual and group descriptions.
• Alternative hypothesis: There is a statistically significant difference in the mean of scores between the individual and group descriptions.

Obtaining the following results, t-value = −5.130, p = 0.00015, and critical value = 2.145.

These results can be used to determine that: t (15) = −5.130; p = 0.00015, p (0.00015) < significance level (0.05) meaning that the null hypothesis was rejected, leading to accepting the alternative.

Variable 2: Improvement in the UC and UF descriptions

The UF groups made only group descriptions of what they understood they should have done about the activity and the activity problem. These were also rated by the coordinator, obtaining the following values, shown in comparison to the UC group understanding scores (

Table 6. UC and UF group scores.

	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
UC Group score	3.1	3.2	2	3	2.5	5	3.8	3	5	3.1	4	2.5	2.8	3.5	3	3.3
UF Group score	3	2.8	3	2.4	2	1.6	2.5	2	3.4	2	3.5	2	1.5	3	2.6	2.49

).

To analyze the difference between the UF and UC groups, a statistical analysis was needed. This made it possible to determine whether or not the differences in the scores of the group description obtained were statistically significant—perhaps even clearly so—and thereby determine whether or not this might be due to the use of the process.

According to the data analyzed, corresponding to two different groups where one received the stimulus and the other did not, it was necessary to initially assess what type of t-test ought to be used. To do this, the F-test was applied, obtaining a value of 0.261, determining that the null hypothesis was accepted, where variances are equal. Accordingly, the type (b) t-test was applied to the values.

To validate “Improvement in the UC and UF descriptions” variable, these statements were considered:

• Null hypothesis: There is no statistically significant difference in the mean of scores for group descriptions between UC and UF participants.
• Alternative hypothesis: There is a statistically significant difference in the mean of scores for group descriptions between UC and UF participants.

The results were as follows, t-value = 2.98, p = 0.0059, and critical value = 2.048.

These results produced a p (0.0059) < significance level (0.05), causing the null hypothesis to be rejected and leading to the alternative being accepted.

Variable 3: Understanding other descriptions

For the UC groups, each participant had to share their individual description in front of the rest of their groupmates, while the others had to rank whether they understood what they heard from their peers (this ranking was from 1 to 5, where 1 was very unclear and 5 was very clear), obtaining the following values (

Table 7. Obtained values of classifying other descriptions.

Very Unclear	Unclear	Neutral	Clear	Very Clear
1.10%	1.70%	15.60%	35.50%	46.10%

):

To validate “The understanding other descriptions” variable, the following aspects were considered:

• Null hypothesis: The percentage of perception about the level of understanding that the participants regarding the descriptions of the other participants in the group is less than 60%.
• Alternative hypothesis: The percentage of perception about the level of understanding that participants have regarding the descriptions of the other participants in the group is greater than or equal to 60%.

From the obtained values, it can be stated (taking answers “Clear” and “Very Clear” as positive responses) that the percentage of perception about the level of understanding was 81.6%, thereby causing the null hypothesis to be rejected and accepting the alternative.

Variable 4: The opinion of other descriptions

Similarly, for the UC groups, each participant classified the description heard by each of their classmates, determining from 1 to 5 whether or not they agreed with it (1 completely disagree, 5 completely agree), obtaining the following values (

Table 8. Obtained values of classifying the opinion of other descriptions.

Completely Disagree	I Disagree	Neutral	I Agree	I Completely Agree
1.10%	1.70%	15.60%	35.50%	46.10%

):

To validate the “Opinion of other descriptions” variable, the following aspects were considered:

• Null hypothesis: The percentage of perception about the level of opinion that the participants have regarding the descriptions of the other participants in the group is less than 60%.
• Alternative hypothesis: The percentage of perception about the level of opinion that the participants have regarding the descriptions of other participants in the group is greater than or equal to 60%.

From the obtained values it can be stated (taking answers “I Agree” and “I Completely Agree” as positive responses) that the percentage of perception about the level of opinion was 73.9%, thus rejecting the null hypothesis and causing the alternative hypothesis to be accepted.

Variable 5, 6: Improvement in homogeneous understanding and Improvement in the discrepancy

The change in shared understanding was monitored and the effects of THUNDERS were evaluated. As such, in the UC groups, two self-assessment measures were performed to determine if, in the judgment of each participant, there was shared knowledge (Measure 1) and/or discrepancy in knowledge (Measure 2) with their groupmates. These measures were made by means of a questionnaire at two moments, at the beginning of the activity (pre) and the end of the activity (post) (questions taken from [13], with values from 1 to 5, where 1 is less understanding and less difference), obtaining the following values (

Table 9. Measures of shared knowledge and discrepancy in knowledge.

		Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
Measure 1	Pre	3	3	1	2.8	3	3.2	2	1.6	2.4	3	2.3	1	2	4	2.5	2.45
	Post	4	3.8	1	3.4	3.8	4	3.8	3.6	3.8	4	3	4	3.5	4.5	2.5	3.51
Measure 2	Pre	1	1	1	1.4	1	0.6	2.2	2.2	1.6	1	2	1.4	2.2	2	1	1.45
	Post	0.2	0.2	1	0.6	0.4	0.00	1.4	0.4	0.4	0.5	1	1.5	2	3	2	0.97

):

Although the groups started with different knowledge and different levels of perceived shared knowledge, all of them experienced a substantial improvement in those measures.

It appears that from the results, the scores improve when analyzing them before and after the use of the THUNDERS process, both for Measure 1 (referring to shared knowledge, where the values closest to 4 indicate that the group has a homogeneous understanding) and for Measure 2 (referring to discrepancy in knowledge, where the values closest to 0 indicate that the individual understandings do not differ from the group understandings). It was therefore sought to assure that these differences are not only apparent but statistically significant. For this reason, the type (a) t-test was applied.

To validate “Improvement in the homogeneous understanding” variable, the following aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the obtained results from the homogeneous understanding of the group before and after the use of the proposed process.
• Alternative hypothesis: There is a statistically significant difference in the mean of the obtained results from the homogeneous understanding of the group before and after the use of the proposed process.

To validate the “Improvement in the discrepancy” variable, these aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the obtained results from differences in individual knowledge versus group knowledge, before and after the use of the proposed process.
• Alternative hypothesis: There is a statistically significant difference in the mean of the obtained results from differences in individual knowledge versus group knowledge, before and after the use of the proposed process.

The following results were obtained (

Table 10. Obtained t-test values.

	t-Value	p	Critical Value
Measure 1	−5.233	0.00013	2.145
Measure 2	2.434	0.029	2.145

):

Table 10 shows how the measures for shared knowledge and differing or discrepant knowledge among the members of each group changed from pre-test to post-test. Shared knowledge increased significantly from a mean of 2.45 to 3.51; differences in knowledge decreased from 1.45 to 0.97. With these results, both for shared knowledge p (0.00013) < significance level (0.05), and for the discrepancy in knowledge p (0.029) < significance level (0.05), it can be determined that the null hypotheses were rejected, and the alternatives accepted.

Variables 7, 8: Improvement in homogeneous understanding in UC and UF and Improvement in the discrepancy in UC and UF

Meanwhile, for the UF groups, only the self-assessment measures of shared knowledge and discrepancy in knowledge were researched, at the end of the activity (post). Only this post-evaluation was carried out since in not applying the THUNDERS process the individual understandings were not analyzed. However, it was sought to analyze how shared understanding in UF behaved after the activity (post) without the use of THUNDERS, comparing with results obtained in the post with the UC groups, obtaining the following values (

Table 11. Measures of shared knowledge and the discrepancy in knowledge in UC vs UF (post).

		Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
Measure 1	UF	1.4	1.1	1.6	2	1.2	3	1.4	1.3	3.1	2	2.5	3.1	1.1	1.4	2	1.88
	UC	4	3.8	1	3.4	3.8	4	3.8	3.6	3.8	4	3	4	3.5	4.5	2.5	3.51
Measure 2	UF	3.2	2.4	2.6	3	3.5	2	1.5	1	4	3.2	3.2	2.6	2	1.5	2.6	2.55
	UC	0.2	0.2	1	0.6	0.4	0.00	1.4	0.4	0.4	0.5	1	1.5	2	3	2	0.97

):

Considering that the data corresponded to two different groups, it was necessary to initially analyze what type of t-test should be used. Due to that, the F-test was again applied, obtaining a value of 0.604 for shared knowledge and 0.933 for discrepancy in knowledge. This factor determines that the null hypotheses were accepted, where the variances are equal and due to this, the type (b) t-test was applied to the values.

To validate the “Improvement in homogeneous understanding in UC and UF” variable, the following aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the obtained results from the homogeneous understanding between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the obtained results from the homogeneous understanding between the UC and UF groups.

To validate the “Improvement in discrepancy in UC and UF” variable, these aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the obtained results from differences in the individual knowledge versus the group knowledge, between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the obtained results from differences in the individual knowledge versus the group knowledge, between the UC and UF groups.

The following results were obtained (

Table 12. Obtained t-test values.

	t-Value	p	Critical Value
Measure 1	5.687	0.000004	2.05
Measure 2	−5.134	0.000019	2.05

):

With the results, both for shared knowledge p (0.000004) < significance level (0.05), and for discrepancy in knowledge p (0.000019) < significance level (0.05), it can be determined that the null hypotheses were rejected leading to the alternatives being accepted.

Variables 9, 10, 11: Improvement in the construction activity, Improvement in the co-construction activity, and Improvement in the constructive conflict activity

At the end of the activity, the participants, both those who used THUNDERS (UC) and those who did not (UF), were asked to complete a survey in which the first group of questions was about the tasks of the stage of shared understanding (construction ©, co-construction (CoC), and constructive conflict (CC)). For this, a version of the nine items proposed by Van den Bossche et al. [10] was used. This questionnaire was principally used to examine how the participants of each group behaved collaboratively while executing the shared understanding process construction. The survey had a set of items measured on a 5-step Likert scale, with 4 = “Strongly Agree” being the highest value and 0 = “Strongly Disagree” being the lowest value, obtaining the following values (

Table 13. Tasks of the shared understanding stage for UC and UF.

		Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
UC	C	3.667	3.600	3.533	3.083	3.429	3.417	3.533	3.000	3.133	3.429	3	3.241	3.6	3.068	3.422	3.344
	CoC	3.625	3.300	3.200	2.625	3.200	3.375	3.300	3.500	3.100	3.451	3.187	2.873	3.625	3.128	3.765	3.283
	CC	3.375	3.300	3.250	2.688	3.150	3.250	3.150	3.083	3.200	3.789	4.231	4.651	3.875	3.654	3.812	3.497
UF	C	2.691	3.000	3.111	2.564	2.314	3.133	2.564	2.143	2.873	3	3.189	2.691	3.111	3.133	2.92	2.834
	CoC	3.000	2.875	2.625	2.431	2.122	2.861	2.901	3.113	2.730	1.876	2.711	2.542	3.165	2.412	2.713	2.672
	CC	2.750	3.111	2.675	2.654	2.125	2.871	2.615	2.243	3.871	2.761	2.213	2.761	1.871	2.716	1.761	2.667

):

Considering that the data corresponded to two different groups, it was necessary to initially analyze what type of t-test should be used. The F-test was applied in each task, obtaining from construction a value of 0.235, co-construction a value of 0.512, and constructive conflict a value of 0.826, which determined that the null hypotheses were accepted in which the variances are equal and due to this fact, the type (b) t-test was applied to the values.

To validate the “Improvement in construction activity” variable, the following aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the results obtained from the activities of construction between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from the activities of construction between the UC and UF groups.

To validate the “Improvement in co-construction activity” variable, these statements were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the results obtained from the activities of co-construction between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from the activities of co-construction between the UC and UF groups.

To validate the “Improvement in constructive conflict activity” variable, these outcomes were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the results obtained from the activities of constructive conflict between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from the activities of constructive conflict between the UC and UF groups.

The following results were thus obtained (

Table 14. Obtained t-test values.

	t-Value	p	Critical Value
Construction	4.932	0.000033	2.048
Co-construction	5.124	0.000020	2.048
Constructive conflict	4.379	0.00015	2.048

):

With the results, for construction p (0.000033) < significance level (0.05); co-construction p (0.0426) < significance level (0.05); and constructive conflict p (0.00015) < significance level (0.05), it was determined that the null hypotheses were rejected, concluding that the alternatives hypotheses be accepted.

Variable 12: Improvement in the quality of the results

At the end of the activity, in both universities, UC and UF, the groups generated results after solving the problem of defining the scope in a process line. With these solutions developed for the problem, the activity coordinator was asked to assign a score to the solution of each group (scores between 5 and 0, where 5 was for results that were closest to the optimal solution and 0 for those that were furthest away from this solution). This same scoring was assigned to both UC and UF groups, in order to compare the results of the activity, thus analyzing the quality of the results obtained. The following values emerged (

Table 15. UC and UF scores for the activity results.

	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
UC	3.3	4.1	3.3	4	3.5	4	4.6	4.3	4.5	4.5	3.2	4	3.2	4	3.9	3.89
UF	2.8	3.2	2.4	3.1	2.8	3	2.9	3	3	2.6	2.1	4	2.1	2.1	2	2.74

):

Again, bearing in mind that the data corresponded to two different groups, it was necessary to initially analyze what type of t-test should be used. The F-test was applied, obtaining a value of 0.702. This determined that the null hypothesis was accepted, in which the variances are equal. Accordingly, the type (b) t-test was applied to the values.

To validate the “Improvement in the quality of the results” variable, the following aspects were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the scores from the results after applying the guide between the UF and UC groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the scores from the results after applying the guide between the UF and UC groups.

Obtaining the following results: t-value = 6.150, p = 0.000001, and critical value = 2.048.

With these results it was determined that p (0.000001) < significance level (0.05), thus rejecting the null hypothesis and concluding that the alternative hypothesis was accepted.

Variables 13, 14, 15: Improvement in perception about achievement of objectives, Improvement in perception about satisfaction with the process elements, and Improvement in perception about satisfaction with the outcome of the activity

In the survey that each participant was required to complete at the end of the execution of the collaborative activity, for both the UC and UF groups, there was another set of questions that evaluated the perception of participants on the achievement of the objectives (AO) within their group, satisfaction with process items (SP), and satisfaction with the activity outcome (SO). These questions were taken from Briggs et al. [59] which sought to assess the perception of existing collaboration and the satisfaction with what has been delivered in the activity. The following values were obtained (

Table 16. UC and UF survey answers of perception of collaboration.

		Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8	Group 9	Group 10	Group 11	Group 12	Group 13	Group 14	Group 15	Average
UC	AO	3.612	4.123	2.831	3.334	3	4.123	4.821	3.415	4.621	3.314	4.126	2.356	3.572	3.455	3.123	3.52
	SP	3.221	3.723	3.240	3.270	3.332	4.381	3.452	3.255	4.169	4.154	3.761	3.797	3.253	4.135	4.681	3.72
	SO	3.542	4.264	2.836	2.642	3.741	2.135	2.145	3.732	4.439	3.761	4.187	2.865	3.654	4.421	3.754	3.54
UF	AO	2.312	3.437	2.541	2.214	2.761	2.234	3.542	3.122	2.763	2.543	2.871	2.123	3.201	2.679	2.134	2.69
	SP	3.242	3.761	3.123	2.876	4.213	3.987	3.767	3.212	3.761	4.123	3.762	3.871	4.321	3.767	3.876	3.71
	SO	2.854	3.212	3.156	2.541	3.866	4.211	2.768	4.341	3.976	4.765	4.321	3.861	2.967	3.921	4.213	3.66

):

Considering that the data corresponded to two different groups, it was once again necessary to initially analyze what type of t-test should be used. The F-test was applied, obtaining for AO a value of 0.407, for SP a value of 0.596, and for SO a value of 0.996, for which it was determined that the null hypotheses were accepted, where the variances are equal and, therefore, the type (b) t-test was applied to the values.

To validate the “Improvement in perception about the achievement of objectives” variable, these statements were considered:

• Null hypothesis: There is no statistically significant difference in the mean of the results obtained from perception of participants about the achievement of objectives between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from perception of participants about the achievement of objectives between the UC and UF groups.

To validate the “Improvement in perception about the satisfaction with the process elements” variable, it was considered that:

• Null hypothesis: There is not any statistically significant difference in the mean of the results obtained from satisfaction perceived by the participants about process items between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from satisfaction perceived by the participants about process items between the UC and UF groups.

To validate the “Improvement in perception about the satisfaction with the activity outcome” variable, these aspects were considered:

• Null hypothesis: There is not any statistically significant difference in the mean of the results obtained from satisfaction perceived by the participants about activity outcomes between the UC and UF groups.
• Alternative hypothesis: There is a statistically significant difference in the mean of the results obtained from satisfaction perceived by the participants about activity outcomes between the UC and UF groups.

The following results were obtained (

Table 17. t-test obtained values.

	t-Value	p	Critical Value
Achievement of the objectives (AO)	4.276	0.0002	2.048
Satisfaction with process items (SP)	0.065	0.948	2.048
Satisfaction with activity outcome (SO)	−0.493	0.625	2.048

):

The results for achievement of objectives (AO) p (0.0002) < significance level (0.05) determined that the null hypothesis was rejected and concluded that the alternative was accepted. For satisfaction of participants with process items (SP) p (0.948) > significance level (0.05) and for satisfaction of participants with activity outcome (SO) p (0.625) > significance level (0.05), it could be determined that the null hypotheses were accepted for both cases, rejecting the alternatives.

4.6. Discussion of Results

Upon executing this exploratory study and applying the Student’s t-test to the resulting values generated from the different surveys and analysis of what was actioned, in variables 3, 4, 9, 9, 10, 11, 12, and 13, the acceptance of the alternative hypotheses was obtained, which refer to: The percentage of perception on the level of understanding and on the level of opinion that the participants have regarding the descriptions of the other participants of the group is greater than or equal to 60%, obtaining 81.6% clear understanding and 73.9% favorable opinion. There is a statistically significant difference in the mean of the scores obtained in the construction, co-construction, and constructive conflict activities between the UC and UF groups. There is a statistically significant difference in the mean of the scores of the results (final deliverables of the collaborative activity) between the UF and UC groups. Finally, there is a statistically significant difference in the mean of the scores obtained from the participants’ perception of the achievement of the objectives between the UC and UF groups. With this, we can establish that the use of THUNDERS generates greater understanding and a favorable opinion about the understandings of the other partners in the collaborative activity, greater individual construction of understanding, debates, and discussions of ideas, which after the conflict generates a group understanding (categories that are defined in [13]), better results, and a perception in the participants of satisfaction for the achievement of the objectives of the collaborative activity. With these hypotheses accepted we can conclude that THUNDERS in this validation context encourages the construction of shared understanding in a problem-solving activity in an acceptable way.

However, in the measurement of variables 14 and 15 the null hypotheses were accepted, which refer to: there is no statistically significant difference in the mean of the results obtained from the satisfaction perceived by the participants on the process elements and on the outcomes of the activity between the UC and UF groups. With this, we can establish that THUNDERS does not generate satisfaction in the participants about the elements (phases, activities, tasks, steps, workflows, work products, and roles) that make up the process and about the results that the participants obtain at the end of the collaborative activity.

However, according to the results obtained in the measurement of variables 1, 2, 5, 6, 7, and 8, their alternative hypotheses were accepted, which refer to: Specifically, in the UC groups (which were the groups that used THUNDERS), there is a statistically significant difference in the mean of the scores between individual and group descriptions. There is a statistically significant difference in the mean of the scores obtained from the homogeneous understanding of the group before and after the use of the proposed process. There is also a statistically significant difference in the mean of the scores obtained from the differences in individual knowledge versus group knowledge, before and after the use of the proposed process. Considering the UC and UF groups, there is a statistically significant difference in the mean of the scores of the group descriptions between the UC and UF participants, in the mean of the results obtained from the homogeneous comprehension, and those obtained from the differences in individual knowledge versus group knowledge between the UC and UF groups. With this, we can establish that the use of THUNDERS generates better individual and group understandings, which are reflected in the deliverables made, a uniform understanding among the members of the groups, and a better group knowledge. With these hypotheses accepted, we can conclude that THUNDERS improves shared understanding in groups that carry out a collaborative activity.

5. Threats

Construct validity: In order to reduce the subjectivity of the instruments used to collect the information necessary for the exploratory study, each of the forms and surveys was reviewed by experts in collaboration issues. To improve the validity of the construct, in this case of THUNDERS, the study was done with differential groups, where results were obtained from two different universities—one with the construct and the other without it—in addition to other tests where the results were analyzed before and after applying the process. In addition, the same collaborative activity was used in both universities. In order to avoid a higher complexity in the concepts of the collaborative activity, an initial contextualization was carried out on process lines, and, in addition, it was sought that the students already had a basic knowledge in this regard, as systems engineering students with knowledge of the topic. Another threat to the validity of the construct was the use of self-assessment surveys to measure shared understanding. To reduce the threat due to the use of a single means of measurement, there was a person in charge of carrying out constant observation of the groups to analyze their performance and the understanding that emerged in them.

Internal validity: To minimize the threat to internal validity, for both universities, each of the phases and the collaborative activity were executed in different sessions, in order to avoid fatigue and the participants losing interest in solving what was requested. To minimize the differences between the participants due to their heterogeneity of characteristics, a software tool was used to form heterogeneous groups that, through their learning style, could complement each other to obtain better collaborative results.

External validity: To minimize one of the threats to external validity, in which the participants tried to answer the questions to analyze the information according to their perception of social desirability or to help the experimenter, all participants were sensitized to the need to be very sincere in their answers and to respond according to what was experienced and observed during the study. In addition, to minimize the effect that the participants saw the process as something appropriate and useful in the context, as a function of the novel, not usual phenomenon, it was sought that the participants not only had knowledge of the activity topic but also that they knew the benefits of working collaboratively and its necessary elements.

6. Conclusions

This paper presents in detail how the statistical analysis was performed, using the Student’s t-test on the data obtained in the execution of an exploratory study to validate the THUNDERS process. The study consisted of the resolution of a problem of scope definition in software process lines by students from two universities, in which a set of variables was validated through their respective hypotheses. According to the measurement of these variables, it was possible to conclude that THUNDERS in this validation context encourages and improves the construction of shared understanding. This is due to the fact that the comparison of the results when applying the Student’s t-test in the groups that used THUNDERS with respect to those that did not, and according to the data taken before and after using the process, showed that there was a statistically significant difference that allowed accepting the hypotheses and guaranteeing said encouragement and improvement. However, it cannot be guaranteed that there is satisfaction on the part of the students with the elements of the process, and with the results obtained from the collaborative activity. This may be due to the fact that the THUNDERS process still needs improvement in the definition of the elements that compose it to make it easier to use, since its documentation is extensive and requires many elements to be used. Similarly, from the observation, it was determined that THUNDERS needs to incorporate new elements to guide and support each of its phases, to make it easier to execute, in addition to the need to include specific roles for monitoring the correct execution of each activity and task, and also to incorporate new mechanisms for measuring shared understanding. This exploratory study allowed us to generate more empirical evidence of the importance of building, maintaining, and assisting true collaboration in this type of activity to achieve better results, and one way to do this is by analyzing cognitive factors and, in this case, obtaining the benefits of shared understanding to generate greater collaboration.

Future work is expected to improve the definition of the different elements that make up THUNDERS, so that it can be subject to further validation in other contexts, not only in the academic environment, and to allow its further improvement; to provide computational support for its execution and monitoring, so that each of the steps provided can be executed with computer support and thus facilitate the observation of the information and the support of the participants; to generate versions of the process that are easier to use depending on the type of collaborative activities and the context, that is, a version for those activities that are simpler to execute and do not require all the rigor and documentation provided by the process, or more complex versions of the process for activities that require it; to incorporate other elements for measuring the shared understanding in such a way that with computational support it allows the execution of actions in time to improve it and correct it in the indicated time. It is also necessary to investigate in greater depth the level of heterogeneity that directly influences the construction of shared understanding, determining which are the best characteristics that a group should have for such understanding to be built in the right way.

Study Limitations

This study had some limitations that are described below. There was a very small number of the sample for the calculation of the Student’s t-test, with which it is not possible to ensure a representative distribution of the population; however, the validation presented here was also intended to identify improvements and corrections to the process, which was achieved with the selected sample. It is expected in later stages of the research to make new validations with larger samples to improve the results obtained. Another limitation is that some measurements were made through the self-evaluation of the participants, which can generate subjective responses; however, it was sought that there should also be an observer who could analyze the actions carried out and thus give appreciations to the study. Another limitation was that the study was intended to be carried out in a software development company to determine that THUNDERS can be used in different contexts; however, due to the amount of time needed to carry out the complete study and the rigorous documentation required by the process, it was not possible to apply it in a company, so the collaborative activity to be carried out was an activity that could take place in a real company.