Computer-Human Interaction and Collaboration: Challenges and Prospects

Round 1

Reviewer 1 Report

The presented article aims to review the progress of the past 20 years of research in Human-Computer Interaction, in the areas of collaborative systems and Ubiquitous Computing in e-Learning systems, conducted within the University of Castilla research group.

In my opinion, the review presented in the article is interesting and significant and shows many aspects of HCI and collaborative systems. However, it focuses mainly on the work of this research group and there is very little reference to other approaches in this research area. For this reason, it is difficult to assess the originality and novelty of the proposed solutions.

The article is significant and open several interesting further research topics in this area (i.e. use of eye tracking to automatic assessment of user engagement and mental workload).

The overall quality of the presentation is acceptable. However, some aspects of presented work are not fully explained which makes it difficult to understand their importance and innovation in terms of Ubiquitous Computing in e-Learning (i.e. line 90: “system uses Artificial Intelligence-based mechanisms to check the quality of the proposed result” – how this AI mechanism work?; 109: “Through these frameworks, precise metrics can measure the collaboration of students  to solve complex problems” – what are those metrics?). There are references to other articles, but in my opinion, a brief explanation of these aspects in this article would help to understand this topic.

Also, figures are too big and don’t fit well on the page, also some figures are not readable.

Overall scientific soundness is acceptable, but in my opinion there are some conclusions that are loosely tied with presented tools (i.e. line 262 “Synchronous collaborative environments will allow the problems to be solved by several students once the problems have been planned in asynchronous collaborative systems. Again, it is vital that these systems should be integrated into LMSs easily and efficiently.“ – there are no references to other articles evaluating LMS in the context of collaborative environments).

I think that article is interesting for the readership of the Journal.

The English language is appropriate and understandable.

Overall merit: The article should be expanded to include an overview of other current approaches of collaborative systems and a should include a comparison with proposed solutions. Also, the contribution of the paper should be clearly stated.

Author Response

1. However, it focuses mainly on the work of this research group and there is very little reference to other approaches in this research area. For this reason, it is difficult to assess the originality and novelty of the proposed solutions.

This is true. The author tries to fix this in point 7 explaining the basic ideas he would like to transmit throughout the proposal to improve MOOCs and LMSs with more advanced collaborative tools.

2. line 90: “system uses Artificial Intelligence-based mechanisms to check the quality of the proposed result” – how this AI mechanism work?

The paper in this revision has a description of the use of the fuzzy logic adding the next paragraph:

To obtain these conclusions, we use variables modeled with fuzzy sets. These sets are compound for attributes or linguistic labels which define the domain of each variable. We considered variables characterized by five specific attributes corresponding to the following labels: Very-Low, Low, Normal, High and Very-High. In DomoSim-TPC we incorporated the necessary functionality to draw conclusions about Initiative, Creativity, Elaboration, Conformity and Disconformity.

3. 109: “Through these frameworks, precise metrics can measure the collaboration of students  to solve complex problems” – what are those metrics?).

The paper in this revision has a description of different variables used in the measure of the collaboration.

This is the added paragraph:

Some of these metrics for the collaborative process are:

 

1. Number of accesses to the system that each student carried out to work in the activity.
2. Number and mean of contributions made.
3. Mean of contributions size.
4. Kind of contributions.
5. Number of replied and refined contributions.
6. Depth of the discussion for each task of the problem.
7. Number of accesses to the last news.
8. Number of sent messages (classified by kind: planning, coordination or system).
9. Number of instantaneous messages.
10. Figures are too big and don’t fit well on the page, also some figures are not readable.

I fixed the size of the figures to fit the page and I changed some of them (Fig. 3, 4 and 11) to be more readable.

5. Overall scientific soundness is acceptable, but in my opinion there are some conclusions that are loosely tied with presented tools (i.e. line 262 “Synchronous collaborative environments will allow the problems to be solved by several students once the problems have been planned in asynchronous collaborative systems. Again, it is vital that these systems should be integrated into LMSs easily and efficiently.“– there are no references to other articles evaluating LMS in the context of collaborative environments).

The author added the following paragraph to explain the necessity of collaboration tools in LMSs extracted from the literature:

This is one of the results of “The Next Generation Digital Learning Environment. A report on Research” [30]. According to Brown et al. “What is clear is that the LMS has been highly successful in enabling the administration of learning but less so in enabling learning itself”. This report argues that the Next Generation Digital Learning Environment (NGDLE) must address five domains of core functionality:

• Interoperability and Integration
• Personalization
• Analytics, Advice, and Learning Assessment
• Collaboration
• Accessibility and Universal Design

In this report, the authors explain that “The support for collaboration must be a lead design goal, not an afterthought. The current LMS is often designed on the transmission model of education—a mechanism to transmit syllabi, content, and assessments.”

6. Overall merit: The article should be expanded to include an overview of other current approaches of collaborative systems and a should include a comparison with proposed solutions.

The author added a new paragraph which presents a survey containing a deep study of these topics that he considers appropriate to answer the proposed question with tables which summarize these systems :

 

A survey on educational process mining considering MOOCs, LMSs and their relationships with Collaborative Systems, as well as a summary and tables relating these systems can be consulted in [30].

 

7. The contribution of the paper should be clearly stated.

This new paragraph reflects the contribution of the paper:

In conclusion, this article presents a series of guidelines for the use of e-Learning systems that can help to design future systems based on the experience of a research group in HCI and E-Learning of more than 20 years.

It is found that although the first intelligent tutor systems and collaborative systems are mature technologies, they have not been introduced conveniently in the current systems. However, their efficient use is proposed by different authors.

Reviewer 2 Report

-please add photo of measurement, if any?;;;
-please add block diagram of the proposed research step by step ;;; what is the result of paper?;;;
-please add block diagram of the proposed method;;;
-please add photo/photos of application of the proposed research ;;;; 
-please add sentences about future analysis;;;
-Please compare with other methods, justify. Advantages or Disadvantages;;;
-Conclusion: point out what are you done;;;;

The most important:
-there are no relation to Electronics I think: Applied Sciences, Symmetry.
-so please add relation to Electronics, for example robots etc.?

Author Response

1. -please add photo of measurement, if any?;;;

There are not measurements of the collaboration that could be shown in a photo. The measurements are calculated with AI methods based on fuzzy logic.

2. -please add block diagram of the proposed research step bystep ;;; what is the result of paper?;;;

In this revision, the author added some new paragraphs to explain the results of the paper.

3. -please add block diagram of the proposed method;;;

In this paper, the author presents guidelines to ensure the quality of collaborative tools but not a method for these developments.

4. -please add photo/photos of application of the proposedresearch ;;;;

There are some figures explaining the different applications that fulfill this requirement.

5. -please add sentences about future analysis;;;

The author added a new paragraph

6. -Please compare with other methods, justify. Advantages orDisadvantages;;;

 

The author added a new paragraph which presents a survey containing a deep study of these topics that he considers appropriate to answer the proposed question with tables which summarize these systems :

 

A survey on educational process mining considering MOOCs, LMSs and their relationships with Collaborative Systems, as well as a summary and tables relating these systems can be consulted in [30].

 

 

 

7. -Conclusion: point out what are you done;;;;

This new paragraph reflects the contribution of the paper:

In conclusion, this article presents a series of guidelines for the use of e-Learning systems that can help to design future systems based on the experience of a research group in HCI and E-Learning of more than 20 years.

It is found that although the first intelligent tutor systems and collaborative systems are mature technologies, they have not been introduced conveniently in the current systems. However, their efficient use is proposed by different authors.

 

 

8. The most important: -there are no relation to Electronics I think: Applied Sciences,Symmetry. -so please add relation to Electronics, for example robots etc.?

 

The paper belongs to a Special Issue in “Computer - Human Interaction and Collaboration: Challenges and Prospects”, and according to this it fits to this topic.

Reviewer 3 Report

The author pointed the challenges and prospects of using e-Learning systems to improve learning effectiveness. It is interesting to see that e-Learning system using synchronous, asynchronous collaborative function, ubiquitous computing and augmented reality. Could the author elaborate more on the e-Learning systems in the paper with popular collaborative learning platform such as Google Collab, Moodle and also online course learning system such as Coursera and Udemy. It is good if the similarity and differences of these systems with the systems explained in the paper can be summarized in a table. I find some figures such as Fig. 3, 4 and 11 are blurry and hard to read due to small fonts. The author should include a higher quality figure.

Author Response

1. Could the author elaborate more on the e-Learning systems in the paper with popular collaborative learning platform such as Google Collab, Moodle and also online course learning system such as Coursera and Udemy. It is good if the similarity and differences of these systems with the systems explained in the paper can be summarized in a table.

 

The author added a new paragraph which presents a survey containing a deep study of these topics that he considers appropriate to answer the proposed question with tables which summarize these systems :

 

A survey on educational process mining considering MOOCs, LMSs and their relationships with Collaborative Systems, as well as a summary and tables relating these systems can be consulted in [30].

 

 

 

2. I find some figures such as Fig. 3, 4 and 11 are blurry and hard to read due to small fonts. The author should include a higher quality figure.

 

The author includes Higher quality figures not only for Fig. 3, 4 and 11 but also for some other figures.

Round 2

Reviewer 1 Report

The article has been corrected sufficiently for publication in the journal.

However, there are some minor issues with the formatting of the text – i.e. line break (line 21) look like new paragraph.  

Author Response

1. However, there are some minor issues with the formatting of the text – i.e. line break (line 21) look like new paragraph.  

The author corrected minor issues with the formatting of the text, in particular the one at line 21.

Reviewer 2 Report

-New references 2018-2021 should be added

-please add something about robots etc., photos of robots, hand prosthesis, leg prosthesis etc.

Author Response

1.) New references 2018-2021 should be added

The author added the next paragraph with a new reference in 2020:

According to Lin et al. [32] a combination of a web-based collaborative problem-solving system and teacher guidance can be implemented to develop students’ collaborative problem-solving skills.

32. Lin, K.-Y., et al. "Effects of web-based versus classroom-based STEM learning environments on the development of collaborative problem-solving skills in junior high school students." International Journal of Technology and Design Education 2020, 30(1): 21-34.

2.) please add something about robots etc., photos of robots, hand prosthesis, leg prosthesis etc.

The author can not add something about robots, Photos of robots, hand prosthesis or leg prosthesis because this paper is about "Computer - Human Interaction and Collaboration: Challenges and Prospects" not about robots.

 

Round 3

Reviewer 2 Report

-

Author Response

There are no suggestions or comments in this case