Harnessing Entropy via Predictive Analytics to Optimize Outcomes in the Pedagogical System: An Artificial Intelligence-Based Bayesian Networks Approach
Round 1
Reviewer 1 Report
The authors describe how to model a data set of students (comprising three assessments and 28 different attributes) using Bayesian Networks and entropy to analyse and predict the impact of the different attributes on assessment performance.
The modelling and analysis was performed using a commercial tool named bayesialab.
The article describes the experiments as well as the results regarding the real and hypothetical scenarios. The results raise several questions, which the authors prefer to leave for other researchers to address.
Wouldn't if be possible to achieve the same results with open-source platforms?
Improvement suggestions:
All variables sould be in italic including H, E, p((H), p(E), p(E|H), the k in k-fold, etc.
According to the recommendations of the International system of Units, in percentages there should be a space between the value and the % symbol (check at SI Unit rules and style conventions).
Please refrase line 351 (page 13).
Author Response
Reviewer’s comments | Response by author |
The authors describe how to model a data set of students (comprising three assessments and 28 different attributes) using Bayesian Networks and entropy to analyse and predict the impact of the different attributes on assessment performance. The modelling and analysis was performed using a commercial tool named bayesialab. The article describes the experiments as well as the results regarding the real and hypothetical scenarios. The results raise several questions, which the authors prefer to leave for other researchers to address. Wouldn't if be possible to achieve the same results with open-source platforms? | Thank you for your valuable comment. Yes, alternative Bayesian network software, some of which are free or opensource, have been added on page 30 in lines 972-975 of the revised manuscript.
|
Improvement suggestions: All variables should be in italic including H, E, p((H), p(E), p(E|H), the k in k-fold, etc. According to the recommendations of the International system of Units, in percentages there should be a space between the value and the % symbol (check at SI Unit rules and style conventions). | Thank you for your valuable comment. Yes, the variables have been amended in italics on page 3, lines 128-138 of the revised manuscript. |
Please rephrase line 351 (page 13). | Thank you for your valuable comment. Line 377 on page 13 has been amended in the revised manuscript. |
Reviewer 2 Report
Thank you for the opportunity to review “Harnessing Entropy via Predictive Analytics to Optimize Outcomes in the Pedagogical System: An Artificial Intelligence-based Bayesian Networks Approach”. The manuscript makes a timely contribution to a significant topic, and identifies an original and highly relevant research approach.
The authors have presented a Bayesian Network case study to assist educational researchers and stakeholders in conducting machine learning with their own schools’ data, the intended aim being to inform and advance their own educational practice. A novel approach based upon the principle of thermodynamics is targeted to explore entropy, more specifically the spread of energy, within a pedagogical system. The authors position of broadening the reach of machine learning tools such as Bayesialab beyond computer scientists, to interested laypeople in the field, is arguably of great importance.
The discussion and conclusion section presents a summary of the purpose and structure of the current research study. However little critical synthesis is provided on the results of the various “what has happened” and “what-if” scenarios. While a significant unexpected finding, that further paid tuition outside the classroom did not contribute to improved grades, was reported in the findings, the theme has been only superficially reported. Additional critical discussion, supported by research in the field, may strengthen the understanding of the findings and the overall cohesiveness of this paper. Further, this may be linked back to the authors original claim to explore entropy within the pedagogical system. In this instance, a detailed future research agenda to guide other practitioners in this field may be an appropriate addition to add to the completeness of the discussion and conclusion section.
Author Response
Reviewer’s comments | Response by author |
The discussion and conclusion section presents a summary of the purpose and structure of the current research study. However little critical synthesis is provided on the results of the various “what has happened” and “what-if” scenarios. While a significant unexpected finding, that further paid tuition outside the classroom did not contribute to improved grades, was reported in the findings, the theme has been only superficially reported. | Thank you for your valuable comment. Additional information from the literature about extra paid tuition outside the classroom has been added on page 18 lines 539-542 of the revised manuscript.
|
Additional critical discussion, supported by research in the field, may strengthen the understanding of the findings and the overall cohesiveness of this paper. Further, this may be linked back to the authors original claim to explore entropy within the pedagogical system. In this instance, a detailed future research agenda to guide other practitioners in this field may be an appropriate addition to add to the completeness of the discussion and conclusion section. | Thank you for your valuable comment. Additional information has been added on page 29 in lines 943-975 of the revised manuscript. |
Reviewer 3 Report
Dear Author(s),
Some comments below:
On page 6, beginning of the paragraph, I think you used interpolation, which can also lead to undesired effects...
End of page 6 and beginning of page 7, you mention the “R2-GenOpt*” algorithms; since this what you used in your study, can you provide more details on it? It would help the reader a lot.
On page 8, for figure 4, a clearer description would be necessary: without explanations, it is difficult to follow the text.
On Page 10, the numbers of the equations should be revised, because equation (1) was indicated on page 3.
In addition, these equations seem to be not clearly written (the text is confused) and thus difficult to understand.
Overall, it would be necessary to explain why and how you selected this approach and method.
On page 11, line 316, when you talk about Kullback Leibler divergence algorithms, can you add more details and a reference?
On page 12, from line 335 to 343, can you add more details? As it is now, it is difficult to understand.
On page 13, figure 8 and following, without a more precise interpretation and explanation, I think it is quite difficult to understand and follow…
On page 14, section 5.10 (line 373), maybe this paragraph would be more useful at the beginning of the paper, where a description of its structure is missing.
On page 15, from line 394, which are the criteria to select the scenarios you proposed?
On page 16, what “extra paid classes” consist? Additional curses for support?
On page 17, line 462, are you considering the differences between scenarios 2 and scenario 1?
Scenarios 3 and 4 seem to me more similar to Research Questions then Scenarios…
How the simulations with BayesianLab tool have been carried out for scenarios 3 and 4? Not enough explanations and details on this (crucial) point.
On page 19, line 532 and on, it seems to be in contradiction with the results from scenario 2; can you comment on it?
On page 21, line 603, substitute the colon with the full-stop.
About the conclusions at the beginning of page 23, are you really sure? They seem to be in contradiction with the ones of scenario 3 8anyway, the differences are not totally clear).
Following page 24, can you indicate some references for lift curve?
Page 26, from line 767 to 776, if you report those parameters and metrics, it is fine, but you need to explain why and write down more details.
In the conclusion Section (page 28), which are the next steps? Do you intend to provide an APP to possible stakeholders for this method?
I hope these comments can be useful to improve the paper.
Author Response
Reviewer 3’s comments | Response by author |
On page 6, beginning of the paragraph, I think you used interpolation, which can also lead to undesired effects...
| Thank you for your valuable comment. Interpolation was not used. According to the Bayesialab webpage http://www.bayesia.com/bayesialab-missing-values-processing, the Structural EM algorithm or the Dynamic Imputation algorithms were used to calculate for any missing values. A reference [31] was also added to the paragraph on page 6 of the revised manuscript. Thank you very much. |
End of page 6 and beginning of page 7, you mention the “R2-GenOpt*” algorithms; since this what you used in your study, can you provide more details on it? It would help the reader a lot.
| Thank you for your valuable comment. A reference to the webpage describing more about R2-GenOpt algorithm has been added in Reference [33]. |
On page 8, for figure 4, a clearer description would be necessary: without explanations, it is difficult to follow the text.
| Thank you very much for your valuable comment. The description is presented in Section 5.6 Descriptive analytics: overview of the Bayesian Network model, on page 7 of the revised manuscript. |
On Page 10, the numbers of the equations should be revised, because equation (1) was indicated on page 3. In addition, these equations seem to be not clearly written (the text is confused) and thus difficult to understand.
| Thank you very much for your valuable comment. The amendments to the equation numbers (2), (3), and (4) have been made on page 10 of the revised manuscript. Again, thank you very much. |
Overall, it would be necessary to explain why and how you selected this approach and method.
| Thank you very much for your valuable comment. The rationale for using the approach is presented in Section 4 of page 3 of the revised manuscript. |
On page 11, line 316, when you talk about Kullback Leibler divergence algorithms, can you add more details and a reference?
| Thank you very much for your valuable comment. Reference [35] has been added. A little more information about Kullback-Leibler divergence has been added on page 11 lines 331-332 of the revised manuscript. |
On page 12, from line 335 to 343, can you add more details? As it is now, it is difficult to understand.
| Thank you very much for your valuable comment. More information about Mutual information has been added in Section 5.8 on page 12 in lines 351-354 of the revised manuscript. Reference [36] has been added. |
On page 13, figure 8 and following, without a more precise interpretation and explanation, I think it is quite difficult to understand and follow…
| Thank you very much for your valuable comment. Additional information has been added in Section 5.9 on page 13 in lines 381-387 of the revised manuscript. Thank you. |
On page 14, section 5.10 (line 373), maybe this paragraph would be more useful at the beginning of the paper, where a description of its structure is missing.
| Thank you very much for your valuable comment. A description of the structure is presented on page 4 lines 153-162 of the revised manuscript. |
On page 15, from line 394, which are the criteria to select the scenarios you proposed?
| Thank you very much for your valuable comment. The criteria are purely exploratory. Information has been added to page 15 in lines 437-438 of the revised manuscript. Again, thank you very much. |
On page 16, what “extra paid classes” consist? Additional classes for support?
| Thank you very much for your query. Extra paid classes refer to “cram schools” or other commercially operated for-profit private companies that offer tutoring outside of schools which the students attend. |
On page 17, line 462, are you considering the differences between scenarios 2 and scenario 1?
| Thank you very much for your query. Yes, there was indeed an initial intention in considering the differences between scenario 2 and scenario 1. however, it is purely exploratory and inconclusive, so it would be contrived for the author to calculate the gains by directly subtracting the counterfactual results between scenario 2 and scenario 1. Information has been added on page 17 in lines 467-474 to reflect this. |
How the simulations with BayesianLab tool have been carried out for scenarios 3 and 4? Not enough explanations and details on this (crucial) point.
| Thank you very much for your valuable comment. Information has been added on page 19 in lines 579-581 and on page 22 in lines 706-709 to address this issue. |
On page 19, line 532 and on, it seems to be in contradiction with the results from scenario 2; can you comment on it?
| Thank you very much for your valuable comment. Indeed, there could be other hidden confounding factors that are not considered yet in this approach for preliminary explorations. For example, non-cognitive factors, such as, for example, psychological well-being, or emotional intelligence to manage stress) might be potential factors that could be included in future studies. More information has been added on page 29 and 30 about this. Again, thank you. |
On page 21, line 603, substitute the colon with the full-stop.
| Amended on page 21 line 657 of revised manuscript. |
About the conclusions at the beginning of page 23, are you really sure? They seem to be in contradiction with the ones of scenario 3 anyway, the differences are not totally clear).
| Thank you very much for your valuable comment. Indeed, there could be other hidden confounding factors that are not considered yet in this approach for preliminary explorations. For example, non-cognitive factors, such as, for example, psychological well-being, or emotional intelligence to manage stress) might be potential factors that could be included in future studies. More information has been added on page 29 and 30 about this. Again, thank you. |
Following page 24, can you indicate some references for lift curve?
| Thank you very much for your valuable comment. Reference [43] has been added for the lift curve, and also for gains curve (reference [42]) and ROC curve (reference [44]). |
Page 26, from line 767 to 776, if you report those parameters and metrics, it is fine, but you need to explain why and write down more details.
| Thank you very much for your valuable comment. |
In the conclusion Section (page 28), which are the next steps? Do you intend to provide an APP to possible stakeholders for this method? | The next step could be the inclusion of non-cognitive factors in the study. Information has been added on page 29 in lines 903-908. |
