Improving Cybersafety Maturity of South African Schools

Round 1

Reviewer 1 Report

This is a paper that addresses the important topic of cyber safety in schools. The paper contains an empirical investigation that stokes some interest, but it also suffers from a number of shortcomings that hinder its publication in the present form.

The major shortcoming is a lack of alignment between research questions, methods, and contributions. While the research questions are never explicitly enumerated as such (and it would be good to do so!), the reader broadly gets the impression that they are in line with the list of questions listed on p.2, on school preparations and equipment, promotion of cyber-safety culture and social protective measures. Indeed, the methodology and data gathering also point in this direction, as does the abstract and conclusions. Taken together, these parts of the paper would constitute an interesting, if somewhat limited, empirical contribution, possibly suitable for a conference publication.

However, the paper also contains parts, most importantly Section 5 but also to some extent Section 4.5, which are seemingly an important part of the contribution, but which do not fit into the methods used and the data collected. The foremost example from Section 4.5 is the claim on p.9 that “The analysis indicates that possible factors influencing the current maturity levels within South African schools are the lack of” a number of bullet points. These possible factors are very different from each other. Some follow by definition of the maturity indices used (e.g. the existence or non-existence of certain policy documents would have a direct influence on Element A, Strand 2.). Thus, it is indisputably true that these factors influence the current maturity levels, but this is an uninteresting truth. Other factors rest on empirical claims (e.g. about the effects of funding, commitment by parents, or national guidance and assistance). These claims are more interesting, and may well be true, but the data and the method do not support them – at least not to the extent needed for publication. In fairness it should be emphasized that the factors identified are only “possible”. This is a good sign of humility. Still, the paper then goes on to single out one aspect (“assistance or guidance from the South African government”) as critical and claiming that a top-down approach is ideal. The point is not that these claims are false – they may well be true. But the investigation described in Section 3 seems to allow at most an empirical snapshot of the state of the art, not inferences about how that state of the art may be changed. For the bullet list on p.10 to make sense, all the empirical claims (but not the definitional ones) should be backed up with references to empirical literature, making the plausibility of the claims explicit. Thus, one would like to see references to studies showing that, in reasonably similar contexts, e.g., commitment by parents, or national guidance and assistance have been shown to have positive effects. Most importantly, one would like to see references to studies showing that a top-down approach is ideal. Again, these claims may be true, but they do not follow from the 360safe measuring tool applied to 24 schools.

As for Section 5, the proposed cyber safety maturity guidelines may well be reasonable and useful. But this is not tested in any meaningful way with the method and data presented in the article. Instead, it is just posited. For it to be published as part of a scientific article, it must undergo some sort of evaluation. Such evaluation could take many forms and be performed in a positivist spirit or a design science spirit, but in the absence of any evaluation, it cannot be published.

In a nutshell: The article contains some interesting empirics, but it also contains several claims (4.5) and artifacts (5) that do not follow from the method used (as described) or from the data collected. These parts must either be removed, or preferably, better substantiated or validated.

Minor issues:

It might be a good idea to discuss the term “cyber safety” and position it with respect to e.g. the cyber security which is part of it (e.g. B.2.2).

For clarity, research questions should be explicitly phrased, either at the end of Section 1, or possibly at the end of Section 2, so that the reader gets the clearest possible picture of the intended contribution of the paper.

On p.1 the cyberspace definition should either be referenced, or it should be noted that it is an original definition for this research (as in the cyber-safety maturity definition on p.2).

It might be interesting to compare schools with higher education, which has been studied before: https://0-doi-org.brum.beds.ac.uk/10.1016/j.cose.2019.07.003

The heading of Section 2, “Cyberspace” could well be expanded into something more descriptive, such as “Cyber safety in schools”.

In B2.3, it should be mentioned that the Protection of Personal Information Act (No. 4 of 2013) is a South African law, and that the tool used in this respect differs from the UK original.

The effect of the small sample size should be further discussed, along with other limitations to the empirical research.

On p.6 it is claimed that “It is evident that the private schools scored slightly higher for maturity than the public schools due to extra funding.” It may well be true that it is extra funding that makes the difference, but it is certainly not evident from any data presented.

It would make the manuscript easier to understand if it were immediately clarified on p.10 that the 10 phases are divided into three steps. Fig.6 could be moved earlier to ease understanding.

On p.14, Fig.7 is said to present the “step-by-step process that occurs in series as each phase builds upon the previous phase.” This seems like a somewhat redundant wording: Isn’t a step-by-step process precisely one that occurs in series and where each phase or step builds upon the previous one?

The syntax of Fig.7 needs to be explained and probably improved upon. While Roles, Artifacts, and Actions seem to be distinguished by boxes with two rounded corners, boxes with four rounded corners, and diamonds respectively, the arrows connecting them all look the same even though they reasonably must depict quite different kinds of relations. Furthermore, there is no explanation for the meaning of the dashed double-arrow line, nor of the distinction between diamonds with both in- and outbound arrows (e.g. “establish”) and diamonds with outbound arrows only (e.g. “develop”). It is not meaningful to draw a picture that gives the impression of formal syntax if no such syntax is adhered to. Unless Fig.7 is formalized so that it adds precise meaning over and above that which can be inferred from Fig.6 and the text, it can simply be removed.

Typos:

p.2 skilto -> skill to

p.2 maturity of schools in respect of cyber-safety involvement -> maturity of schools with respect to cyber-safety involvement

p.11 Each step consists of a number phases -> Each step consists of a number of phases

Author Response

Review 1:

Comment: The major shortcoming is a lack of alignment between research questions, methods, and contributions. While the research questions are never explicitly enumerated as such (and it would be good to do so!), the reader broadly gets the impression that they are in line with the list of questions listed on p.2, on school preparations and equipment, promotion of cyber-safety culture and social protective measures. Indeed, the methodology and data gathering also point in this direction, as does the abstract and conclusions. Taken together, these parts of the paper would constitute an interesting, if somewhat limited, empirical contribution, possibly suitable for a conference publication.

Response: Research questions, outcomes and contribution of the paper were added to the Introduction section to focus the article more. The research focus was included and two focus points were included.

Comment: However, the paper also contains parts, most importantly Section 5 but also to some extent Section 4.5, which are seemingly an important part of the contribution, but which do not fit into the methods used and the data collected. The foremost example from Section 4.5 is the claim on p.9 that “The analysis indicates that possible factors influencing the current maturity levels within South African schools are the lack of” a number of bullet points. These possible factors are very different from each other. Some follow by definition of the maturity indices used (e.g. the existence or non-existence of certain policy documents would have a direct influence on Element A, Strand 2.). Thus, it is indisputably true that these factors influence the current maturity levels, but this is an uninteresting truth. Other factors rest on empirical claims (e.g. about the effects of funding, commitment by parents, or national guidance and assistance). These claims are more interesting, and may well be true, but the data and the method do not support them – at least not to the extent needed for publication. In fairness it should be emphasized that the factors identified are only “possible”.

Response: Section 5 and 4.5 were aligned a bit more in the methodology section to indicate that these sections are part of the proposed guidelines the includes possible cyber safety countermeasures that is based upon the findings of the data analysis and the literature review. Additional references were added to support the bullet points indicating the possible short comings. The author mentioned in the conclusion this could be future research.

Line 296 – 310 (#17), Line 425 (#24)

Comment: This is a good sign of humility. Still, the paper then goes on to single out one aspect (“assistance or guidance from the South African government”) as critical and claiming that a top-down approach is ideal. The point is not that these claims are false – they may well be true. But the investigation described in Section 3 seems to allow at most an empirical snapshot of the state of the art, not inferences about how that state of the art may be changed.

Response: The comment was address by adding additional references to indicate that in an ultimate situation government should provide the guidelines, process and supporting material to improve cyber safety within school. Until this is done, school should create and grow the maturity within the school (depending on resources) to the benefit of the learners. Line 317 (#14)

Comment: For the bullet list on p.10 to make sense, all the empirical claims (but not the definitional ones) should be backed up with references to empirical literature, making the plausibility of the claims explicit. Thus, one would like to see references to studies showing that, in reasonably similar contexts, e.g., commitment by parents, or national guidance and assistance have been shown to have positive effects.

Response: Additional references were included to indicate the issues raised within South African published research.  Line 142-144 (#12)

Comment: As for Section 5, the proposed cyber safety maturity guidelines may well be reasonable and useful. But this is not tested in any meaningful way with the method and data presented in the article. Instead, it is just posited. For it to be published as part of a scientific article, it must undergo some sort of evaluation. Such evaluation could take many forms and be performed in a positivist spirit or a design science spirit, but in the absence of any evaluation, it cannot be published.

Response: The main aim of the article was the provide data on the maturity of schools and to identify aspects that can hinders the growth of  cyber safety (maturity) within schools. The guidelines are just a representation of the maturity related aspects that was obtained by the data collection and literature review. It was also mentioned in the conclusion this is a limitation and can be included in future research.  Line 425 (#24)

Comment: In a nutshell: The article contains some interesting empirics, but it also contains several claims (4.5) and artifacts (5) that do not follow from the method used (as described) or from the data collected. These parts must either be removed, or preferably, better substantiated or validated.

Response: The comments were addressed to add a bit more in the methodology section that contains three parts: 1) Literature review 2) data gathering of the maturity of schools and 3) proposing guidelines that was based upon the findings of the literature review and the data analysis. The research identified all three approaches within the methodology section.

Minor issues:

Comment: It might be a good idea to discuss the term “cyber safety” and position it with respect to e.g. the cyber security which is part of it (e.g. B.2.2). #6,7

Response: The term (definition) of cyber safety was included as well as cyber security. The author included that the article is more focused on cyber safety and not cyber security. Line 55-63

Comment: For clarity, research questions should be explicitly phrased, either at the end of Section 1, or possibly at the end of Section 2, so that the reader gets the clearest possible picture of the intended contribution of the paper.

Response: The author added research questions, objectives and contribution of the research to provide the reader with a wider understanding of the research and the focus of the research. Line 69-74 (#1)

Comment: On p.1 the cyberspace definition should either be referenced, or it should be noted that it is an original definition for this research (as in the cyber-safety maturity definition on p.2).

Response:  Reference added to the requested definition. Line 69- 74 (#1)

Comment: It might be interesting to compare schools with higher education, which has been studied before: https://0-doi-org.brum.beds.ac.uk/10.1016/j.cose.2019.07.003 

Response: This is a very good idea. The author added this idea to future research as a possible investigation into the higher education and the link to this research.  Line 425 (#24)

Comment: Comment:  The heading of Section 2, “Cyberspace” could well be expanded into something more descriptive, such as “Cyber safety in schools”.

Response: The heading was changed for clarity.  Line 86 (#9)

Comment: In B2.3, it should be mentioned that the Protection of Personal Information Act (No. 4 of 2013) is a South African law, and that the tool used in this respect differs from the UK original.

Response: The Act number of the POPI act in South Africa were included. It was noted in the text that as this is a South African study and the South African legislation is in effect.  Line 156 (#15)

Comment: The effect of the small sample size should be further discussed, along with other limitations to the empirical research. #10

Response:  The comment was addresses by including that the research acknowledges the small data set. Added is that the saturation point was used. The author noted that this is possible limitation of the study. In the conclusion it was added that the study can be re-done on a majority schools in South Africa.  Line 171-176 (#10)

Comment: On p.6 it is claimed that “It is evident that the private schools scored slightly higher for maturity than the public schools due to extra funding.” It may well be true that it is extra funding that makes the difference, but it is certainly not evident from any data presented.

Response: It was included in the conclusion that in future research this can be investigated to find out why private schools scored better than government schools. Line 425 (#24)

Comment: It would make the manuscript easier to understand if it were immediately clarified on p.10 that the 10 phases are divided into three steps. Fig.6 could be moved earlier to ease understanding. #18

Response: Fig 6 moved earlier in the text to make the understanding and flow a bit better.  Line 399 (#18)

Comment: On p.14, Fig.7 is said to present the “step-by-step process that occurs in series as each phase builds upon the previous phase.” This seems like a somewhat redundant wording: Isn’t a step-by-step process precisely one that occurs in series and where each phase or step builds upon the previous one?

Response: Wording was taken out to not duplicate meaning of words. Line 391 (#16)

Commments: The syntax of Fig.7 needs to be explained and probably improved upon. While Roles, Artifacts, and Actions seem to be distinguished by boxes with two rounded corners, boxes with four rounded corners, and diamonds respectively, the arrows connecting them all look the same even though they reasonably must depict quite different kinds of relations. Furthermore, there is no explanation for the meaning of the dashed double-arrow line, nor of the distinction between diamonds with both in- and outbound arrows (e.g. “establish”) and diamonds with outbound arrows only (e.g. “develop”). It is not meaningful to draw a picture that gives the impression of formal syntax if no such syntax is adhered to. Unless Fig.7 is formalized so that it adds precise meaning over and above that which can be inferred from Fig.6 and the text, it can simply be removed.

Response:  The syntax of Figure was changed to follow the approved syntax for a flow diagram. The dotted lines were removed. Line 390 (#20)

Comment - Typos:

Response: The document was sent to editing again

Author Response File: Author Response.docx

Reviewer 2 Report

The paper is very interesting and deals with issues important in contemporary world.

The text is well structured and methodologically correct. some minor improvements suggested include:

1. Definition of cyber security maturity and its level, the scale that was implemneted in the research (it is not directly explained, rough presentation in lines 120-130)
2. Some background on South African education system - what was the size of the sample (what percentage of schools was analyzed, is it enough to present general concusion?)
3. Improving conclusion section - the list of actions suggested is long, is it applicable fr all the schools analyzed? maybe some actions should be differentatied? who is responsible for taking them? who should support/finance the actions? when should they be taken? in what sequence? when they should be finished/are exected to be finished? (these are just exaples of issues the author could refer to)

Thank you for your interesting and valuable work

Author Response

Review 2:

Comment 2.1: Definition of cyber security maturity and its level, the scale that was implemneted in the research (it is not directly explained, rough presentation in lines 120-130)

Response 2.1: Definition of cyber safety maturity is included in the text. An additional explanation was added to provide a bit more information on the on the different levels used in the study. Line 151-154. #11

Comment 2.2: Some background on South African education system - what was the size of the sample (what percentage of schools was analyzed, is it enough to present general concusion?) #10

Response 2.2:  This was addressed by including that the data gathering used saturation point were the results were within the same result area. The author included this as a possible shortcoming and addressed the point in future research (conclusion) as well. #10,#24

Comment 2.3: Improving conclusion section - the list of actions suggested is long, is it applicable fr all the schools analyzed? maybe some actions should be differentatied? who is responsible for taking them? who should support/finance the actions? when should they be taken? in what sequence? when they should be finished/are exected to be finished? (these are just exaples of issues the author could refer to) #17 #23

Response 2.3: These questions are all valid and relevant to the study. The questions were addressed to mention that schools could have different approaches they can take, short term, long term and dependent on resources within the school. The growth for each school will be different.  The sequence issue is addressed by indicating that knowledge/skills level, workload and resources can have an impact.  #17 #23

Author Response File: Author Response.docx

Reviewer 3 Report

This paper presents 'Improving Cyber Safety Maturity of South African
Schools'. The paper is well written. I have some suggestions:

1. Please highlight major contribution in introduction. 
2. Please discuss about each section in introduction such as Section 2 presents bla bla.
3. Introduction is very short and brief, please improve it. 
4. Please provide some future directions in conclusion. 

Author Response

Review 3:

Comment 3.1: Please highlight major contribution in introduction.

Response 3.1: The contribution of the article was included in the Introduction. Line 98. (#3)

Comment 3.2: Please discuss about each section in introduction such as Section 2 presents bla bla.

Response 3.2: A brief overview sentence of each section was included in the Introduction. Lines 55-63, 69-87 (#4)

Comment 3.3: Introduction is very short and brief, please improve it. 

Response 3.3: The Introduction is a bit long now with the including of the research questions, objective, contribution and overview of the sections. #1

Comment 3.4: Please provide some future directions in conclusion.

Response 3.4:  I have included into the conduction some options for future research.  Future reach included is to investigate if why private schools have a bit better maturity regarding cyber safety. As well as obtain permission to conduct the maturity test in all schools in South Africa. #24

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

My comments from the previous round of review have been appropriately addressed to the extent possible without new data collection.

Typos:

p.4 dived into different strands -> divided into different strands

p.6 a national viewpoint of cybersecurity maturity -> a national viewpoint of cybersafety maturity

[41] No journal or conference proceedings given, just title and authors

Author Response

All three issues below have been corrected:

• p.4 dived into different strands -> divided into different strands
• p.6 a national viewpoint of cybersecurity maturity -> a national viewpoint of cybersafety maturity
• [41] No journal or conference proceedings given, just title and authors

Author Response File: Author Response.docx