A Model to Manage Cooperative Project Risks to Create Knowledge and Drive Sustainable Business

Round 1

Reviewer 1 Report

The paper is well written and focuses on the interesting topic of how to improve the management of cooperative projects. The evaluation gives a good overview of the achieved results and their implications for the practice. On the other hand, the paper lacks more technical details, e.g. the calculation of the model is available as some software tool or particular calculations like experiments? 

I recommend checking the paper according to the journal template.

page 2, line 64 "to the lack of"

section 2.1 - add more use cases

section 2.4 - add more use cases

The comprehensive overview of existing use cases in all four aspects of the proposed model will create an important basis for the final comparison.

Table 3 has a relatively high redundancy with the Literature review.

The paper has two tables no.3. 

The second table 3, part D5, mentioned decision-making organizations. I recommend distinguishing between the formal and informal decision-making roles. For example, the formal is based on the official agreements;  the informal role can be extracted from the data.

The model was tested on the network of the three organizations. What about scalability? How will a higher number affect the calculation?

It is possible to decompose the model to persons in the organizations?

I recommend extending the conclusion with the final comparison with existing case studies or similar models, i.e. highlight the advantages.

Author Response

Journal: MDPI Sustainability

Special number form Sustainability: Project Risk Assessment and Corporate Behavior: Creating Knowledge for Sustainable Business

Summitted paper: A model to manage cooperative projects risks to create knowledge and drive sustainable business

Manuscript ID sustainability-1208388

Authors: Marco Nunes * , António Abreu *, Célia Saraiva *

We would like to thank you all your comments and improvement suggestions as well the time you spend in the analysis of the proposed manuscript.  We believe that your comments and improvement suggestions do contribute to a better overall quality and interest of the proposed manuscript. All changes done in the original manuscript are marked in red. All reviewers’ suggestions and comments as follow were taken into consideration and properly assessed in the submitted manuscript. We hope that we have this way incorporated in an efficiently way all the considered impartments suggestions for the reviewers in the new version of the manuscript. Thank you very much for your help.

Answers to reviewers’ suggestions (yellow marked):

Reviewer 1:

The paper is well written and focuses on the interesting topic of how to improve the management of cooperative projects. The evaluation gives a good overview of the achieved results and their implications for the practice. On the other hand, the paper lacks more technical details, e.g. the calculation of the model is available as some software tool or particular calculations like experiments? 

Thank you for this note.

I recommend checking the paper according to the journal template.

Thank you for this note. The paper was revised and it is now written according to the journal template

page 2, line 64 "to the lack of"

Thank you for this note. Correction has been made.

section 2.1 - add more use cases

Thank you for this note. More use cases have been added.

section 2.4 - add more use cases

Thank you for this note. More use cases have been added.

The comprehensive overview of existing use cases in all four aspects of the proposed model will create an important basis for the final comparison.

Thank you for this note. We fully agree with this note.

Table 3 has a relatively high redundancy with the Literature review.

Thank you for this note. We partially agree with this point. The reason for it is that with the table 3 we want to clearly highlight the specific contributions of each of the scientific pillars to the development of the MCPs model. Nevertheless, we decided to illustrate such information with text instead of the use of a Table.

The paper has two tables no.3. 

Thank you for this note. Correction has been made.

The second table 3, part D5, mentioned decision-making organizations. I recommend distinguishing between the formal and informal decision-making roles. For example, the formal is based on the official agreements;  the informal role can be extracted from the data.

Thank you very much for this note. We fully agree with this point. However, as several literatures says it is very difficult if not impossible to distinguish formal from informal relationships within an organization, therefore we assume in the present work that the captured interactive dynamic relationships are the mix of formal and informal relationships. Otherwise, to the model output accurate results we would have to compare between entities with higher power or responsibilities with those with less power, and still conduct a parallel analysis regarding the correlation between power and centrality. Such analysis could very well be part of a further development of the model and therefore we introduced this aspect as an improvement for future developments. 

The model was tested on the network of the three organizations. What about scalability? How will a higher number affect the calculation?

Thank you very much for this note. There’s absolutely no problem regarding scalability to the proposed model in the manuscript. This is because the proposed model is not affected by any type of estimation such as for example the R2 in linear regression. The mathematical approach deals only with absolute results (pure quantitatively results). Nevertheless, is there is a huge number of entities to be analyzes it is recommendable to use indexation (ranging from 0 to1) instead of absolute numbers in the outputted results scale.

It is possible to decompose the model to persons in the organizations?

Thank you very much for this note. Yes, it is possible to decompose the model to persons in the organizations. The model analysis entities, and such entities can be persons, organizations, groups of people and so on. This information is also in the paper available in the conclusion section.

I recommend extending the conclusion with the final comparison with existing case studies or similar models, i.e. highlight the advantages.

Thank you very much for this note. We fully agree with this point and therefore we introduced in the conclusion section the advantages and disadvantages of the model when comparing with other models.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors,

Thank you for coming up with this framework. I am glad I was able to review and below are my comments.

1. There are many citations are missing in the article please cite.
2. As we are in 2021 and there are only very few references from 2019 to 2021. Some of the references are very old. I strongly recommend referring only 5 years old articles unless it is a classic article related to the current research/Topic.
3. Some of the reference are just URLs without author’s name, please incorporate authors name in the reference where it is missing.
4. As mentioned in the article MCPx model was developed based on four scientific pillars but the literature reviews section lacking latest literature. Also, 2.4. Business Intelligence in organizations section is lacking literature support.
5. As this is a framework, please separate, Conclusions, Implications and further section separately and elaborate each sections.

Regards,

Author Response

Reviewer 2:

Thank you for coming up with this framework. I am glad I was able to review and below are my comments.

Thank you very much for this note.

1. There are many citations are missing in the article please cite.

Thank you very much for this note. After revising the document, we corrected all references and adjusted them to the journal standards.

1. As we are in 2021 and there are only very few references from 2019 to 2021. Some of the references are very old. I strongly recommend referring only 5 years old articles unless it is a classic article related to the current research/Topic.

Thank you very much for this note. We agree with your point to the extent possible and therefore we searched for more recent references regarding the literature reviews that supports the actual document. However, for the purpose of a continuous line of thought and scientific support of what is presented in this work, we kept important references essentially regarding the pillars that support the proposed model in the manuscript.

1. Some of the reference are just URLs without author’s name, please incorporate authors name in the reference where it is missing.

Thank you very much for this note. We did correct all references according to the journal standards.

1. As mentioned in the article MCPx model was developed based on four scientific pillars but the literature reviews section lacking latest literature. Also, 2.4. Business Intelligence in organizations section is lacking literature support.
2. Thank you very much for this note. We made improvements regarding the latest literature reviews in all the four scientific pillars that support the propose model in the manuscript.

1. As this is a framework, please separate, Conclusions, Implications and further section separately and elaborate each sections.

Thank you very much for this note. We also agree that conclusions, Implications and further developments should be separately elaborated in each section. We have done this change in the new version of the document.

Author Response File: Author Response.pdf

Reviewer 3 Report

Authors have proposed a model called MCPx for quantitatively measuring the cooperation between different organizations in collaborative projects. They have measured the communication, information sharing, trust, problem solving and decision making entities using graph centrality metrics. Overall, the paper is quite understandable and describes clearly all the relevant topics related to the proposed model, such as PMI, risk management, cooperative networks and  business intelligence tools. They have applied the model to the case study and explained in detail the insights based on the model output.

However, the paper required substantial proofreading by native english speakers and correction of many typos.

Line 64: repetition of “the lack”

Figure 3: pictures b & d have low resolution. Not legible

Line 363: typo. three -> four

Line 365: same as above. 3 -> 4

Table 3: in Project risk management row, can you elaborate how 1) Behavioral and 2) assignment of tasks,  are considered as risk factors?

Table 3: What is the idea behind highlighting cooperative networks as one pillar? Doesn’t Social Network Analysis inherently include cooperative networks?

Table 3: in SNA row, correct “five key networks” -> “five key factors”

Label Table 3 is duplicated. 

Table 3(2): why are you just considering email as a communication medium? How about  meetings, sync-ups, chats, etc?

Table 3(2): you need to clearly explain how you compute xji in each of the equations from 1 to 4 separately. How do you count the number of links and their weights? Are the links directed or undirected? You can dedup trust, problem solving and Decision making sections as they basically rely on the same graph centrality metric.

Line 424: why is email considered an undirected link? Do you count the email reply as a separate link?

Figures 7 to 16 have very low resolution and hard to read. 

Line 640: typo. organization 2 -> organization 1.

Line 752: typo. 2 -> 1.

Line 783: typo. four -> five

Line 831: typo. gute -> gut

In general, how do you derive action items from given data and patterns?  You have explained the correlation of different data points. but that is a manual process and probably involves expert analysts. Can you automate the insights gathering from given data points?

Can you verify your finding with actually what happened in the project? Basically, correlation is not causation. How do you know your data based insights were actually true? And risks were indeed the factors which led to slow down of the project in particular phase?

Some of the conclusions you made about organization 2 being a critical enterprise might be perfectly normal based on type of project or phase of the project. So, it is not insightful to say that it is central based on graph theory sense. Can you come up with a single metric which combines various factors and can holistically tell the health of cooperation between different organizations?

If BI systems use machine learning, especially CNN or reinforcement learning, will the MCPx become redundant or obsolete as the NN will automatically find the patterns from given data?

Author Response

Reviewer 3:

Authors have proposed a model called MCPx for quantitatively measuring the cooperation between different organizations in collaborative projects. They have measured the communication, information sharing, trust, problem solving and decision making entities using graph centrality metrics. Overall, the paper is quite understandable and describes clearly all the relevant topics related to the proposed model, such as PMI, risk management, cooperative networks and  business intelligence tools. They have applied the model to the case study and explained in detail the insights based on the model output.

However, the paper required substantial proofreading by native english speakers and correction of many typos.

Thank you very much for this note. We took into consideration all the reviewers’ comments and done our best to improve the current manuscript. The actual version of the manuscript was read and corrected by a native English speaker.

Line 64: repetition of “the lack”

Thank you very much for this note. Correction has been done.

Figure 3: pictures b & d have low resolution. Not legible

Thank you very much for this note. Pictures have been reedited to improve quality.

Line 363: typo. three -> four

Thank you very much for this note. Correction has been done.

Line 365: same as above. 3 -> 4

Thank you very much for this note. Correction has been done.

Table 3: in Project risk management row, can you elaborate how 1) Behavioral and 2) assignment of tasks,  are considered as risk factors?

Thank you very much for this note. This point is very well outlined and enabled us to be more precise regarding to what we developed and how it works. In the present work we will only consider behavioral risks as proposed by Abreu 2018 (in a high-level approach), which comprise the different dynamic relationships that are established between the different project partners across the accomplishment of a project. In the actual work, we defined five dimensions (((1) communication, (2) information sharing, (3) trust, (4) problem-solving and (5) decision-making) to characterize the different dynamic relationships established between the different project partners across a project lifecycle. So, we will not consider the assignment of tasks as a risk factor in the present work. We deleted Table 3 by suggestion of another reviewer (which we also agree) and the detailed information about the individual contributions of each pillar is now in text form.

Table 3: What is the idea behind highlighting cooperative networks as one pillar? Doesn’t Social Network Analysis inherently include cooperative networks?

Thank you very much for this note.  We do not consider that SNA inherently includes cooperative networks. SNA are a set of tools based on graph theory that translate dynamic interactions also called of dynamic behaviors among a set of actors, where such behaviors have a certain interpretation in a specific context, while being able to quantitatively measuring such interactions. Cooperative networks is the theory or scientific field that studies how cooperation emerges, evolves and develops within a social network, but not only, where it may or may not use the theory of SNA. IN he case of the present work, the cooperative networks pillar aims to define or assume that within a project environment there is a level of joint work denominated as cooperation which includes some interactive dynamic activities such as, information sharing, problem-solving, communication and so on. The next working version of the proposed model in this work will introduce the different joint work levels (networking, coordination, cooperation and collaboration) to perform a more detailed approach regarding to collaboration within project environments.

Table 3: in SNA row, correct “five key networks” -> “five key factors”

 Thank you very much for this note. We deleted Table 3 by suggestion of another reviewer (which we also agree) and the detailed information about the individual contributions of each pillar is now in text form.

Label Table 3 is duplicated. 

Thank you very much for this note. Correction has been done.

Table 3(2): why are you just considering email as a communication medium? How about  meetings, sync-ups, chats, etc?

Thank you very much for this note. In this version of the proposed model we did only considered exchanged information via email as communication (general communication) because information that flows within the email communication network is not able to be detailed accessed unlike information collected in surveys as for example information sharing, problem solving or even decision making. However, it is absolutely correct that all form of interaction comprise communication.

Table 3(2): you need to clearly explain how you compute xji in each of the equations from 1 to 4 separately. How do you count the number of links and their weights? Are the links directed or undirected?

Thank you very much for this note. We have done improvements in explaining how the mathematical approach is to be carried out. These are available in Table: MCP model´s Social Network Analysis centrality metrics

You can dedup trust, problem solving and Decision making sections as they basically rely on the same graph centrality metric.

Thank you very much for this note. Yes, that is correct. We can only deduct, and therefore we only need one same metric which in this case is the in-degree. What the model is looking for is the most or less central entity (person, group or organization) within a given network and how the difference is related to the other members within a project social network. For this matter one metric such as the in-degree is very suitable to do the job. However, other metrics could be used such as out-degree, betweenness, eigenvector and even closeness. But when applying such typical network centrality metrics (by opposition to node metrics such as the typical in, out and total degrees) we would be focusing the catherization of the group instead of the individual entity. By doing so, we would be missing the impact of some entities within the project social network. That is why the analysis are displayed always with an absolute degree metric (in this version of the manuscript is the in-degree) and the average values in order to translate the impact of some entities within the project social network as a whole. The version of the propose model in this work is to characterize both, group (by using average in-degree) and individual (by using in-degree), however giving much more focus on individual analysis and the impact that they may represent for a given project social network, rather than focusing in group analysis, where very often may lead to misinterpretation due the application of normative metrics such as simple modes, averages and standard deviations   

Line 424: why is email considered an undirected link? Do you count the email reply as a separate link?

Thank you very much for this note. In this work Email communication is considered an undirected link because there might be no authorization according to GDPR restrictions, to which most organization must be according to in developed countries (namely in Europe), unless there is a special authorization and consensual agreement between organization and its employees. To consider email as an in directed link we would have to have access to the email communication in a way that is not (still) legal and filter what, when, to whom and how was sent and replied. For this reason, we decided to use email communication as an undirected link and only count the amount of sent / received emails without accessing their content in detail. However this point is considered as a further development in the current manuscript.

Figures 7 to 16 have very low resolution and hard to read. 

 Thank you very much for this note. The quality of picture 7 to 16 has been improved.

Line 640: typo. organization 2 -> organization 1.

Thank you very much for this note. Correction has been done.

Line 752: typo. 2 -> 1.

Thank you very much for this note. Correction has been done.

Line 783: typo. four -> five

Thank you very much for this note. Correction has been done.

Line 831: typo. gute -> gut

Thank you very much for this note. Correction has been done.

In general, how do you derive action items from given data and patterns? 

Thank you very much for this note. As we state in the manuscript, We do not effectively derive action from given data. That would simply be impossible due the high dynamic environment of projects and the huge amount of uncertainty that is comprised in human behavior. Essentially what the model does is a resent “call of attention” regarding past cooperation trend, highlighting some interactive pattern too much or too less centralized.

You have explained the correlation of different data points. but that is a manual process and probably involves expert analysts. Can you automate the insights gathering from given data points?

Thank you very much for this note. In fact, the analysis presented in the manuscript is an almost step-by-step analysis in order to detailed explain how the automated model analysis data and outputs results. The involvement of expert social network analysts is definitely an advantage when a project social network is being analyzed, however is not critical its presence. Yes, it is possible to automate the insights collected from data points. That is the purposed of the business intelligence architecture incorporated in the proposed model. Once the model is in operation, all results are automatically outputted. The interpretation of results however, it is always recommendable to be analyses and followed up by network analysts or someone that has high experience in team management.

Can you verify your finding with actually what happened in the project? Basically, correlation is not causation. How do you know your data based insights were actually true? And risks were indeed the factors which led to slow down of the project in particular phase?

Thank you very much for this note. It is only possible to establish an effective correlation between past behaviors and past outcomes once a substantial number of samples have been analyzed under the same conditions. It is possible to do that, however the proposed model would need to first create a knowledge database where information regarding past behavior and a respective outcome would be recorded, and thus estimate a correlation that could be used for future estimations. Causation will never be able to be done, because the model only analysis five dimensions of dynamic interaction, and in behavioral science causation is very unlikely to be accurately determined. In this line of thought, one is not absolutely sure that a particular behavioral trend within a given project phase will lead to a given outcome. However, as mentioned before, the model only aims to make a quantitative call of attention in a timely manner in order to spot unbalanced cooperative interactions. After that, the experience of well-rounded project responsible or network analysts will definitely conduct the rest of the assessment.

Some of the conclusions you made about organization 2 being a critical enterprise might be perfectly normal based on type of project or phase of the project. So, it is not insightful to say that it is central based on graph theory sense. Can you come up with a single metric which combines various factors and can holistically tell the health of cooperation between different organizations?

Thank you very much for this note. It is absolutely correct that is not insightful to simply say that an organization is critical or not only based on graph there. However, we believe it is insightful to state that an organization is or not central based on graph theory sense, specially if a substantial number of dimensions are analyzed (more than those five dimensions proposed in the present work). That is why it is always described in the manuscript the need for a follow up assessment in order to really access the reasons behind such observed behaviors. The impacts of observed behaviors can only be suspected that will derive into something which can turn into a risk to the successful deliver of a project. In this line of thought we believe that organization 2 has indeed a very distinct behavior from the other organization when it comes to cooperation. This distinct behavior needs to be followed up to ascertain if the observed behavior is within the expected behavior (a certain behavior where nobody would get surprised to quantitatively acknowledge) on one side, and if it is according to good principles of cooperation in project environments on the other side, which usually are determined by experience of project responsible or network analysts.

If BI systems use machine learning, especially CNN or reinforcement learning, will the MCPx become redundant or obsolete as the NN will automatically find the patterns from given data?

Thank you very much for this note. In the version of the present manuscript we do only use the capabilities of a business intelligence architecture to automate the collection and analysis data process. However, If BI systems use machine learning, especially CNN (Convolutional Neural Networks) or reinforcement learning the MCPx would never become redundant and obsolete. IN order to use either CNN or reinforcement teaching one needs first to have data in order to train the model either in a supervised or other type of machine learning model. The application of a ML or NN type would only increase the capabilities of the propose model. This happens because in order to get quantitative insights regarding dynamic behavioral interactions within a project social network one will need the metrics used by the proposed model in this work, in order to identify unbalanced central roles or trends. The incorporation of such ML or NN techniques would be considered as an upgrade to the propose model in this work which would give the proposed MCPx model a particular powerful AI capability in order to generate knowledge and estimate future outcomes based on the analysis of past collaborative trends. We did add this information in the further developments section.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Thanks for the substantial revision of the article and detailed responses to the questions raised in the first review.  Except few minor English grammar/style issues, the article is acceptable.