Towards Analyzing the Complexity Landscape of Solidity Based Ethereum Smart Contracts

Round 1

Reviewer 1 Report

Thank you for giving me the possibility to review your work. I hope the feedback provided help you improve your paper and research as well as a further application of your work for companies, researchers and society. 

Your work is very well structured, the information is clearly presented and the objectives are accurate. 

The description, scope and method is clearly described. 

There is room for improvement regarding literature review as you go very straight to the point and understanding of the framework and other related theories is necessary. I will recommend you to include also information about location-based cited in your review as a current trend for cryptocurrency and blockchain as Location Based Services (LBS) and proof on location for smart contract. Some works to be cited are  

Lee, K. (2017). Towards on blockchain standardization including blockchain as a service. Journal Of Security Engineering, 14(3), 231-238. doi: 10.14257/jse.2017.06.05

Palos-Sanchez, P.R.; Saura, J.R.; Reyes-Menendez, A.; Esquivel, I.V. Users Acceptance of Location-BasedMarketing Apps in Tourism Sector: An Exploratory Analysis.J. Spat. Organ. Dyn.2018,6, 258–270 

Regarding the conclusions, it is necessary to present clear implications for companies that the different types of code and contracts have in practical terms so i will ask the authors to reinforce this point too in a clear way. Maybe using a table to present and compare results. Try to stay not in a very high level as this is one of the weaknesses of blockchain research. 

The conclusions are also clear. Congratulations for your work. 

Author Response

Dear Reviewer,

Thank you very much for your valuable comments, it really helped me improve the paper. Please find below my modifications in the paper according to your observations.

Comment1: There is room for improvement regarding literature review as you go very straight to the point and understanding of the framework and other related theories is necessary. I will recommend you to include also information about location-based cited in your review as a current trend for cryptocurrency and blockchain as Location Based Services (LBS) and proof on location for smart contract. Some works to be cited are  

Lee, K. (2017). Towards on blockchain standardization including blockchain as a service. Journal Of Security Engineering, 14(3), 231-238. doi: 10.14257/jse.2017.06.05

Palos-Sanchez, P.R.; Saura, J.R.; Reyes-Menendez, A.; Esquivel, I.V. Users Acceptance of Location-BasedMarketing Apps in Tourism Sector: An Exploratory Analysis.J. Spat. Organ. Dyn.2018,6, 258–270

Answer1: I have extended the related work section of the paper. The very first paragraph cites the above works as well as other, more fundamental works on blockchains.

Comment2: Regarding the conclusions, it is necessary to present clear implications for companies that the different types of code and contracts have in practical terms so i will ask the authors to reinforce this point too in a clear way. Maybe using a table to present and compare results. Try to stay not in a very high level as this is one of the weaknesses of blockchain research.

Answer2: Thank you for this comment, I also favor research work that is practically applicable. Assuming you referred to the discussion section instead of the conclusions (you write that conclusion section is appropriate in the next paragraph of your review), I have extended it with several paragraph and a table summarizing the outcome of the study. I also compare the metric values with typical OO metric values and provide some practical implications of the findings in this section.

Reviewer 2 Report

In the paper the author propose a static analysis of the metrics of smart contracts written in the Solidity contract-oriented language on the Ethereum network, based on a proposed static OO metrics.

The paper is well written and data supports conclusions about the hypothesis.

However some sections of the paper seems to be out of place and the structure results confusing

abstract: the introduction seems to be quite extendend with respect to data discussion a balance between these two parts is recommended. Aims and scope of the paper should be highlighted. There is no explanation for the choice of the proposed OO metrics  and no mention on the static analysis of smart contracts on the Ethereum network which is the real focus of the paper

line 37 'QA' abbreviation should be explained as first introduced in the text

line 40 the same for DAO and in line 44: for 'OO' 

line 50: I suggest to convert footnotes to references

line from 53 to 61: discussion of data and conclusions should be reported in the appropriate section of the paper not in the introduction 

line 156 and line 168: what is the real final number of smart contracts ? (208,639....178,987)

line 414 to 483: related work

This section should, in my opinion, be transferred in the introduction section, just before line 53 and as a prosecution of statement  in lines  43-47: this may reinforce the choice the author made, and justify the used tool.

line 484: conclusions: few results were reported instead this section seems to be an explanation of future research activities

references:

I suggest to mention studies on sustainability of the blockchain and/or Solidity Based Ethereum Smart Contracts, if present as in:

https://0-doi-org.brum.beds.ac.uk/10.1007/s11625-016-0412-2

https://0-doi-org.brum.beds.ac.uk/10.3390/su9122214

https://0-doi-org.brum.beds.ac.uk/10.1016/j.cosust.2017.04.011

Author Response

Dear Reviewer,

Thank you very much for your valuable comments. It really helped me to improve the paper. PLease find the list of modifications in the paper according to your suggestions.

Comment1: the introduction seems to be quite extendend with respect to data discussion a balance between these two parts is recommended.

Answer1: I have removed the last paragraph from the introduction as you suggested below. Moreover, discussion section is extended significantly with a new table and text. However, I would like to note that the data discussion part is not only in the Discussion section, but in the preceding Metric Distributions section as well. They together significantly longer than the introduction.

Comment2: Aims and scope of the paper should be highlighted. There is no explanation for the choice of the proposed OO metrics  and no mention on the static analysis of smart contracts on the Ethereum network which is the real focus of the paper.

Answer2: As suggested, I have moved the static analysis tools part from the related work as well as the related OO studies into the introduction. Several sentences have been added to highlight the aim of the work, with the long term goal of having the same SE toolset for smart contract developers than that available for OO program developers. The referenced previous OO work also justifies my metric choices (C&K metrics, best bug predictor metrics, etc.).

Comment3: line 37 'QA' abbreviation should be explained as first introduced in the text

Answer3: Quality Assurance is added to the tex.

Comment4: line 40 the same for DAO and in line 44: for 'OO'

Answer4: Decentralized Autonomous Organization and object-oriented are added to the text.

Comment5: line 50: I suggest to convert footnotes to references

Answer5: I have removed all the footnotes from the paper according to the style-guide of the journal.

Comment6: line from 53 to 61: discussion of data and conclusions should be reported in the appropriate section of the paper not in the introduction

Answer6: I have moved this part to discussion.

Comment7: line 156 and line 168: what is the real final number of smart contracts ? (208,639....178,987)

Answer7: 178,987 is the number of contracts, while 208,639 is the number of contracts, libraries, and interfaces together. I have added a sentence to the text clarifying this fact.

Comment8: line 414 to 483: related work

This section should, in my opinion, be transferred in the introduction section, just before line 53 and as a prosecution of statement  in lines  43-47: this may reinforce the choice the author made, and justify the used tool.

Answer8: Thank you for this comment, a very good suggestion, I have moved the part in question to the introduction as described in Answer2.

Comment9: line 484: conclusions: few results were reported instead this section seems to be an explanation of future research activities

Answer9: This part just briefly summarizes our findings, the detailed discussion is in the preceding section. However, I have changed the title of this section to "Conclusion and Future Work" to better reflect its content.

Comment10: references:

I suggest to mention studies on sustainability of the blockchain and/or Solidity Based Ethereum Smart Contracts, if present as in:

https://0-doi-org.brum.beds.ac.uk/10.1007/s11625-016-0412-2

https://0-doi-org.brum.beds.ac.uk/10.3390/su9122214

https://0-doi-org.brum.beds.ac.uk/10.1016/j.cosust.2017.04.011

Answer10: Thank you for the suggestion. I have extended related work with the proposed publications (first paragraph).