Sustainable Urban Development: A Review of Urban Sustainability Indicator Frameworks

Round 1

Reviewer 1 Report

Dear authors,

Thank you very much for your efforts writing this paper.

This manuscript is a review of urban sustainability indicator frameworks. The topic of the paper is interesting and relevant.

The analysis is based on the content analysis of 50 reviewed frameworks. The main contribution of this paper is to identify the key indicators of urban sustainability

The following comments aim to improve the quality of this paper:

C1. Line 52: Please improve grammar and meaning.

C2. Section 1: Please include research question/s and hypothesis/es.

C3. Section 1: It is recommended to explain the manuscript organization at the end of section 1.

C4. Section 5: Please consider adding the limitations of this investigation.

C5. Section 5: Please consider adding suggestions for future research.

I congratulate the authors for this interesting investigation and wish them the most success in their research activities.

Thank you very much for your efforts and for your valuable scientific contribution.

Author Response

Dear reviewer, many thanks for your review and appreciation of our research. We considered your valuable comments very carefully and tried to incorporate them into our manuscript – we hope that it improved the quality and soundness of the article. Other changes in the manuscript are based on the comments of the second reviewer.

Here are our answers to your comments:

C1 Line 52: Please improve grammar and meaning.

The manuscript was checked grammatically and verified by a native speaker. The mentioned shortcomings were corrected (see rows 52-56 of the corrected manuscript with tracked changes).

C2 Section 1: Please include research question/s and hypothesis/es.

Research questions were added at the end of the Introduction section (r. 151-160). We didn´t formulate specific hypotheses; they would be closely related to the research questions. However, we discuss some of our expectations and findings in the Discussion section.

C3 Section 1: It is recommended to explain the manuscript organization at the end of section 1.

Thank you for your comment - however, we believe that the introduction of the manuscript is compiled following most scientific articles and concludes with research objectives. The methodology and the main research steps are then given in the Material and Methods section. The presentation of the results is identical to the main steps of content analysis shown in Figure 2.

C4 Section 5: Please consider adding the limitations of this investigation.

A new sub-section “4.4. Limitations of the study“ was added at the end of the Discussion section.

C5 Section 5: Please consider adding suggestions for future research.

A new sub-section “4.5. Insights into further research“ was added at the end of the Discussion section.

Once more, many thanks for your review!

Peter Mederly

Reviewer 2 Report

The article presented for review is interesting and concerns an important topic of defining and, above all, measuring sustainable development in cities. Before being published, however, it requires some significant corrections.
1. The authors in the second paragraph and a quite general abstract, present the purpose of the paper. In my opinion, this information should appear in the first part of the introduction.
2. In my opinion, the purpose of the study should be more precisely formulated. I have doubts whether the analysis itself can be the goal of the research. In my opinion, analysis is a standard element of research, and as such, should not be indicated as a goal of scientific study.
3. I feel a lack of explanations regarding the research gap.
4. In my opinion, it also lacks a clear explanation of the added value of the paper. What new knowledge this paper provides?
5. It is also necessary to describe the limitations of the conducted study.
6. It also lacks directions for future research.
7. It is necessary to explain what this means: so-called Bellagio principles.
8. The authors analyze the indicators of sustainable development grouping in three and four dimensions. In my opinion, they forgot about spatial development, which is very important for cities. Some of the indicators on the created list directly relate to this dimension, such as indicators from the groups: land use, housing, green space. If the authors decided to separate the institutional area from the social dimension of sustainable development, the spatial area, which can be distinguished from the environmental dimension, should be also described in the paper. 
9. Some conclusions made by the authors are quite general eg.: In the developed countries environmental categories and indicators are overrepresented, in the developing countries there are mostly social and economic categories and indicators. It would be more important to try to answer and explain why this is happening?
10. In the introduction, a lot of space is devoted to the synergy between the dimensions and indicators of sustainable development. For me, it was a preview of a very interesting issue concerning the study of the relationship between these areas. Meanwhile, there is no explanation of this issue in the further part of the paper. We do not learn from it how these relationships are created, nor how they could be created or should be created. In my opinion, this is missing from the paper.
11. I also have comments on the methods of creating the database of the papers. Why were searches for whole phrases in the English language like: "urban sustainability frameworks"; "sustainable cities indicators" and a single word in German: "Indikatoren". It is not important for the article because the authors have analyzed 50 different databases with indicators but unfortunately causes that the applied keywords selection, may be questioned. It must be clarified for sure.
12. I also note some linguistic shortcomings. It is necessary to complete the sentence:
Linked to the growing danger to the environment from cities and their inhabitants, the topic of urban development has become the subject of both European and? discussions.
It is also worth paying attention to the style of speech:
Sustainable cities and urban sustainability are defined in various manners, characterized by various criteria that concern various areas according to the specific conditions and needs of cities [12, 13].
13. What does the abbreviation ICLEI mean? 

Author Response

Dear reviewer, many thanks for your review and for your valuable comments. We considered them very carefully. Based on this and the comments of the second reviewer, we updated our manuscript – we hope that it improved the quality and soundness of the article.

Here are our answers to your comments:

1. The authors in the second paragraph and a quite general abstract, present the purpose of the paper. In my opinion, this information should appear in the first part of the introduction.

Thank you for your comment - however, we believe that the introduction of the manuscript is compiled following most scientific articles and concludes with research purpose and objectives. Nevertheless, we added one sentence at the end of the abstract and also more elaborate the research objectives – see rows 151-160 of the corrected manuscript with tracked changes.

2. In my opinion, the purpose of the study should be more precisely formulated. I have doubts whether the analysis itself can be the goal of the research. In my opinion, analysis is a standard element of research, and as such, should not be indicated as a goal of scientific study.

Thanks for your comment and reflection. We re-formulated and more elaborated on the research objectives in the form of research questions – see the previous answer.  

3. I feel a lack of explanations regarding the research gap.

Thanks for your comment and reflection. A new sub-section “4.4. Limitations of the study“ was added at the end of the Discussion section.

4. In my opinion, it also lacks a clear explanation of the added value of the paper. What new knowledge this paper provides?

Thanks for your comment and reflection. Nevertheless, we believe, that the main results and added value of our research are sufficiently stated in several places - (1) briefly in the Abstract, (2) at the end of section 1. Introduction (r. 161-169) and (3) in the conclusion of the article (r. 574-588).

5. It is also necessary to describe the limitations of the conducted study.

The limitations of the research are connected to the research gaps. As mentioned earlier, a new sub-section “4.4. Limitations of the study“ was added at the end of the Discussion section.

6. It also lacks directions for future research.

A new sub-section “4.5. Insights into further research“ was added at the end of the Discussion section.

7. It is necessary to explain what this means: so-called Bellagio principles.

The full term "Bellagio Sustainability Assessment and Measurement Principles" has been added. However, the explanation would be lengthy and, moreover, it is directly and in detail provided in the reference (29).

8. The authors analyze the indicators of sustainable development grouping in three and four dimensions. In my opinion, they forgot about spatial development, which is very important for cities. Some of the indicators on the created list directly relate to this dimension, such as indicators from the groups: land use, housing, green space. If the authors decided to separate the institutional area from the social dimension of sustainable development, the spatial area, which can be distinguished from the environmental dimension, should be also described in the paper.

Thanks for your comment and reflection. The spatial level of sustainability is indeed essential. Still, only in one of the 50 evaluated frameworks, we found that it was defined as a separate dimension of sustainability (the same is valid for the cultural dimension). The absolute majority of approaches state three or four dimensions, which we also consider in our research. We understand spatial development to be one of the possible approaches (of course, very important and useful) by which it is appropriate to assess sustainability (but not only the environment but also the social and economic aspects). On the contrary, the institutional dimension is mentioned in many cases. In the Introduction (1.1), we explain why it should be distinguished as a separate fourth dimension. We also have added a brief text on the spatial aspect and cultural dimension of urban sustainability (subsection 3.2, r. 308-315).

9. Some conclusions made by the authors are quite general eg.: In the developed countries environmental categories and indicators are overrepresented, in the developing countries there are mostly social and economic categories and indicators. It would be more important to try to answer and explain why this is happening?

The basic explanation of this fact was added in the discussion section (r. 512-515).

10. In the introduction, a lot of space is devoted to the synergy between the dimensions and indicators of sustainable development. For me, it was a preview of a very interesting issue concerning the study of the relationship between these areas. Meanwhile, there is no explanation of this issue in the further part of the paper. We do not learn from it how these relationships are created, nor how they could be created or should be created. In my opinion, this is missing from the paper.

Thanks for your comment and reflection. Our research understands dimensions as key areas generally recognized in sustainability issue (there is only a difference whether authors consider 3 or 4 dimensions). Indicators, in turn, are specific, measurable parameters expressing the state or progress of a given area and are often specific to different cities. We understand thematic areas as a certain "intermediate space" connecting indicators with dimensions - they are more specific than dimensions and include various indicators.

If the reviewer had in mind the relationships between these different levels, these were not the subject of our research. Nevertheless, we consider it important that all the main thematic areas are represented in the system through specific (and appropriate) indicators not to have a "knowledge gap" in a certain area. It is also important to maintain a balance between the individual dimensions. We discuss this in subsection 4.3. and this is also stated by most of the authors quoted in the Introduction section. However, it must not be forgotten that although some level of standardisation of the procedures in determining the main categories and indicators of sustainability is needed, it is always necessary to take the specific conditions and goals of the city into consideration.

11. I also have comments on the methods of creating the database of the papers. Why were searches for whole phrases in the English language like: "urban sustainability frameworks"; "sustainable cities indicators" and a single word in German: "Indikatoren". It is not important for the article because the authors have analyzed 50 different databases with indicators but unfortunately causes that the applied keywords selection, may be questioned. It must be clarified for sure.

Thanks for your comment – in fact, not the right German keywords were listed in the manuscript. We corrected this mistake - the right terms are „Indikatoren der nachhaltigen Stadtentwicklung“; „Nachhaltigkeitsbericht der Stadt“. We have used German keywords due to the fact, that the issue of sustainable cities measuring and assessment is the most commonly used and very advanced in Germany. In our view, some misinterpretation or inaccuracies in our research due to the selection of these keywords could not have occurred. Of course, it is questionable whether we have managed to choose the most representative approaches worldwide, as some of the possible frameworks may not have been published in the given languages (see new subsection 4.4). However, this is limiting for most research of a similar type.

12. I also note some linguistic shortcomings. It is necessary to complete the sentence...:

It is also worth paying attention to the style of speech...

The manuscript was checked grammatically and verified by a native speaker. The mentioned shortcomings were corrected (see rows 52-56 of the corrected manuscript with tracked changes).

13. What does the abbreviation ICLEI mean?

The full term of this organisation “Local Governments for Sustainability“ has been added (https://www.iclei.org/).

Once more, many thanks for your review!

Peter Mederly

Round 2

Reviewer 2 Report

The article has been corrected in line with the submitted comments. However, in my opinion, the procedure of searching keywords adopted by authors could be better and more precisely prepared. The authors searched for specific complex phrases. In my opinion, it would be worth using a different search method, including * as a symbol that replaces the ending of individual words. It would also allow the identification of the plural words. At this stage, however, I do not comment on the search method adopted by the authors. I recommend that the authors look at the different ways of searching the databases of articles and use them in subsequent works.