Ten Priority Science Gaps in Assessing Climate Data Record Quality

Round 1

Reviewer 1 Report

Your paper is written well and structured clearly, though you may consider providing more guidance to the reader at one or two occasions because you deviate from the structure suggested by the journal, which is, after introduction:

2. Materials and methods

3. Results

4. Discussion

5. Conclusions

In general, I prefer this structure and think you should restructure your subsections accordingly but I leave it to the editors to decide what structure is appropriate.

My main concern is that your concluding section does not summarise the Ten Priority Science Gaps in Assessing Climate Data Record Quality and gives no recommendations on how to resolve them. 

Typos:

Line 331: The abbreviation of Portable Document Format is "PDF" not "pdf". Either write Portable Document Format or PDF format.

Line 388: In particular, the use of the words ...

Line 540: time-*series*

Line 613: However, *the* key problem …

Table 3, row 2, column 2: There is a closing brace ")"  that does not match to an opening brace "(". Delete the closing brace.

Minor comments and suggestions: 

Lines 97 - 104: Could you refer the reader explicitly to the subsequent sections? Like e.g. "Section 2 outlines ... Section 3 presents ... Section 4 identifies and discusses ... Section 5 ..."

Line 106: Could you add a motivating overview to Section 2?

Table 3: Are there dependencies among the science gaps? E.g. in my understanding 2. depends on 5. and 6. Could you think about it and add such dependencies, if there are any, as a remark to the 2nd column and comment on dependencies also in the main text?

Line 406: There have some conference posters been published on harmonisation [see FIDCEO web site http://www.fiduceo.eu/publications] and a paper by Giering et al. is presumably to appear in a Special issue of Remote Sensing [see https://0-www-mdpi-com.brum.beds.ac.uk/journal/remotesensing/special_issues/assessment_cdr] which you could cite when accepted.

Major comments and suggestions:

To me, the text from Line 652 on reads rather "bureaucratic" than scientific. The text mainly refers to the EQC and CMS, but not to the 10 priority science gaps anymore.

Could you bring to the point what the main scientific outcome of your analysis is? What is the most important science gap? What is the gap that could be addressed most easily in relation to its impact on improving the scientific value of the data? What is the gap that requires the most research effort? How can these science gaps be addressed? What is the main message that data producers and algorithm developers should take home in your opinion? What is the homework you would give to them, if you could? You may describe how EQC and CMS can, will or must be improved to help resolve the science gaps. But your concluding section must come back to what you claim in the title of your paper.

Author Response

Dear Reviewer,

Thank you for taking the time to review our article. Below we have provided some responses to your concerns.

Restructuring: As our paper is not a typical science /experimental paper it was difficult to structure strictly following the suggested outline. We discuss the design and development of a framework for assessing the quality of climate data records and in the process identify priority areas that product developers should address in the future to improve understanding of the derived parameters. Hence results and discussion are somewhat combined.

Resolving the Science Gaps: Suggestions for resolving the science gaps are highlighted in table 3.  The recommendations are really aimed at the data producers and the agencies who fund the product development. However, the information provided in this paper is also highly relevant to the general data user who should be aware of the quality issues when using climate data sets. If more users request information about the data quality, then this will inevitably put pressure on funding agencies to start to address these science knowledge gaps in more detail.

I have added in some text in Section 4 and 5 to reflect this.

Dependencies among the science gaps: Indeed there are dependencies, and understanding each builds on the knowledge and understanding of the derived variables.

I have included a remark to reflect this in section 5.

Summary and Recommendations: I have made some edits to reflect your comments as well as the above 2 comments.

Reviewer 2 Report

It is a nicely written paper which provides very good QA guidelines for CDR provision. 

Few minor comments:

Line 182: interpolated (Level 4) - Interpolated in time?

Table 2: It is interesting to see that the last few rows of the table conatins only EU QA4ECV provided data. If I understand correctly, this project is over and I wonder how continous provision of the data is possible?

Author Response

Dear Reviewer,

Thank you for taking the time to review our article.

Amended: temporally and spatially interpolated.

QA4ECV products: Correct. We included them in the development process given our experience with the products and direct contact with the producers. My understanding is the “QA4ECV” products will no longer be developed/funded, but the algorithms may live on in the future with new funding avenues. I will clarify this in the footer of the table.

Reviewer 3 Report

Comments on “Ten priority science gaps in assessing CDR quality” by J. Nightingale et al.

How to assist end-users in selecting appropriate climate datasets for their specific purposes from the increasing amount of available products is always a challenging task. This manuscript addresses this issue with the most up-to-date projects, tools and relevant developments on the Evaluation and Quality Control (EQC) perspectives of climate data products. After the introduction of the relevant developed tools, frameworks, the authors step into the Ten priority science gaps in assessing CDR quality. Although the manuscript is well written and organized, some parts of the manuscript seem written for an internal purpose. It’s therefore suggested for some revisions before being accepted for being published.

Major concerns:

1.       In several places throughout the manuscript, this reviewer found that the statement of “… ensure all quality aspects of climate data sets can be ascertained …” seems a bit overstated. It is certainly that we should aim for addressing all quality aspects, on the other hand, are we able to do that currently with EQC? This reviewer would suggest the author think more critically on this statement.

2.       This reviewer found there is a jump from the introduction of EQC framework and tools to the discussion on “Ten priority science gaps”. The authors implemented the EQC tools (QAT, QAR, QEM, etc.) for a selection of 24 individual products across the land, ocean, and atmosphere ECV derived from satellite and in-situ observations. Nevertheless, how the results linked to the ten science knowledge gaps is not presented. This reviewer found that it is necessary to show readers how the science knowledge gap was derived from the EQC results.

3.       The recommendations for addressing the ten science gaps are very relevant. On the other hand, some parts of the discussion lack of necessary references for readers to understand further the topic.

4.       In the current manuscript, there is no discussion linked to existing project/results. For example, CORE-CLIMAX’s deliverable report on “European ECV CDR capacity assessment report”, also the work carried out in CEOS Climate WG on ECV inventory. To the least, this reviewer found that the comparison of what’ve been implemented in this study with the existing CORE-CLIMAX evaluation results would be beneficial for the community to understand the EQC concept.

Minor comments:

See attached file.

Comments for author File: Comments.pdf

Author Response

Dear Reviewer,

Thank you for taking the time to review our article. Below we have provided some responses to your concerns.

Major concerns:

1.       Statements have been removed and/or edited as appropriate and additional references have been added.

2.       A statement has been included in section 4 to more directly reflect this link.

3.       Reviewer 1 also mentioned this. We have addressed this concern with some clarification text in sections 4 and 5. The recommendations are really aimed at the data producers and the agencies who fund the product development. However, the information provided in this paper is also highly relevant to the general data user who should be aware of the quality issues when using climate data sets. If more users request information about the data quality, then this will inevitably put pressure on funding agencies to start to address these science knowledge gaps in more detail.

4.       Extensive review of existing projects (Section 2.1) was undertaken and informed the wider development of the EQC framework. Further, this paper reflects the initial development of the EQC and further development/refinement is being undertaken in a new contract led by another European Contractor. It is likely the concepts will develop and change during the next phases and this will be published in due course.

Minor comments: Amended and references provided.