Triple Collocation-Based Assessment of Satellite Soil Moisture Products with In Situ Measurements in China: Understanding the Error Sources
Round 1
Reviewer 1 Report
The authors have done extensive analysis of three soil moisture data sets, namely SMAP L3, SMOS L3SM, and ESA CCI over China using triple collocation approach. The results are of interest to the scientific community and are presented well. The paper concludes that SMAP L3 and CCI products significantly perform better than the SMOS L3SM over China. This is not surprising and has been known due to the following elements:
1) presence of RFI over China which significantly contaminates SMOS observations, and
2) known algorithmic issues in the SMOS L3SM, which must be indicated in this paper. Please see the news:
https://www.catds.fr/News/Correction-of-the-L3-SM-and-VOD-from-CATDS
While the bug has been sorted out and operational data is corrected from mid July 2019 onward, the data used by the authors (2015 to 2018) is further degraded in quality due to bugs.
Given the above issues with the SMOS L3SM, it may have been a better choice to use the SMOS-IC product instead. The authors indicate (Line 134) that they did try SMOS-IC, however found spatial coverage for this product to be low. The low spatial coverage for SMOS-IC is most likely due to significant amount of RFI over China making retrievals either impossible, or of degraded quality. The product quality flags should have been used to decide which retrievals are safe to use in the study. Did the authors use the SMOS L3SM quality flags and flags from other products in their study? Did they disqualify retrievals due to quality flags? More information on SMOS L3SM product can be found in the reference given in the comment on Line 126 below.
The authors also miss to adequately take into account the different sensitivities of active and passive sensors on the accuracy of the products when considering CCI products. For example active and passive can behave differently with respect to vegetation and roughness.
Other comments include:
Lines 14 and 54: SMOS stands for "Soil Moisture and Ocean Salinity"
Line 96: ", it is also the first study ...". Not clear what "it" refers to. Please rephrase.
Line 105: " ... as follows: ...". Perhaps use "." instead of ":" so there is continuity with the rest of the paragraph.
Lines 112-123: need to describe exactly SMAP products used for the study (version, name). The paragraph seems a bit general.
Line 126, Section 2.2: need also add the following references:
"Al Bitar Ahmad, Mialon Arnaud, Kerr Yann H., Cabot Francois, Richaume Philippe, Jacquette Elsa, Quesney Arnaud, Mahmoodi Ali, Tarot Stephane, Parrens Marie, Al-Yaari Amen, Pellarin Thierry, Rodriguez-Fernandez Nemesio, Wigneron Jean-Pierre (2017). The global SMOS Level 3 daily soil moisture and brightness temperature maps. Earth System Science Data, 9(1), 293-315."
Please see:
https://www.catds.fr/Products/Available-products-from-CPDC/Catalogue/Catds-products-from-Sextant#/metadata/9cef422f-ed3f-4090-9556-b2e895ba2ca8
Kerr, Y. H., Jacquette, E., Al Bitar, A., Cabot, F., Mialon, A., and Richaume, P.:
CATDS SMOS L3 soil moisture retrieval processor, Algorithm Theoretical Baseline Document (ATBD),CATDS, 73 pp., 2013.
Line 172: Sentence starting with "Soil moisture ..." need to be rephrased.
Line 175: Rephrase sentence starting with "When driven ..." and improve the rest of the paragraph, as the flow seems lost.
Line 205: please explain what "RFI data from SMOS ancillary" and how it is used.
Lines 216: why for SMOS, CCI, and VIC use an area-averaged method, but for in-situ use "nearest neighbor"? Do you use nearest to the center of the grid? What if there are multiple in-situ sites inside and EASE_v2 grid. The method of resampling needs to be better explained and justified.
Line 220: replace "extend" with "extended".
Line 295: "The significantly worse performance of the SMOS product ..." is also explained by the existence of algorithmic bugs.
Line 391 and beyond: Can any of the differences be explained by active versus passive sensors? The behavior over vegetation, others?
Line 404: "...similar to VIC." change to "...unlike VIC.".
Line 457: "...are exclude ..." change to "...are excluded ...".
Lines 476-477: SMAP radiometer contains onboard special hardware for detection and filtering of RFI. Hence RFI detection and mitigation is not done in the retrieval algorithm. Please correct.
Line 484: "... should be focus ..." change to " ... should focus ...".
Line 487: "Compare with ..." change to "Compared with ..."
Author Response
General:
The authors have done extensive analysis of three soil moisture data sets, namely SMAP L3, SMOS L3SM, and ESA CCI over China using triple collocation approach. The results are of interest to the scientific community and are presented well. The paper concludes that SMAP L3 and CCI products significantly perform better than the SMOS L3SM over China. This is not surprising and has been known due to the following elements:
1) presence of RFI over China which significantly contaminates SMOS observations, and
2) known algorithmic issues in the SMOS L3SM, which must be indicated in this paper. Please see the news:
https://www.catds.fr/News/Correction-of-the-L3-SM-and-VOD-from-CATDS
While the bug has been sorted out and operational data is corrected from mid July 2019 onward, the data used by the authors (2015 to 2018) is further degraded in quality due to bugs.
Given the above issues with the SMOS L3SM, it may have been a better choice to use the SMOS-IC product instead. The authors indicate (Line 134) that they did try SMOS-IC, however found spatial coverage for this product to be low. The low spatial coverage for SMOS-IC is most likely due to significant amount of RFI over China making retrievals either impossible, or of degraded quality. The product quality flags should have been used to decide which retrievals are safe to use in the study. Did the authors use the SMOS L3SM quality flags and flags from other products in their study? Did they disqualify retrievals due to quality flags? More information on SMOS L3SM product can be found in the reference given in the comment on Line 126 below.
The authors also miss to adequately take into account the different sensitivities of active and passive sensors on the accuracy of the products when considering CCI products. For example active and passive can behave differently with respect to vegetation and roughness.
Response: Thanks for your approval of this study. As you said, the reason for SMOS' poor performance could be due to RFI interference and algorithmic issues. We have added this description to the manuscript. For SMOS-IC, data with quality flag 1 (data not recommended) were filtered out. After filtering, we found that the spatial coverage was too low to use, so we decided to still use the SMOS L3SM product. For SMAP, data with quality flag not recommended are filtered out, for SMOS, DQX>0.06 are filtered out. The same strategy was adopted in the study of Chen et al. (2018) and Al-Yaari et al. (2014). We have added the description in section 2.7 to make it clear.
For CCI, the main objective of this study is to validate the most widely used product (CCI combined). More studies on the differences between CCI active and passive products will be carried out in future studies, we have added the corresponding text in the revised manuscript. Please refer to Point 12 for more details.
References:
Chen, F., Crow, W. T., Bindlish, R., Colliander, A., Burgin, M. S., Asanuma, J., & Aida, K. (2018). Global-scale evaluation of SMAP, SMOS and ASCAT soil moisture products using triple collocation. Remote Sensing of Environment, 214, 1-13.
Al-Yaari, A., Wigneron, J. P., Ducharne, A., Kerr, Y. H., Wagner, W., De Lannoy, G., ... & Mialon, A. (2014). Global-scale comparison of passive (SMOS) and active (ASCAT) satellite based microwave soil moisture retrievals with soil moisture simulations (MERRA-Land). Remote Sensing of Environment, 152, 614-626.
More issues:
1) Lines 14 and 54: SMOS stands for "Soil Moisture and Ocean Salinity"
Response: Sorry for the mistake. We have revised it correctly.
2) Line 96: ", it is also the first study ...". Not clear what "it" refers to. Please rephrase.
Response: Sorry for the ambiguity. “it” refers to this study. We have replaced “it” with “this”.
3) Line 105: " ... as follows: ...". Perhaps use "." instead of ":" so there is continuity with the rest of the paragraph.
Response: Thanks for your reminding. We have revised it.
4) Lines 112-123: need to describe exactly SMAP products used for the study (version, name). The paragraph seems a bit general.
Response: Thanks for your suggestion. We have added the corresponding description in this paragraph. “In this study, SMAP L3 passive product (version 5) is used, with 36km spatial resolution.” More detail could be found from table 1.
5) Line 126, Section 2.2: need also add the following references:
"Al Bitar Ahmad, Mialon Arnaud, Kerr Yann H., Cabot Francois, Richaume Philippe, Jacquette Elsa, Quesney Arnaud, Mahmoodi Ali, Tarot Stephane, Parrens Marie, Al-Yaari Amen, Pellarin Thierry, Rodriguez-Fernandez Nemesio, Wigneron Jean-Pierre (2017). The global SMOS Level 3 daily soil moisture and brightness temperature maps. Earth System Science Data, 9(1), 293-315."
Please see:
https://www.catds.fr/Products/Available-products-from-CPDC/Catalogue/Catds-products-from-Sextant#/metadata/9cef422f-ed3f-4090-9556-b2e895ba2ca8
Kerr, Y. H., Jacquette, E., Al Bitar, A., Cabot, F., Mialon, A., and Richaume, P.:
CATDS SMOS L3 soil moisture retrieval processor, Algorithm Theoretical Baseline Document (ATBD),CATDS, 73 pp., 2013.
Response: Thanks for your suggestion. We have added the mentioned references.
6) Line 172: Sentence starting with "Soil moisture ..." need to be rephrased.
Response: Sorry for the mistake. We realized that there is a repetition here, it has been rephrased.
7) Line 175: Rephrase sentence starting with "When driven ..." and improve the rest of the paragraph, as the flow seems lost.
Response: Thanks for your suggestion. We have rephrased the sentence and added the description of forcing data used in this study.
8) Line 205: please explain what "RFI data from SMOS ancillary" and how it is used.
Response: The RFI data came from Oliva research (Oliva et al., 2016). We matched the RFI map with each grid to study the impact of RFI on data accuracy.
Reference:
Oliva, R., Daganzo, E., Richaume, P., Kerr, Y., Cabot, F., Soldo, Y., ... & Lopes, G. (2016). Status of Radio Frequency Interference (RFI) in the 1400–1427 MHz passive band based on six years of SMOS mission. Remote sensing of environment, 180, 64-75.
9) Lines 216: why for SMOS, CCI, and VIC use an area-averaged method, but for in-situ use "nearest neighbor"? Do you use nearest to the center of the grid? What if there are multiple in-situ sites inside and EASE_v2 grid. The method of resampling needs to be better explained and justified.
Response: Thanks for your suggestion. Because for grid soil moisture products, it would be easily to resample them into the EASE_v2 grid, but for in situ soil moisture, we can only choose the station that nearest the grid center. There are some pixels (less than 20) that contain two or three stations. We chose the nearest neighbor station for these pixels. The same strategy was adopted in Chen’s et al. (2017) paper. We have added the description in the section 2.7.
Reference:
Chen, F., Crow, W. T., Colliander, A., Cosh, M. H., Jackson, T. J., Bindlish, R., ... & Goodrich, D. C. (2017). Application of triple collocation in ground-based validation of Soil Moisture Active/Passive (SMAP) level 2 data products. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(2), 489-502.
10) Line 220: replace "extend" with "extended".
Response: Sorry for the mistake. We have revised it.
11) Line 295: "The significantly worse performance of the SMOS product ..." is also explained by the existence of algorithmic bugs.
Response: Thanks for your reminding. We have added this explanation for the error source of SMOS.
12) Line 391 and beyond: Can any of the differences be explained by active versus passive sensors? The behavior over vegetation, others?
Response: Thanks for your suggestion. It is possible, as the active product may reduce its accuracy when there is more open water. After checked the proportion of active and passive products in the irrigated area, and the degree of accuracy decline of active product and combined product respectively, we believe that the main reason is caused by the rescaling with GLADS. That is why the next generation of CCI products will be committed to adopting the model-independent data rescaling method.
13) Line 404: "...similar to VIC." change to "...unlike VIC.".
Response: Thanks for your suggestion. It has been revised.
14) Line 457: "...are exclude ..." change to "...are excluded ...".
Response: Thanks for your reminding. We have revised it.
15) Lines 476-477: SMAP radiometer contains onboard special hardware for detection and filtering of RFI. Hence RFI detection and mitigation is not done in the retrieval algorithm. Please correct.
Response: Thanks for your reminding. We have revised it as follows, “The result indicates that the special hardware for detection and filtering of RFI installed in the SMAP radiometer can effectively reduce the impact of RFI”.
16) Line 484: "... should be focus ..." change to " ... should focus ...".
Response: Thanks for your reminding. We have revised it.
17) Line 487: "Compare with ..." change to "Compared with ..."
Response: Thanks for your reminding. We have revised it correctly.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The article addresses the triple collocation-based assessment of SMAP, SMOS, and ESA CCI soil moisture products with in situ observations over China. This study evaluates the accuracy and error characteristics of those satellite soil moisture products and conducts a comprehensive analysis between product accuracy and impact factors to improve the understanding of error mechanisms.
To me, the manuscript is well written and concise and appears suitable for publication after some clarifications and modifications that may be found below.
Minor comments
- In this manuscript, SMAP, SMOS, and CCI are listed in many places, but the listed order is not always consistent. It would be helpful to arrange them uniformly in text, figures, and tables.
- I can guess that TC in Line 18 is shortly written by triple collocation, but clearly note what TC is in that sentence.
- In the Data section, there are many datasets that missed their references.
Line 178: specify the description of meteorological forcing data used in this study with any reference.
Line 199: What does “land surface temperature from GEOS-5” mean? Does it mean MERRA or MERRA-2 reanalysis simulated by GEOS-5? It also specifies which dataset is used in this study with the reference. Line 200-201: need the reference for “clay content from Harmonized World Soil Database (HWSD)”. - This study assesses the SMAP, SMOS, and ESA CCI soil moisture products with the triple collocation method against in situ observations over China. The soil moisture products are evaluated by several RMSE-based metrics (e.g., fRMSE, RMSE, ubRMSE), and temporal variation is evaluated by the correlation coefficient (R). There are other metrics to assess the temporal variation no only R but also the anomaly correlation coefficient. It is the same calculation with R, but the anomalies are defined as the difference of the data from the 31-day moving average (Draper at al., 2012; Liu et al., 2011).
Liu, Q., Reichle, R. H., Bindlish, R., Cosh, M. H., Crow, W. T., de Jeu, R., ... & Jackson, T. J. (2011). The contributions of precipitation and soil moisture observations to the skill of soil moisture estimates in a land data assimilation system. Journal of Hydrometeorology, 12(5), 750-765.
Draper, C. S., Reichle, R. H., De Lannoy, G. J. M., & Liu, Q. (2012). Assimilation of passive and active microwave soil moisture retrievals. Geophysical Research Letters, 39(4).
Author Response
General:
The article addresses the triple collocation-based assessment of SMAP, SMOS, and ESA CCI soil moisture products with in situ observations over China. This study evaluates the accuracy and error characteristics of those satellite soil moisture products and conducts a comprehensive analysis between product accuracy and impact factors to improve the understanding of error mechanisms.
To me, the manuscript is well written and concise and appears suitable for publication after some clarifications and modifications that may be found below.
Response: Thank you for approving the study. We have made a comprehensive revision of the original manuscript to make it more readable and scientific.
More issues:
1) In this manuscript, SMAP, SMOS, and CCI are listed in many places, but the listed order is not always consistent. It would be helpful to arrange them uniformly in text, figures, and tables.
Response: Thanks for your suggestion. We have rearranged the order of SMAP, CCI, and SMOS to be make it consistent, including abstract, introduction, method, results, Table 1, and Figure 3.
2) I can guess that TC in Line 18 is shortly written by triple collocation, but clearly note what TC is in that sentence.
Response: Thanks for your reminding. We have added the abbreviation of triple collocation (TC) in the sentence before.
3) In the Data section, there are many datasets that missed their references.
Line 178: specify the description of meteorological forcing data used in this study with any reference.
Response: We are very sorry for missing the description of meteorological forcing data. The forcing data is based on the daily precipitation and temperature data of 756 meteorological stations. We have added the description and references of forcing data in the manuscript.
Line 199: What does “land surface temperature from GEOS-5” mean? Does it mean MERRA or MERRA-2 reanalysis simulated by GEOS-5? It also specifies which dataset is used in this study with the reference.
Response: It means Modern-Era Retrospective Analysis for Research and Applications (MERRA) reanalysis simulated by GEOS-5. We have added the description of the land surface temperature data in the manuscript.
Line 200-201: need the reference for “clay content from Harmonized World Soil Database (HWSD)”.
Response: Thanks for your reminding. We have added the references of the HWSD in the manuscript.
4) This study assesses the SMAP, SMOS, and ESA CCI soil moisture products with the triple collocation method against in situ observations over China. The soil moisture products are evaluated by several RMSE-based metrics (e.g., fRMSE, RMSE, ubRMSE), and temporal variation is evaluated by the correlation coefficient (R). There are other metrics to assess the temporal variation no only R but also the anomaly correlation coefficient. It is the same calculation with R, but the anomalies are defined as the difference of the data from the 31-day moving average (Draper at al., 2012; Liu et al., 2011).
Liu, Q., Reichle, R. H., Bindlish, R., Cosh, M. H., Crow, W. T., de Jeu, R., ... & Jackson, T. J. (2011). The contributions of precipitation and soil moisture observations to the skill of soil moisture estimates in a land data assimilation system. Journal of Hydrometeorology, 12(5), 750-765.
Draper, C. S., Reichle, R. H., De Lannoy, G. J. M., & Liu, Q. (2012). Assimilation of passive and active microwave soil moisture retrievals. Geophysical Research Letters, 39(4).
Response: Thanks for your suggestion. Considering the anomaly correlation coefficient using the moving average can effectively remove the impact of seasonal cycle, which is a good metrics. We have added these two references and the corresponding text in section 3.2. We will consider using this metrics in future studies.
Author Response File: Author Response.docx
Reviewer 3 Report
Line 45: Please qualify “most promising” (e.g. most promising for large-scale monitoring). In situ probes are likely to be more accurate but are not spatially representative of large areas and also sparse due to expense. Also, LSMs provide spatially and temporally continuous estimates of soil moisture, which satellites do not.
Line 59: I do not think SMAP is included in CCI? If it is not included in CCI, please state: “the ESA project consortium merged a selection of these satellite soil moisture products…” and reference Section 2.3 for more details.
Line 63: Please remove the word “high” because this is a broad term that is not defined here.
Line 80: Grammar change: “after the data is collocated”.
Line 93: Please include reference to Lawston et al. (2017) (Irrigation Signals Detected from SMAP Soil Moisture Retrievals)
Line 96: Please include the context of Kumar et al. (2015) (Evaluating the utility of satellite soil moisture retrievals over irrigated areas and the ability of land data assimilation methods to correct for unmodeled processes), which highlights the issues of rescaling satellite soil moisture retrievals to model climatology over irrigated areas.
Sentence starting on line 106 has grammatical issues. Please fix this. “Section 4 presents the TC-based validation results as well as irrigation and five land surface characteristics impact on product accuracy and representativeness error of in situ sites.” For example: Section 4 presents the overall TC-based validation results, and the results in the context of irrigation, land surface characteristics and spatial representativeness.
Section 2.1: Please provide SMAP’s spatial resolution and specify which SMAP product was used in this study.
Line 158: “21st, and 26th"
Line 161: 220 observations in over 3 years is a lower temporal resolution than SMAP. Is this correct?
Line 165: please cite previous studies you are referring to.
Section 2.1 – 2.3: Is a quality flag applied to screen remotely sensed retrievals with low quality data? If not, are the results robust to this?
Section 2.4: Please state if in situ stations are located on irrigated land.
Section 2.4: I assume that there are multiple in situ stations in some remotely sensed pixels. Do you choose the nearest neighbor station? Please clarify this point.
Sentence starting in line 172 and ending in line 174 states that VIC is the most widely used SM model in China 2 times. Please remove this repetitiveness.
Fig 2. It looks like half of the red frame is lightly irrigated. Please make the red frame contain only the region of interest.
Line 198: Please include citation of Das et al. (2019) (The SMAP and Copernicus Sentinel 1A/B microwave active-passive high resolution surface soil moisture product)
Line 200: Please specify which MODIS product is use for vegetation water content. Is there any post-processing of the data, e.g. converting NDVI to vegetation water content, or is there a specific MODIS vegetation water content product being used
Line 225: Can you please provide a sentence or two that clarifies either mechanistically or through example how TC minimizes the scale issue?
Line 277: in situ should be two words to be consistent with the text throughout the manuscript.
Line 342: should read “while a single product may contain large errors in certain landcovers”
Line 365: grammar change: “with R values between…”
Line 369: misplaced “.” after “SMAP”
Line 369: grammar change: “the R values of CCI are usually…”
Line 381: are these decreases statistically significant? Please clarify what is considered “significant” here.
Line 383-384: Are in situ stations on irrigated land? Please state if in situ observations are expected to maintain irrigation signal.
Section 4.2: The TC method estimates that the “true” soil moisture is a compromise between the 3-independent products. If only 1 product, e.g. modeled soil moisture, does not maintain irrigation signal, I would expect the truth estimate to be further from the 2 products that maintain irrigation signal, and therefore decrease the correlation of products that maintain irrigation signal. Please reconcile your discussion in the paragraph (line 375-390) with this.
Line 439-440: Rescaling via CDF matching is typically used in data assimilation systems that do not allow for systematic biases between the observations and the model. In this application, model-dependent rescaling is appropriate. I do not know of an applicable benefit in rescaling CCI with SMAP, unless CCI is known to have systematic biases relative to truth soil moisture that SMAP does not. Rather, it may make more sense to use the native CCI product, without rescaling, over irrigated areas.
Fig 9. SMAP reports a retrieval quality flag when vegetation water content exceeds 5 kg/m2 (O’Neill et al., 2019; Table 1; Soil Moisture Active Passive (SMAP) Algorithm Theoretical Basis Document Level 2 & 3 Soil Moisture (Passive) Data Products; https://nsidc.org/sites/nsidc.org/files/technical-references/L2_SM_P_ATBD_rev_E_Aug2019.pdf). Please discuss results in context of these quality flags. Also, please show that the results in this manuscript are robust to quality flag screening or state otherwise.
Paragraph 473-480: Please compare the differences in R across RFI for VIC as a “control” dataset that should be insensitive to RFI. This will support that these differences are “caused” by RFI (line 475).
Line 487: grammar correction: “Compared with…”
Line 498: typo: “used in this study or, 2)…”
Line 577: typo: “in situ sites…”
Line 606: in situ is hyphenated, which is inconsistent with the above text.
Author Response
1) Line 45: Please qualify “most promising” (e.g. most promising for large-scale monitoring). In situ probes are likely to be more accurate but are not spatially representative of large areas and also sparse due to expense. Also, LSMs provide spatially and temporally continuous estimates of soil moisture, which satellites do not.
Response: Thanks for your suggestion. Indeed, it is not appropriate to say that remote sensing products is the most promising. The in situ measurements are more accurate, and the model products can provide temporally continuous data. We have revised the sentences as follows: remote sensing has been recognized as the most promising way for large-scale soil moisture monitoring (especially in ungauged regions).
2) Line 59: I do not think SMAP is included in CCI? If it is not included in CCI, please state: “the ESA project consortium merged a selection of these satellite soil moisture products…” and reference Section 2.3 for more details.
Response: Thanks for your reminding. It’s true that SMAP is not included in CCI yet. We have revised it as follows: the ESA project consortium merged a selection of these individual satellite soil moisture products together into one product namely Climate Change Initiative (CCI) soil moisture product, reference section 2.3 for more details.
3) Line 63: Please remove the word “high” because this is a broad term that is not defined here.
Response: Thanks for your suggestion, it has been removed.
4) Line 80: Grammar change: “after the data is collocated”.
Response: Thanks for your reminding. We have revised it as you suggested.
5) Line 93: Please include reference to Lawston et al. (2017) (Irrigation Signals Detected from SMAP Soil Moisture Retrievals)
Response: Thanks for your suggestion. We have added the references you mentioned.
6) Line 96: Please include the context of Kumar et al. (2015) (Evaluating the utility of satellite soil moisture retrievals over irrigated areas and the ability of land data assimilation methods to correct for unmodeled processes), which highlights the issues of rescaling satellite soil moisture retrievals to model climatology over irrigated areas.
Response: Thanks for your suggestion. It is a very good references. As mentioned in the article: “we demonstrate that rescaling to a model climatology that is not representative of the observations may distort the scale of the actual observational features and may lead to loss of valuable signals”, which is consistent with the conclusions in this study (rescaling CCI products through hydrological models lacking irrigation module can lead to signals loss over irrigated areas). We have added this reference and the corresponding context to the introduction and results.
7) Sentence starting on line 106 has grammatical issues. Please fix this. “Section 4 presents the TC-based validation results as well as irrigation and five land surface characteristics impact on product accuracy and representativeness error of in situ sites.” For example: Section 4 presents the overall TC-based validation results, and the results in the context of irrigation, land surface characteristics and spatial representativeness.
Response: Thanks for your suggestion. We have deleted the original sentence and revised it as you suggested.
8) Section 2.1: Please provide SMAP’s spatial resolution and specify which SMAP product was used in this study.
Response: Thanks for your suggestion. We have added the description of SMAP as follows: In this study, SMAP L3 passive product is used, with 36km spatial resolution.
9) Line 158: “21st, and 26th"
Response: Thanks for your reminding. It has been revised.
10) Line 161: 220 observations in over 3 years is a lower temporal resolution than SMAP. Is this correct?
Response: It is correct. In situ soil moisture used in the study were measured every 5 days.
11) Line 165: please cite previous studies you are referring to.
Response: Thanks for your reminding. We have added the corresponding references.
12) Section 2.1 – 2.3: Is a quality flag applied to screen remotely sensed retrievals with low quality data? If not, are the results robust to this?
Response: For SMAP, data with quality flag not recommended are filtered out, for SMOS, DQX>0.06 are filtered out. The same strategy was adopted in the study of Chen et al. (2018) and Al-Yaari et al. (2014). We have added the description in section 2.7 to make it clear.
References:
Chen, F., Crow, W. T., Bindlish, R., Colliander, A., Burgin, M. S., Asanuma, J., & Aida, K. (2018). Global-scale evaluation of SMAP, SMOS and ASCAT soil moisture products using triple collocation. Remote Sensing of Environment, 214, 1-13.
Al-Yaari, A., Wigneron, J. P., Ducharne, A., Kerr, Y. H., Wagner, W., De Lannoy, G., ... & Mialon, A. (2014). Global-scale comparison of passive (SMOS) and active (ASCAT) satellite based microwave soil moisture retrievals with soil moisture simulations (MERRA-Land). Remote Sensing of Environment, 152, 614-626.
13) Section 2.4: Please state if in situ stations are located on irrigated land.
Response: For those areas where crops are grown (e.g., North China Plain), the stations are located on irrigated land to accurately obtain the soil moisture of the field. Thus, these observations can maintain the irrigation signal.
14) Section 2.4: I assume that there are multiple in situ stations in some remotely sensed pixels. Do you choose the nearest neighbor station? Please clarify this point.
Response: Thanks for your suggestion. Yes, there are some pixels (less than 20) that contain two or three stations. We chose the nearest neighbor station for these pixels. We have added the description in the section 2.7.
15) Sentence starting in line 172 and ending in line 174 states that VIC is the most widely used SM model in China 2 times. Please remove this repetitiveness.
Response: Sorry for the mistake, it has been removed.
16) Fig 2. It looks like half of the red frame is lightly irrigated. Please make the red frame contain only the region of interest.
Response: Thanks for your suggestion. We have revised the Fig 2. to make the red frame contain only the region of interest.
17) Line 198: Please include citation of Das et al. (2019) (The SMAP and Copernicus Sentinel 1A/B microwave active-passive high resolution surface soil moisture product)
Response: Thanks for your suggestion. The citation has been added.
18) Line 200: Please specify which MODIS product is use for vegetation water content. Is there any post-processing of the data, e.g. converting NDVI to vegetation water content, or is there a specific MODIS vegetation water content product being used
Response: Thanks for your suggestion. The VWC used here was constructed based on MODIS NDVI and IGBP landcover data. We have added instructions in section 2.6.
19) Line 225: Can you please provide a sentence or two that clarifies either mechanistically or through example how TC minimizes the scale issue?
Response: There will be scale issue when compare point-scale in situ measurements with grid-scale satellite data. Crow et al. (2012) found statistically significant stable linear relationships between point and grid-scale soil moisture dynamics. And error characteristics from such the linear relationship can be quantified by TC (Chen et al., 2017). We have added more description about how TC works in the method section.
References:
Crow, W. T., Berg, A. A., Cosh, M. H., Loew, A., Mohanty, B. P., Panciera, R., ... & Walker, J. P. (2012). Upscaling sparse ground‐based soil moisture observations for the validation of coarse‐resolution satellite soil moisture products. Reviews of Geophysics, 50(2).
Chen, F., Crow, W. T., Colliander, A., Cosh, M. H., Jackson, T. J., Bindlish, R., ... & Goodrich, D. C. (2017). Application of triple collocation in ground-based validation of Soil Moisture Active/Passive (SMAP) level 2 data products. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(2), 489-502.
20) Line 277: in situ should be two words to be consistent with the text throughout the manuscript.
Response: Thanks for your reminding, it has been corrected revised to be consistent.
21) Line 342: should read “while a single product may contain large errors in certain landcovers”
Response: Thanks for your reminding, we have revised the sentences as you suggested.
22) Line 365: grammar change: “with R values between…”
Response: Sorry for the mistake, it has been revised.
23) Line 369: misplaced “.” after “SMAP”
Response: Sorry for the mistake, it has been revised.
24) Line 369: grammar change: “the R values of CCI are usually…”
Response: Sorry for the mistake, it has been revised.
25) Line 381: are these decreases statistically significant? Please clarify what is considered “significant” here.
Response: Sorry for the controversial word here. It’s more visual rather than statistical. We have revised it according to the accurate number. “the R values of VIC model in Henan, Shandong, Hebei provinces decrease by 0.2-0.4…”
26) Line 383-384: Are in situ stations on irrigated land? Please state if in situ observations are expected to maintain irrigation signal.
Response: Yes, for those areas where crops are grown (e.g., North China Plain), the stations are located on irrigated land to accurately obtain the soil moisture of the field. Thus, these observations can maintain the irrigation signal. We have added the corresponding description in section 2.4.
27) Section 4.2: The TC method estimates that the “true” soil moisture is a compromise between the 3-independent products. If only 1 product, e.g. modeled soil moisture, does not maintain irrigation signal, I would expect the truth estimate to be further from the 2 products that maintain irrigation signal, and therefore decrease the correlation of products that maintain irrigation signal. Please reconcile your discussion in the paragraph (line 375-390) with this.
Response: Thanks for your reminding. Indeed, in irrigated areas, if we use a triplet of in situ, SMAP and CCI active product in TC, the correlation coefficient of SMAP will increase (which is caused by the low accuracy of VIC model in these areas). This indicates that the actual correlation coefficient of SMAP in irrigated areas may be higher than the results in this study. We have added the relevant discussion in this paragraph.
28) Line 439-440: Rescaling via CDF matching is typically used in data assimilation systems that do not allow for systematic biases between the observations and the model. In this application, model-dependent rescaling is appropriate. I do not know of an applicable benefit in rescaling CCI with SMAP, unless CCI is known to have systematic biases relative to truth soil moisture that SMAP does not. Rather, it may make more sense to use the native CCI product, without rescaling, over irrigated areas.
Response: Thanks for your suggestion. In fact, CCI offers three soil moisture products, the active product rescaled by ASCAT, the passive product rescaled by AMSR-E, and the combined product rescaled by GLDAS (which is merged by active and passive product) (Gruber et al., 2019). We found that the combined product suffers a decrease in accuracy over irrigated areas, the reason comes from the use of reference dataset (GLDAS). Therefore, using SMAP or models with irrigation modules as reference dataset may improve the data accuracy over irrigated areas.
References:
Gruber, A., Scanlon, T., Schalie, R. V. D., Wagner, W., & Dorigo, W. (2019). Evolution of the ESA CCI Soil Moisture climate data records and their underlying merging methodology. Earth System Science Data, 11(2), 717-739.
29) Fig 9. SMAP reports a retrieval quality flag when vegetation water content exceeds 5 kg/m2 (O’Neill et al., 2019; Table 1; Soil Moisture Active Passive (SMAP) Algorithm Theoretical Basis Document Level 2 & 3 Soil Moisture (Passive) Data Products; https://nsidc.org/sites/nsidc.org/files/technical-references/L2_SM_P_ATBD_rev_E_Aug2019.pdf). Please discuss results in context of these quality flags. Also, please show that the results in this manuscript are robust to quality flag screening or state otherwise.
Response: Thanks for your suggestion. We have added the corresponding analysis in the results. “In addition, when the grid VWC is less than 5kg/m2, the ubRMSE of SMAP is 0.04 m3m-3, which meets the accuracy requirement of SMAP algorithm O’Neill et al. (2019)”.
Please refer to point 12 for quality flag screening.
30) Paragraph 473-480: Please compare the differences in R across RFI for VIC as a “control” dataset that should be insensitive to RFI. This will support that these differences are “caused” by RFI (line 475).
Response: Thanks for your suggestion. We have added VIC as a control dataset to investigate the impact of RFI in Fig 9d. The results indicate that VIC is actually insensitive to RFI.
31) Line 487: grammar correction: “Compared with…”
Response: Thanks for your reminding, it has been revised correctly.
32) Line 498: typo: “used in this study or, 2)…”
Response: Thanks for your reminding, it has been revised correctly.
33) Line 577: typo: “in situ sites…”
Response: Sorry for the mistake, it has been revised correctly.
34) Line 606: in situ is hyphenated, which is inconsistent with the above text.
Response: Thanks for your reminding, it has been revised.
Author Response File: Author Response.docx
Round 2
Reviewer 3 Report
Thank you for the thoughtful replies and edits. The large majority of the comments I made to version 1 of the manuscript are fully addressed. I have one remaining question/comment below that I would like to have addressed prior to publishing.
Regarding my comment (29) on version 1 about vegetation water content: SMAP reports a retrieval quality flag when vegetation water content exceeds 5 kg/m2 (O'Neill et al., 2019), yet in Fig. 9, SMAP data is clearly considered for vegetation water contents that exceed 5 kg/m2. This makes me believe that SMAP's retrieval quality flag has not been used to filter data, despite the note added to section 2.7. That said, it is up to the discretion of the user to use "flagged" data with caution because soil moisture retrievals are not validated for such regions. Can you please reconcile how SMAP data is screened based do the quality flag while Fig. 9 shows that the study considers data that should have been flagged for low quality?
Author Response
1) Thank you for the thoughtful replies and edits. The large majority of the comments I made to version 1 of the manuscript are fully addressed. I have one remaining question/comment below that I would like to have addressed prior to publishing.
Regarding my comment (29) on version 1 about vegetation water content: SMAP reports a retrieval quality flag when vegetation water content exceeds 5 kg/m2 (O'Neill et al., 2019), yet in Fig. 9, SMAP data is clearly considered for vegetation water contents that exceed 5 kg/m2. This makes me believe that SMAP's retrieval quality flag has not been used to filter data, despite the note added to section 2.7. That said, it is up to the discretion of the user to use "flagged" data with caution because soil moisture retrievals are not validated for such regions. Can you please reconcile how SMAP data is screened based do the quality flag while Fig. 9 shows that the study considers data that should have been flagged for low quality?
Response: Thanks for your update. We did use SMAP quality flag, However, we had an error in calculating the average VWC for each station. In the original manuscript, we calculated average VWC based on VWC data throughout all the validation period (that is why the VWC value exceeds 5 kg/m2 for some stations), which is not correct. In the revised manuscript, for each station, we calculated average VWC only when the SMAP quality flag is recommended in corresponding date. We have recalculated and drew Figure 9b in the revised manuscript, please check it in attachment.
Author Response File: Author Response.docx
