On the Performances of Trend and Change-Point Detection Methods for Remote Sensing Data
Round 1
Reviewer 1 Report
Dear Authors,
I took a very careful look in your MS and I think your work could be published after medium revision. My major concern in this MS is related to 1. the absence of a "Conclusions" section and 2. "the Discussion" section where you should have discussed your findings and compare the results of your research with similar published works. In addition to the above mentioned comment:
- Caption of Figure 4: please explain what the circle means.
- Captions of Figures 5, 6, 7, 8, 9, 10, 11, 12, 13. Please, add 1-2 sentences to explain what the reader must keep in mind from each figure. Each figure must be stand-alone, I mean when the reader looks at the figure and read the figure caption to understand the essence of the figure.
- Line 265: do you mean change-points?
- Lines 410-413: Can you provide an explanation for the effect you describe in these lines? It would be helpful for the reader.
- In the text (lines 397-398) you refer "Generally, the changes of LST over months is visible, with lower temperature and higher variation for LST night." It would be helpful to add more details in this intepretation taking into account Figs. 9, 10. For example, you could analyze your findings using cold and warm time periods.
Author Response
Dear reviewer
We attach you a pdf document with the reply to your comments, and the new version of the paper.
We have also revised English expressions
Thanks for your suggestions and comments, that have contributed to improve the paper very much.
Note: We thank the reviewer for the constructive comments. The corresponding changes to the
comments in the revised manuscript are highlighted in red.
1.I took a very careful look in your MS and I think your work could be published after medium
revision. My major concern in this MS is related to 1. the absence of a “Conclusions” section and
2. “the Discussion” section where you should have discussed your findings and compare the results
of your research with similar published works.
Authors’ response:
Thanks for your feedback. We have now renamed Section “Discussion” to “Conclusions
and Discussion”. In the first paragraph, we summarise the structure of the paper with the
focus on our simulation study. This is then followed by the conclusions of our simulation
study, discussing the performance of the modified versions of Mann-Kendall, discussing
the performance of the methods that demand identifying the minimum number of data
points between any two consecutive change-points in advance, and the difference between
univariate and multivariate methods. Future works are also discussed.
Regarding the effect of temporal autocorrelation on the type I error probability of changepoint/trend detection methods our results are consistent with the literature, and we have
now added a sentence emphasising this alongside with referring to proper references.
2. Caption of Figure 4: please explain what the circle means.
Authors’ response:
Thanks for pointing this out. That circle meant an artificial cloud. In other words,
the pixels falling in that circle are those where we produce abrupt changes. To avoid
confusion, we have replaced that circle with a set of pixels displayed in white colour. We
have now clarified this in both the text and the caption.
The caption reads as “Simulated raster from the first scenario, where pixel time series
are generated from the standard normal distributions, with an artificial randomly located
cloud containing 13 pixels (displayed in white) in which we produce abrupt changes”.
3. Captions of Figures 5, 6, 7, 8, 9, 10, 11, 12, 13. Please, add 1-2 sentences to explain what the
reader must keep in mind from each figure. Each figure must be stand-alone, I mean when the
reader looks at the figure and read the figure caption to understand the essence of the figure.
Authors’ response:
Thanks for your suggestion. We have now rephrased and/or added extra information to
the captions. In the captions of Figures 5,6,7 and 8, we have included the information
regarding the distribution of data, the magnitude of change, the starting time-periods of
artificial change-points, and the criteria measured. What the reader must keep in mind
is in fact the behaviour of the power of the test and MAE with respect to the magnitudes
of change and starting time-periods of change after taking into account the distribution
of data. We hope enough information is now provided.
In the new version of the manuscript, Figure 9 shows the map of Navarre. Figures 10 and
11 show the LST data of Navarre during 2001, in which we have updated their captions
highlighting the variation of LST in the region and changes of LST over months.
1
In Figure 12 we have added that the first lag autocorrelation is obtained from pixel time
series individually. In Figure 13 we have added a sentence regarding the meaning of
positive/negative values of Kendall’s τ . The caption of Figure 14 has also been updated.
4. Line 265: do you mean change-points?
Authors’ response:
Thanks. This was a typo, and it is now revised.
5. Lines 410-413: Can you provide an explanation for the effect you describe in these lines? It would
be helpful for the reader.
Authors’ response:
Thanks. A clarifying comment is added. It reads as “This variation might be due to the
variety of land cover, vegetation, altitude, moisture, precipitation and so forth. Generally, taking Figures 10 and 11 into account, for LST day the regions with low temperature
(north) have weaker autocorrelation, while for LST night the regions with lower temperature (north-east) have stronger autocorrelation.”
6. In the text (lines 397-398) you refer “Generally, the changes of LST over months is visible, with
lower temperature and higher variation for LST night.” It would be helpful to add more details in
this interpretation taking into account Figs. 9, 10. For example, you could analyze your findings
using cold and warm time periods.
Authors’ response:
Thanks for pointing this out. That sentence is now rephrased, and extra information is
provided. Furthermore, we have added Table 4 providing the descriptive statistics of LST
day and night over seasons. In addition, we have explained our finding concerning the
relationship between the cold and warm regions with their estimated autocorrelation of
LST day and night (see our answer to your 5th comment).
Reviewer 2 Report
Manuscript entitled “On the performance of trend change change-point detection methods for remote sensing data” by Militino et al. submitted to Remote Sensing. The authors compared different statistical tests, which have been applied in some other papers, for trend/change-points. They also applied tests for the case study of Land Surface Temperature (LST) from monthly MODIS images in Navarre, Spain.
In general, the manuscript is well written and has a good structure. The study does not contribute much in term of novelty in methodology, but it may be of interest of readers. This is because the authors reviewed different trend tests in remote sensing and gives recommendation for future research. I have no doubt with the results of the study, but only have several minor comments which hopefully would be helpful for the authors.
Abstract: the abstract needs a kind of concluding remarks or future implications.
Introduction:
Figures are only useful when they go with texts which explain the figures. I suggest the authors should either add a short description or move Figure 1 to another place.
Line 61: LST data from which product? And source of data?
Line 34: “data” is a plural term
Line 43: this paragraph should have another topic sentence. Additionally, not only R but also Matlab/python provides packages to implement trend analysis. Otherwise, they could be reproduced easily in other programming languages. Why the authors focus on R (likely because later they used R packages?)?
Trend and change-point detection (methodology): this part can be shortened because most of the tests are common in data analytics.
I am not so sure what is the y-axis of Figures 3 and 14?Which type of data is analysed in Figure 3?
Figures 5 to 8 should be redesigned.
Figures 9 and 10 only shows raw data, so they can be removed or added as supplemental materials. A figure shows the location of the study area should be added.
The authors should add a conclusion to summary the novelty of this study and provide future implication.
Author Response
Dear reviewer
We attach you a pdf document with the reply to your comments, and the new version of the paper.
We have also revised English expressions.
Thanks for your suggestions and comments, that have contributed to improve the paper very much.
Note: We thank the reviewer for the constructive comments. The corresponding changes to the
comments in the revised manuscript are highlighted in blue.
1.Abstract: the abstract needs a kind of concluding remarks or future implications.
Authors’ response:
Thanks for your feedback. Our major conclusion concerns the effect of tails and temporal
autocorrelation on the power of the test, type I error probability, and Mean Absolute Error
(MAE). Such effects are mentioned in the abstract. Further, among all methods we have
considered in our simulation study, we have concluded that the original Mann-Kendall is
generally the preferable choice. We have now rephrased our conclusion in the abstract to
make this clearer. The future implications have been added to the Discussion.
In the abstract we have added the following text. “In this paper, we draw a simulation
study based on including different artificial abrupt changes at different time-periods of
image time series to assess the performance of such methods. The power of the test, type
I error probability, and Mean Absolute Error (MAE) are used as performance criteria
in which MAE is only calculated for change-point detection methods. The study reveals
that if the magnitude of change (or trend slope) is high, and/or the change does not
occur in the first or last time-periods, the methods generally have a high power and a low
Mean Absolute Error (MAE). However, in the presence of temporal autocorrelation, MAE
raises, and the probability of introducing false positives increases noticeably. The modified
versions of the Mann-Kendall method for autocorrelated data reduce/moderate its type
I error probability, but this reduction comes with an important power diminution. In
conclusion, taking a trade-off between the power of the test and type I error probability, we
conclude that the original Mann-Kendall test is generally the preferable choice. Although
Mann-Kendall is not able to identify the time-period of abrupt changes, it is more reliable
than other methods when detecting the existence of such changes.”
2. Figures are only useful when they go with texts which explain the figures. I suggest the authors
should either add a short description or move Figure 1 to another place.
Authors’ response:
Thanks for pointing this out. In our simulation study, we have considered a set of joint
pixels (called cloud in the manuscript) in which we produce abrupt changes. In Figure
1 we had tried to show this cloud with a circle. Nevertheless, to avoid confusion we
have replaced that circle with the pixels displayed in white. We have produced abrupt
changes in the pixels displayed in white in Figure 1. Furthermore, we have added a short
clarifying description in both the text and caption of Figure 1. The caption reads as
“Simulated raster from the first scenario, where pixel time series are generated from the
standard normal distributions, with an artificial randomly located cloud containing 13
pixels (displayed in white) in which we produce abrupt changes”.
3. Line 61: LST data from which product? And source of data?
Authors’ response:
We have now clarified this in the introduction, and the corresponding text reads as “The
1
methods with better performance are also used to study the monthly remote sensing LST
data of Navarre, Spain, from version-6 MOD11A2 product of Moderate Resolution Imaging Spectroradiometer (MODIS) [36], in the period of January 2001 to December 2018”.
However, the complete details regarding the product and source of data is provided at
the beginning of Section 4. This reads as “The original images are eight-days composite
images coming from version-6 MOD11A2 product of MODIS. The images are acquired
through Terra satellite at 10:30 and 22:30 local time. This product is derived from the two
Thermal Infrared (TIR) channels: 31 (10.78 to 11.28 µm) and 32 (11.77 to 12.27 µm)
[59], where the split-window algorithm [60] is used for correcting the atmospheric effects.
Images have been downloaded, customized, and monthly averaged using the R package
RGISTools [51,61]. The images are UTM projected, loaded into R as multi-layer rasters,
and LST values are in Kelvin degrees. The original MOD11A2 images are factor scaled
by 0.02, meaning that we obtain the Kelvin degrees when multiplying by this factor”.
4. Line 34: “data” is a plural term
Authors’ response:
Thanks. It is now read as “data are generated”.
5. Line 43: this paragraph should have another topic sentence. Additionally, not only R but also
Matlab/python provides packages to implement trend analysis. Otherwise, they could be reproduced easily in other programming languages. Why the authors focus on R (likely because later
they used R packages?)?
Authors’ response:
We agree with the reviewer that there are other software to use for the problem of trend
and change-point detection. However, the methods considered in the manuscript are all
implemented in several R packages. To avoid confusion, we have moved that sentence to
the beginning of Section 3.1 “Computation details”, and also rephrased the corresponding
text in the Introduction.
6. Trend and change-point detection (methodology): this part can be shortened because most of the
tests are common in data analytics.
Authors’ response:
Thanks for this suggestion. We do agree with the reviewer in the sense that these methods
are common in such data analysis. However, we believe a minimum background of each
method can facilitate reading the manuscript avoiding to switch between papers looking
for further details of each method. We have tried to keep the details of each method short
but still informative.
7. I am not so sure what is the y-axis of Figures 3 and 14? Which type of data is analysed in Figure 3?
Authors’ response:
Thanks. Labels are added to the y-axis of both Figures.
The y-axis of Figure 3 gives the values of the simulated data. Figure 14 that you mentioned is now Figure 15 in the revised version, in which its y-axis provides the deseasoned
and aggregated LST night values.
8. Figures 5 to 8 should be redesigned.
Authors’ response:
It is not clear to us what kind of design the reviewer means. In Figures 5 to 8, we
need to demonstrate the power of the test and MAE with respect to different magnitudes
0.5, 1, 1.5 and starting time-periods 40, 60, . . . , 120. Different methods are displayed in
2
different colours/symbols, and the changes in both the power of the test and MAE are
visible with respect to the starting time-periods of change as well as the magnitude of
change. For instance, when the change is produced in the middle time-period 80, the
methods show a higher power and lower MAE. On the contrary, when the abrupt change
is produced in the first or last time-periods, the methods show a lower power of the test
and a higher MAE. Nevertheless, we have tried to improve our explanation regarding the
details of our simulation study, and the captions of these Figures.
We highlight that these Figures are obtained using the R package ggplot2, and we believe
they properly show the power of the test and MAE of all methods with respect to both
the magnitude and starting time-period of change in one frame. However, it is greatly
appreciated if the reviewer informs us about a better design.
9. Figures 9 and 10 only shows raw data, so they can be removed or added as supplemental materials.
A figure shows the location of the study area should be added.
Authors’ response:
In the revised manuscript, we have added a Figure showing the location of the study area
(Figure 9 in the new version). Regarding the raw data, we have presented them to give an
insight into the difference between the LST day and LST night together with the variation
of LST across the region and over time. This is now highlighted in the text and captions.
Furthermore, these raw data are in close relation with Figure 12,13 and 14 according
to which we have added clarifying comment before Figure 12, explaining the relation. It
reads as “This variation might be due to the variety of land cover, vegetation, altitude,
moisture, precipitation and so forth. Generally, taking Figures 10 and 11 into account, for
LST day the regions with low temperature (north) have weaker autocorrelation, while for
LST night the regions with lower temperature (north-east) have stronger autocorrelation.”
10. The authors should add a conclusion to summary the novelty of this study and provide future
implication.
Authors’ response:
Thanks for your feedback. We have renamed the Section “Discussion” to “Conclusions
and Discussion”.
The first paragraph of this section summarises the novelty of the manuscript. This concerns the comparison on the performance of several methods looking for change-points
and trends in time series of remote sensing data by considering different distributions for
the time series, placing change-points at different time periods , and providing different
magnitudes of change.
Moreover, we have highlighted how the temporal autocorrelation affects the type I error
probability and Mean Absolute Error (MAE). Furthermore, there is a reduction in the
performance of methods when a change has happened in the first or last time periods.
Concerning the modifications proposed to moderate/reduce the type I error probability of
Mann-Kendall, we have found that these modifications generally reduce the type I error
probability of Mann-Kendall, but this reduction comes with an important power diminution. We have further summarised the effect of identifying the minimum number of data
points between any two consecutive change-points in advance over the performance of
BFAST, Strucchange, Meanvar and E.divisive methods. The differences in the performance of univariate and multivariate methods are also discussed. We hope that Section
“Conclusions and Discussion” is now clearer.
In addition, we have now added a paragraph regarding future implication which reads as
“Regarding future works, since trend detection methods generally show a lower type I error probability than the methods inherently designed for change-point detection, it would
be relevant and interesting to modify them adding the possibility of finding the time where
the change occurs. Another relevant idea might be to draw a simulation study based on
another type of abrupt changes such as sudden changes in the trend direction”.
Reviewer 3 Report
This paper by Militino, Moradi, and Ugarte entitled "On the performance of trend and change-point detection methods for remote sensing data" submitted to MDPI's 'Remote Sensing' is an excellent paper and is a joy to read. It systematically describes and compares different algorithms for trend and change-point detection, and applies these algorithms to both simulated data and to surface-temperature data for the the Navarre region of Spain. The results are very impressive, and point to multivariate methods for E.divisive and Mann-Kendall as being superior, and the results also discuss the best univariate methods (that can also detect the location of the change).
I ask for two minor improvements in the revision:
1) clarification for what time or time-period during both the day and the night for which the surface-temperature data is acquired by MODIS. Was it the same time each day or night?
2) a more-thorough description of the detrending algorithm used for the monthly-averaged MODIS data. Does the choice of detrending technique affect the subsequent analysis?
I also ask for one major improvement in the revision:
1) a study and discussion comparing the dominant land-cover type and topography to the detected pixels where the trend or change-points were detected in the Navarre MODIS data. Were these urban regions? Mountainous regions? Water-covered regions? Agricultural regions? Desert/barren regions? etc.
Again, I think this is an excellent paper, and I only am asking for a major revision (as opposed to a minor revision) so that I have some certainty of being able to review the changes prior to publication.
Author Response
Dear reviewer
We attach you a pdf document with the reply to your comments, and the new version of the paper.
We have also revised English expressions.
Thanks for your suggestions and comments, that have contributed to improving the paper very much.
Note: We thank the reviewer for the constructive comments. The corresponding changes to the comments in the revised manuscript are highlighted in brown.
1.Clarification for what time or time-period during both the day and the night for which the surface temperature data is acquired by MODIS. Was it the same time each day or night?
Authors’ response:
Thanks for pointing this out. The LST images are acquired through Terra satellite at 10:30 and 22:30 local time. We have now explained this in the text.
2. A more thorough description of the detrending algorithm used for the monthly-averaged MODIS data. Does the choice of detrending technique affect the subsequent analysis?
Authors’ response:
Thanks for your feedback. We have now added a comment explaining this in the text. The corresponding text reads as “Therefore, we first deseason the data using the deseason function of the R package remote[63] that creates seasonal anomalies of data. In other words, each deseasoned image is obtained by subtracting the average image of its corresponding month over all the time period from the target image”. Concerning the effect of the choice of detrending technique on the subsequent analysis, it is hard to answer certainly, however, we expect negligible changes.
3. A study and discussion comparing the dominant land-cover type and topography to the detected pixels where the trend or change-points were detected in the Navarre MODIS data. Were these urban regions? Mountainous regions? Water-covered regions? Agricultural regions? Desert/barren
regions? etc.
Authors’ response:
Thanks. We have now added clarifying comments. The corresponding text reads as “Taking the map of Navarre, displayed in Figure 9, into account we underline that the detected locations in the south of Navarre belong to a zone with both agricultural and urban land-cover. This area is also exposed by a warmer weather than north (see Figures 10 and 11). The Ebro river, that is the most important river in Spain in terms of length and area of drainage basin, also flows through this area. Although, a few detected pixels in the north-west and north-east of Navarre are located in urban regions, most of the detected pixels belong to green areas and mountainous regions”.
