remotesensing-logo

Journal Browser

Journal Browser

Advances in Deep Learning Techniques for the Analysis of Remote Sensing Time Series

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (30 November 2022) | Viewed by 22284

Special Issue Editor

IRISA, UMR CNRS 6074, Université de Bretagne-Sud, Campus de Tohannic, BP 573, 56 000 Vannes, France
Interests: time series classification; satellite image time series; mislabeled training data; domain adaptation; gapfilling

Special Issue Information

Dear Colleagues,

New sensors are now acquiring massive and dense remote sensing time series of the Earth’s surface at a large scale. The joint use of different sensors provides a rich description of a phenomenon or a scene in various modalities (optical, radar, hyperspectral, lidar, street views, etc.). For example, both Sentinel-1 and Sentinel-2 constellations’ sensors (from the European Copernicus programme) acquire satellite images about every five days, at a medium spatial resolution, of all emerged surfaces in radar and optical domains, respectively. The availability of this unprecedented quantity of remote sensing time series data opens new opportunities in many applications such as land use land cover mapping, crop mapping, change detection, yield estimation, characterization of urban growth, and forest and soil monitoring. They also offer the possibility to track land surface dynamics through long-term analysis and in near real-time.

In recent years, deep learning techniques have arisen as an efficient tool for processing (remote sensing) time series data in various tasks (classification, clustering, forecasting, or regression). Although these approaches can handle the massive quantity of time series data acquired by remote sensors, they require much high-quality labelled data to be applied at regional or continental scales. This is not always available in remote sensing because it is hard to label large areas of the Earth at a high resolution and land cover changes more frequently than the labels are updated. Other difficulties to apply deep learning techniques developed in computer vision come from the complexity and specificity of remote sensing time series data that are massive, multivariate, noisy, and irregularly sampled. Furthermore, decisions and predictions from those models cannot be easily explained, which prevents a meaningful understanding of the dynamic phenomena observed.

This Special Issue will feature significant and innovative contributions on topics such as the following:

  • Innovative deep learning algorithms that handle the complexity and specificity of remote sensing time series (spatio-temporal data cubes, multivariate, noisy, and irregular sampled) and their processing (gap-filling, time series segmentation, super-resolution).
  • Multi-sensor data fusion techniques, which efficiently combine EO time series acquired by several sensors in various modalities.
  • Novel frameworks to deal with the scarcity and/or the low quality of labelled data including unsupervised, semi-supervised, self-supervised, active, adversarial, and transfer learning.
  • Explainable deep learning approaches to improve the understanding of soil surface dynamics.
  • Long-term and data stream analyses in the scope of land cover mapping, land use land cover change detection, yield estimation, crop and forest mapping, urban growth.
  • New datasets to benchmark deep learning for remote sensing time series analysis.

Dr. Charlotte Pelletier
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Time series analysis
  • Satellite Image Time Series
  • Deep learning
  • Data fusion
  • Data stream
  • Explainability
  • Unsupervised and semi-supervised domain adaptation
  • Multi-sensors
  • Long term analysis
  • Large scale analysis
  • Land cover mapping
  • Land use land cover
  • Change detection
  • Benchmarked datasets

Published Papers (6 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

21 pages, 6811 KiB  
Article
Convolutional Neural Network Shows Greater Spatial and Temporal Stability in Multi-Annual Land Cover Mapping Than Pixel-Based Methods
by Tony Boston, Albert Van Dijk and Richard Thackway
Remote Sens. 2023, 15(8), 2132; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15082132 - 18 Apr 2023
Cited by 3 | Viewed by 1283
Abstract
Satellite imagery is the only feasible approach to annual monitoring and reporting on land cover change. Unfortunately, conventional pixel-based classification methods based on spectral response only (e.g., using random forests algorithms) have shown a lack of spatial and temporal stability due, for instance, [...] Read more.
Satellite imagery is the only feasible approach to annual monitoring and reporting on land cover change. Unfortunately, conventional pixel-based classification methods based on spectral response only (e.g., using random forests algorithms) have shown a lack of spatial and temporal stability due, for instance, to variability between individual pixels and changes in vegetation condition, respectively. Machine learning methods that consider spatial patterns in addition to reflectance can address some of these issues. In this study, a convolutional neural network (CNN) model, U-Net, was trained for a 500 km × 500 km region in southeast Australia using annual Landsat geomedian data for the relatively dry and wet years of 2018 and 2020, respectively. The label data for model training was an eight-class classification inferred from a static land-use map, enhanced using forest-extent mapping. Here, we wished to analyse the benefits of CNN-based land cover mapping and reporting over 34 years (1987–2020). We used the trained model to generate annual land cover maps for a 100 km × 100 km tile near the Australian Capital Territory. We developed innovative diagnostic methods to assess spatial and temporal stability, analysed how the CNN method differs from pixel-based mapping and compared it with two reference land cover products available for some years. Our U-Net CNN results showed better spatial and temporal stability with, respectively, overall accuracy of 89% verses 82% for reference pixel-based mapping, and 76% of pixels unchanged over 33 years. This gave a clearer insight into where and when land cover change occurred compared to reference mapping, where only 30% of pixels were conserved. Remaining issues include edge effects associated with the CNN method and a limited ability to distinguish some land cover types (e.g., broadacre crops vs. pasture). We conclude that the CNN model was better for understanding broad-scale land cover change, use in environmental accounting and natural resource management, whereas pixel-based approaches sometimes more accurately represented small-scale changes in land cover. Full article
Show Figures

Figure 1

16 pages, 8848 KiB  
Article
SAR and Optical Data Applied to Early-Season Mapping of Integrated Crop–Livestock Systems Using Deep and Machine Learning Algorithms
by Ana P. S. G. D. D. Toro, Inacio T. Bueno, João P. S. Werner, João F. G. Antunes, Rubens A. C. Lamparelli, Alexandre C. Coutinho, Júlio C. D. M. Esquerdo, Paulo S. G. Magalhães and Gleyce K. D. A. Figueiredo
Remote Sens. 2023, 15(4), 1130; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15041130 - 18 Feb 2023
Cited by 1 | Viewed by 2007
Abstract
Regenerative agricultural practices are a suitable path to feed the global population. Integrated Crop–livestock systems (ICLSs) are key approaches once the area provides animal and crop production resources. In Brazil, the expectation is to increase the area of ICLS fields by 5 million [...] Read more.
Regenerative agricultural practices are a suitable path to feed the global population. Integrated Crop–livestock systems (ICLSs) are key approaches once the area provides animal and crop production resources. In Brazil, the expectation is to increase the area of ICLS fields by 5 million hectares in the next five years. However, few methods have been tested regarding spatial and temporal scales to map and monitor ICLS fields, and none of these methods use SAR data. Therefore, in this work, we explored the potential of three machine and deep learning algorithms (random forest, long short-term memory, and transformer) to perform early-season (with three-time windows) mapping of ICLS fields. To explore the scalability of the proposed methods, we tested them in two regions with different latitudes, cloud cover rates, field sizes, landscapes, and crop types. Finally, the potential of SAR (Sentinel-1) and optical (Sentinel-2) data was tested. As a result, we found that all proposed algorithms and sensors could correctly map both study sites. For Study Site 1(SS1), we obtained an overall accuracy of 98% using the random forest classifier. For Study Site 2, we obtained an overall accuracy of 99% using the long short-term memory net and the random forest. Further, the early-season experiments were successful for both study sites (with an accuracy higher than 90% for all time windows), and no significant difference in accuracy was found among them. Thus, this study found that it is possible to map ICLSs in the early-season and in different latitudes by using diverse algorithms and sensors. Full article
Show Figures

Graphical abstract

23 pages, 62828 KiB  
Article
Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset
by Romain Wenger, Anne Puissant, Jonathan Weber, Lhassane Idoumghar and Germain Forestier
Remote Sens. 2023, 15(1), 151; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15010151 - 27 Dec 2022
Cited by 3 | Viewed by 2536
Abstract
In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as [...] Read more.
In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise understanding of a given territory. Few studies have combined information from Sentinel-1 and Sentinel-2 imagery, but merging radar and optical imagery has been shown to have several benefits for a range of study cases, such as semantic segmentation or classification. For this study, we used a newly produced dataset, MultiSenGE, which provides a set of multitemporal and multimodal patches over the Grand-Est region in France. To merge these data, we propose a CNN approach based on spatio-temporal and spatio-spectral feature fusion, ConvLSTM+Inception-S1S2. We used a U-Net base model and ConvLSTM extractor for spatio-temporal features and an inception module for the spatio-spectral features extractor. The results show that describing an overrepresented class is preferable to map urban fabrics (UF). Furthermore, the addition of an Inception module on a date allowing the extraction of spatio-spectral features improves the classification results. Spatio-spectro-temporal method (ConvLSTM+Inception-S1S2) achieves higher global weighted F1Score than all other methods tested. Full article
Show Figures

Figure 1

24 pages, 6933 KiB  
Article
Scalable Crop Yield Prediction with Sentinel-2 Time Series and Temporal Convolutional Network
by Maria Yli-Heikkilä, Samantha Wittke, Markku Luotamo, Eetu Puttonen, Mika Sulkava, Petri Pellikka, Janne Heiskanen and Arto Klami
Remote Sens. 2022, 14(17), 4193; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14174193 - 25 Aug 2022
Cited by 10 | Viewed by 7935
Abstract
One of the precepts of food security is the proper functioning of the global food markets. This calls for open and timely intelligence on crop production on an agroclimatically meaningful territorial scale. We propose an operationally suitable method for large-scale in-season crop yield [...] Read more.
One of the precepts of food security is the proper functioning of the global food markets. This calls for open and timely intelligence on crop production on an agroclimatically meaningful territorial scale. We propose an operationally suitable method for large-scale in-season crop yield estimations from a satellite image time series (SITS) for statistical production. As an object-based method, it is spatially scalable from parcel to regional scale, making it useful for prediction tasks in which the reference data are available only at a coarser level, such as counties. We show that deep learning-based temporal convolutional network (TCN) outperforms the classical machine learning method random forests and produces more accurate results overall than published national crop forecasts. Our novel contribution is to show that mean-aggregated regional predictions with histogram-based features calculated from farm-level observations perform better than other tested approaches. In addition, TCN is robust to the presence of cloudy pixels, suggesting TCN can learn cloud masking from the data. The temporal compositing of information do not improve prediction performance. This indicates that with end-to-end learning less preprocessing in SITS tasks seems viable. Full article
Show Figures

Figure 1

13 pages, 4451 KiB  
Communication
A Deep Multitask Semisupervised Learning Approach for Chlorophyll-a Retrieval from Remote Sensing Images
by Melike Ilteralp, Sema Ariman and Erchan Aptoula
Remote Sens. 2022, 14(1), 18; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14010018 - 22 Dec 2021
Cited by 5 | Viewed by 3419
Abstract
This article addresses the scarcity of labeled data in multitemporal remote sensing image analysis, and especially in the context of Chlorophyll-a (Chl-a) estimation for inland water quality assessment. We propose a multitask CNN architecture that can exploit unlabeled satellite imagery and that can [...] Read more.
This article addresses the scarcity of labeled data in multitemporal remote sensing image analysis, and especially in the context of Chlorophyll-a (Chl-a) estimation for inland water quality assessment. We propose a multitask CNN architecture that can exploit unlabeled satellite imagery and that can be generalized to other multitemporal remote sensing image analysis contexts where the target parameter exhibits seasonal fluctuations. Specifically, Chl-a estimation is set as the main task, and an unlabeled sample’s month classification is set as an auxiliary network task. The proposed approach is validated with multitemporal/spectral Sentinel-2 images of Lake Balik in Turkey using in situ measurements acquired during 2017–2019. We show that harnessing unlabeled data through multitask learning improves water quality estimation performance. Full article
Show Figures

Graphical abstract

16 pages, 10069 KiB  
Article
Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series
by Félix Quinton and Loic Landrieu
Remote Sens. 2021, 13(22), 4599; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13224599 - 16 Nov 2021
Cited by 3 | Viewed by 3123
Abstract
While annual crop rotations play a crucial role for agricultural optimization, they have been largely ignored for automated crop type mapping. In this paper, we take advantage of the increasing quantity of annotated satellite data to propose to model simultaneously the inter- and [...] Read more.
While annual crop rotations play a crucial role for agricultural optimization, they have been largely ignored for automated crop type mapping. In this paper, we take advantage of the increasing quantity of annotated satellite data to propose to model simultaneously the inter- and intra-annual agricultural dynamics of yearly parcel classification with a deep learning approach. Along with simple training adjustments, our model provides an improvement of over 6.3% mIoU over the current state-of-the-art of crop classification, and a reduction of over 21% of the error rate. Furthermore, we release the first large-scale multi-year agricultural dataset with over 300,000 annotated parcels. Full article
Show Figures

Graphical abstract

Back to TopTop