Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Browser

Deep Learning-Based Cloud Detection for Remote Sensing Images

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (31 July 2022) | Viewed by 26220

Share This Special Issue

Special Issue Editor

Prof. Dr. Luis Gómez-Chova

E-Mail Website
Guest Editor

Image Processing Laboratory, University of Valencia, E-46100 Burjassot (Valencia), Spain
Interests: machine learning; image and signal processing; remote sensing; multispectral images; cloud detection

Special Issue Information

Dear Colleagues,

The number of Earth observation satellites is growing exponentially. However, satellite images acquired by optical sensors can be affected significantly by the presence of clouds, which can be considered as a source of uncertainty, when the objective is to study the surface, or as the signal, when studying the atmosphere. In any case, automatic detection of clouds becomes mandatory for the operational exploitation of Earth observation satellite images and further retrieval of derived bio-physical products. Moreover, classification of cloud types is also gaining importance since clouds represent one of the bigger sources of uncertainty in climate projections.

Despite the efforts in developing robust and reliable cloud detection algorithms, there are still significant deficiencies in their overall accuracy, notably false detection over bright surfaces, omissions of optically thin clouds or confusion in snow/ice and cloud classification. Standard machine learning algorithms have provided promising results in this field but they do not efficiently cope with some aspects of cloud detection in satellite images. Deep learning-based algorithms could address key challenges that require urgent attention in cloud detection for current and upcoming satellites. Novel deep learning architectures and training procedures are required to better capture the spatial and spectral properties of Earth observation satellite images. Deep learning models excel exploiting the wealth of information contained in available labeled datasets, however, the generation of reference and public multi-mission datasets of satellite images for cloud detection is a key requirement that has to be better handled by the community.

In short, this Special Issue will review the state of the art in deep learning-based cloud detection algorithms for remote sensing images. Articles covering recent research about the following topics are invited for this Special Issue:

New machine learning algorithms based on deep architectures for cloud detection.
Cloud detection applications in multispectral and high-spatial resolution sensors.
Deep learning-based cloud detection algorithms exploiting the temporal information.
Cloud type classification.
Cloud and cloud shadows removal from optical satellite images.
Transfer learning approaches across similar satellite sensors.
Improving domain adaptation among different satellites for cloud detection.
Relevant datasets particularly interesting for cloud detection.
Intelligent and active data labeling methods for cloud detection.
Search and retrieval of cloud-free satellite images.

Prof. Dr. Luis Gómez-Chova
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Cloud detection
Cloud removal
Cloud shadows
Cloud type classification
Earth observation satellites
Multispectral images
Image processing
Machine learning
Deep learning
Transfer learning

Published Papers (8 papers)

Download All Papers

Research

Jump to: Other

22 pages, 11227 KiB

Open AccessArticle

Optical Remote Sensing Image Cloud Detection with Self-Attention and Spatial Pyramid Pooling Fusion

by Weihua Pu, Zhipan Wang, Di Liu and Qingling Zhang

Remote Sens. 2022, 14(17), 4312; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14174312 - 01 Sep 2022

Cited by 11 | Viewed by 2009

Abstract

Cloud detection is a key step in optical remote sensing image processing, and the cloud-free image is of great significance for land use classification, change detection, and long time-series landcover monitoring. Traditional cloud detection methods based on spectral and texture features have acquired certain effects in complex scenarios, such as cloud–snow mixing, but there is still a large room for improvement in terms of generation ability. In recent years, cloud detection with deep-learning methods has significantly improved the accuracy in complex regions such as high-brightness feature mixing areas. However, the existing deep learning-based cloud detection methods still have certain limitations. For instance, a few omission alarms and commission alarms still exist in cloud edge regions. At present, the cloud detection methods based on deep learning are gradually converted from a pure convolutional structure to a global feature extraction perspective, such as attention modules, but the computational burden is also increased, which is difficult to meet for the rapidly developing time-sensitive tasks, such as onboard real-time cloud detection in optical remote sensing imagery. To address the above problems, this manuscript proposes a high-precision cloud detection network fusing a self-attention module and spatial pyramidal pooling. Firstly, we use the DenseNet network as the backbone, then the deep semantic features are extracted by combining a global self-attention module and spatial pyramid pooling module. Secondly, to solve the problem of unbalanced training samples, we design a weighted cross-entropy loss function to optimize it. Finally, cloud detection accuracy is assessed. With the quantitative comparison experiments on different images, such as Landsat8, Landsat9, GF-2, and Beijing-2, the results indicate that, compared with the feature-based methods, the deep learning network can effectively distinguish in the cloud–snow confusion-prone region using only visible three-channel images, which significantly reduces the number of required image bands. Compared with other deep learning methods, the accuracy at the edge of the cloud region is higher and the overall computational efficiency is relatively optimal. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Graphical abstract

20 pages, 6752 KiB

Open AccessArticle

A Novel Method for Ground-Based Cloud Image Classification Using Transformer

by Xiaotong Li, Bo Qiu, Guanlong Cao, Chao Wu and Liwen Zhang

Remote Sens. 2022, 14(16), 3978; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14163978 - 16 Aug 2022

Cited by 8 | Viewed by 2252

Abstract

In recent years, convolutional neural networks (CNNs) have achieved competitive performance in the field of ground-based cloud image (GCI) classification. Proposed CNN-based methods can fully extract the local features of images. However, due to the locality of the convolution operation, they cannot well establish the long-range dependencies between the images, and thus they cannot extract the global features of images. Transformer has been applied to computer vision with great success due to its powerful global modeling capability. Inspired by it, we propose a Transformer-based GCI classification method that combines the advantages of the CNN and Transformer models. Firstly, the CNN model acts as a low-level feature extraction tool to generate local feature sequences of images. Then, the Transformer model is used to learn the global features of the images by efficiently extracting the long-range dependencies between the sequences. Finally, a linear classifier is used for GCI classification. In addition, we introduce a center loss function to address the problem of the simple cross-entropy loss not adequately supervising feature learning. Our method is evaluated on three commonly used datasets: ASGC, CCSN, and GCD. The experimental results show that the method achieves 94.24%, 92.73%, and 93.57% accuracy, respectively, outperforming other state-of-the-art methods. It proves that Transformer has great potential to be applied to GCI classification tasks. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Graphical abstract

25 pages, 6576 KiB

Open AccessArticle

Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

by Luiz E. Christovam, Milton H. Shimabukuro, Maria de Lourdes B. T. Galo and Eija Honkavaara

Remote Sens. 2022, 14(1), 144; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14010144 - 29 Dec 2021

Cited by 11 | Viewed by 3158

Abstract

Clouds are one of the major limitations to crop monitoring using optical satellite images. Despite all efforts to provide decision-makers with high-quality agricultural statistics, there is still a lack of techniques to optimally process satellite image time series in the presence of clouds. In this regard, in this article it was proposed to add a Multi-Layer Perceptron loss function to the pix2pix conditional Generative Adversarial Network (cGAN) objective function. The aim was to enforce the generative model to learn how to deliver synthetic pixels whose values were proxies for the spectral response improving further crop type mapping. Furthermore, it was evaluated the generalization capacity of the generative models in producing pixels with plausible values for images not used in the training. To assess the performance of the proposed approach it was compared real images with synthetic images generated with the proposed approach as well as with the original pix2pix cGAN. The comparative analysis was performed through visual analysis, pixel values analysis, semantic segmentation and similarity metrics. In general, the proposed approach provided slightly better synthetic pixels than the original pix2pix cGAN, removing more noise than the original pix2pix algorithm as well as providing better crop type semantic segmentation; the semantic segmentation of the synthetic image generated with the proposed approach achieved an F1-score of 44.2%, while the real image achieved 44.7%. Regarding the generalization, the models trained utilizing different regions of the same image provided better pixels than models trained using other images in the time series. Besides this, the experiments also showed that the models trained using a pair of images selected every three months along the time series also provided acceptable results on images that do not have cloud-free areas. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Figure 1

25 pages, 9268 KiB

Open AccessArticle

UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition

by Zhanjie Wang, Jianghua Zhao, Ran Zhang, Zheng Li, Qinghui Lin and Xuezhi Wang

Remote Sens. 2022, 14(1), 104; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14010104 - 26 Dec 2021

Cited by 31 | Viewed by 3712

Abstract

Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Figure 1

33 pages, 14075 KiB

Open AccessArticle

Controllably Deep Supervision and Multi-Scale Feature Fusion Network for Cloud and Snow Detection Based on Medium- and High-Resolution Imagery Dataset

by Guangbin Zhang, Xianjun Gao, Yuanwei Yang, Mingwei Wang and Shuhao Ran

Remote Sens. 2021, 13(23), 4805; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13234805 - 26 Nov 2021

Cited by 18 | Viewed by 2268

Abstract

Clouds and snow in remote sensing imageries cover underlying surface information, reducing image availability. Moreover, they interact with each other, decreasing the cloud and snow detection accuracy. In this study, we propose a convolutional neural network for cloud and snow detection, named the cloud and snow detection network (CSD-Net). It incorporates the multi-scale feature fusion module (MFF) and the controllably deep supervision and feature fusion structure (CDSFF). MFF can capture and aggregate features at various scales, ensuring that the extracted high-level semantic features of clouds and snow are more distinctive. CDSFF can provide a deeply supervised mechanism with hinge loss and combine information from adjacent layers to gain more representative features. It ensures the gradient flow is more oriented and error-less, while retaining more effective information. Additionally, a high-resolution cloud and snow dataset based on WorldView2 (CSWV) was created and released. This dataset meets the training requirements of deep learning methods for clouds and snow in high-resolution remote sensing images. Based on the datasets with varied resolutions, CSD-Net is compared to eight state-of-the-art deep learning methods. The experiment results indicate that CSD-Net has an excellent detection accuracy and efficiency. Specifically, the mean intersection over the union (MIoU) of CSD-Net is the highest in the corresponding experiment. Furthermore, the number of parameters in our proposed network is just 7.61 million, which is the lowest of the tested methods. It only has 88.06 GFLOPs of floating point operations, which is less than the U-Net, DeepLabV3+, PSPNet, SegNet-Modified, MSCFF, and GeoInfoNet. Meanwhile, CSWV has a higher annotation quality since the same method can obtain a greater accuracy on it. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Figure 1

24 pages, 8401 KiB

Open AccessArticle

Light-Weight Cloud Detection Network for Optical Remote Sensing Images with Attention-Based DeeplabV3+ Architecture

by Xudong Yao, Qing Guo and An Li

Remote Sens. 2021, 13(18), 3617; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13183617 - 10 Sep 2021

Cited by 17 | Viewed by 2744

Abstract

Clouds in optical remote sensing images cause spectral information change or loss, that affects image analysis and application. Therefore, cloud detection is of great significance. However, there are some shortcomings in current methods, such as the insufficient extendibility due to using the information of multiple bands, the intense extendibility due to relying on some manually determined thresholds, and the limited accuracy, especially for thin clouds or complex scenes caused by low-level manual features. Combining the above shortcomings and the requirements for efficiency in practical applications, we propose a light-weight deep learning cloud detection network based on DeeplabV3+ architecture and channel attention module (CD-AttDLV3+), only using the most common red–green–blue and near-infrared bands. In the CD-AttDLV3+ architecture, an optimized backbone network-MobileNetV2 is used to reduce the number of parameters and calculations. Atrous spatial pyramid pooling effectively reduces the information loss caused by multiple down-samplings while extracting multi-scale features. CD-AttDLV3+ concatenates more low-level features than DeeplabV3+ to improve the cloud boundary quality. The channel attention module is introduced to strengthen the learning of important channels and improve the training efficiency. Moreover, the loss function is improved to alleviate the imbalance of samples. For the Landsat-8 Biome set, CD-AttDLV3+ achieves the highest accuracy in comparison with other methods, including Fmask, SVM, and SegNet, especially for distinguishing clouds from bright surfaces and detecting light-transmitting thin clouds. It can also perform well on other Landsat-8 and Sentinel-2 images. Experimental results indicate that CD-AttDLV3+ is robust, with a high accuracy and extendibility. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Figure 1

20 pages, 41970 KiB

Open AccessArticle

Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images

by Dan López-Puigdollers, Gonzalo Mateo-García and Luis Gómez-Chova

Remote Sens. 2021, 13(5), 992; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13050992 - 05 Mar 2021

Cited by 33 | Viewed by 6091

Abstract

The systematic monitoring of the Earth using optical satellites is limited by the presence of clouds. Accurately detecting these clouds is necessary to exploit satellite image archives in remote sensing applications. Despite many developments, cloud detection remains an unsolved problem with room for improvement, especially over bright surfaces and thin clouds. Recently, advances in cloud masking using deep learning have shown significant boosts in cloud detection accuracy. However, these works are validated in heterogeneous manners, and the comparison with operational threshold-based schemes is not consistent among many of them. In this work, we systematically compare deep learning models trained on Landsat-8 images on different Landsat-8 and Sentinel-2 publicly available datasets. Overall, we show that deep learning models exhibit a high detection accuracy when trained and tested on independent images from the same Landsat-8 dataset (intra-dataset validation), outperforming operational algorithms. However, the performance of deep learning models is similar to operational threshold-based ones when they are tested on different datasets of Landsat-8 images (inter-dataset validation) or datasets from a different sensor with similar radiometric characteristics such as Sentinel-2 (cross-sensor validation). The results suggest that (i) the development of cloud detection methods for new satellites can be based on deep learning models trained on data from similar sensors and (ii) there is a strong dependence of deep learning models on the dataset used for training and testing, which highlights the necessity of standardized datasets and procedures for benchmarking cloud detection models in the future. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures

Graphical abstract

Other

Jump to: Research

13 pages, 1841 KiB

Open AccessTechnical Note

A Novel Ground-Based Cloud Image Segmentation Method Based on a Multibranch Asymmetric Convolution Module and Attention Mechanism

by Liwen Zhang, Wenhao Wei, Bo Qiu, Ali Luo, Mingru Zhang and Xiaotong Li

Remote Sens. 2022, 14(16), 3970; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14163970 - 16 Aug 2022

Cited by 2 | Viewed by 1640

Abstract

Cloud segmentation is a fundamental step in accurately acquiring cloud cover. However, due to the nonrigid structures of clouds, traditional cloud segmentation methods perform worse than expected. In this paper, a novel deep convolutional neural network (CNN) named MA-SegCloud is proposed for segmenting cloud images based on a multibranch asymmetric convolution module (MACM) and an attention mechanism. The MACM is composed of asymmetric convolution, depth-separable convolution, and a squeeze-and-excitation module (SEM). The MACM not only enables the network to capture more contextual information in a larger area but can also adaptively adjust the feature channel weights. The attention mechanisms SEM and convolutional block attention module (CBAM) in the network can strengthen useful features for cloud image segmentation. As a result, MA-SegCloud achieves a 96.9% accuracy, 97.0% precision, 97.0% recall, 97.0% F-score, 3.1% error rate, and 94.0% mean intersection-over-union (MIoU) on the Singapore Whole-sky Nychthemeron Image Segmentation (SWINySEG) dataset. Extensive evaluations demonstrate that MA-SegCloud performs favorably against state-of-the-art cloud image segmentation methods. Full article

(This article belongs to the Special Issue Deep Learning-Based Cloud Detection for Remote Sensing Images)

► Show Figures