Research

22 pages, 8732 KiB

Open AccessArticle

Anthropogenic Object Localization: Evaluation of Broad-Area High-Resolution Imagery Scans Using Deep Learning in Overhead Imagery

by J. Alex Hurt, Ilinca Popescu, Curt H. Davis and Grant J. Scott

Sensors 2023, 23(18), 7766; https://0-doi-org.brum.beds.ac.uk/10.3390/s23187766 - 08 Sep 2023

Viewed by 563

Abstract

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has [...] Read more.

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has exceeded 99% when evaluated using cross-validation techniques. However, real-world remote sensing data are truly big data, which often exceed billions of pixels. Therefore, one of the greatest challenges regarding the evaluation of machine learning models taken out of the clean laboratory setting and into the real world is the difficulty of measuring performance. It is necessary to evaluate these models on a grander scale, namely, tens of thousands of square kilometers, where it is intractable to the ground truth and the ever-changing anthropogenic surface of Earth. The ultimate goal of computer vision model development for automated analysis and broad area search and discovery is to augment and assist humans, specifically human–machine teaming for real-world tasks. In this research, various models have been trained using object classes from benchmark datasets such as UC Merced, PatternNet, RESISC-45, and MDSv2. We detail techniques to scan broad swaths of the Earth with deep convolutional neural networks. We present algorithms for localizing object detection results, as well as a methodology for the evaluation of the results of broad-area scans. Our research explores the challenges of transitioning these models out of the training–validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art deep convolutional neural networks for the search, detection, and annotation of objects within large swaths of imagery, with the ultimate goal of providing a methodology for evaluating object detection machine learning models in real-world scenarios. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

16 pages, 2154 KiB

Open AccessArticle

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network

by Mingtian Li, Yu Lu, Shixian Cao, Xinyu Wang and Shanjuan Xie

Sensors 2023, 23(6), 3190; https://0-doi-org.brum.beds.ac.uk/10.3390/s23063190 - 16 Mar 2023

Cited by 5 | Viewed by 1712

Abstract

Recently, convolution neural networks have been widely used in hyperspectral image classification and have achieved excellent performance. However, the fixed convolution kernel receptive field often leads to incomplete feature extraction, and the high redundancy of spectral information leads to difficulties in spectral feature [...] Read more.

Recently, convolution neural networks have been widely used in hyperspectral image classification and have achieved excellent performance. However, the fixed convolution kernel receptive field often leads to incomplete feature extraction, and the high redundancy of spectral information leads to difficulties in spectral feature extraction. To solve these problems, we propose a nonlocal attention mechanism of a 2D–3D hybrid CNN (2-3D-NL CNN), which includes an inception block and a nonlocal attention module. The inception block uses convolution kernels of different sizes to equip the network with multiscale receptive fields to extract the multiscale spatial features of ground objects. The nonlocal attention module enables the network to obtain a more comprehensive receptive field in the spatial and spectral dimensions while suppressing the information redundancy of the spectral dimension, making the extraction of spectral features easier. Experiments on two hyperspectral datasets, Pavia University and Salians, validate the effectiveness of the inception block and the nonlocal attention module. The results show that our model achieves an overall classification accuracy of 99.81% and 99.42% on the two datasets, respectively, which is higher than the accuracy of the existing model. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

23 pages, 19683 KiB

Open AccessArticle

YOLOv4 with Deformable-Embedding-Transformer Feature Extractor for Exact Object Detection in Aerial Imagery

by Yiheng Wu and Jianjun Li

Sensors 2023, 23(5), 2522; https://0-doi-org.brum.beds.ac.uk/10.3390/s23052522 - 24 Feb 2023

Cited by 5 | Viewed by 2195

Abstract

The deep learning method for natural-image object detection tasks has made tremendous progress in recent decades. However, due to multiscale targets, complex backgrounds, and high-scale small targets, methods from the field of natural images frequently fail to produce satisfactory results when applied to [...] Read more.

The deep learning method for natural-image object detection tasks has made tremendous progress in recent decades. However, due to multiscale targets, complex backgrounds, and high-scale small targets, methods from the field of natural images frequently fail to produce satisfactory results when applied to aerial images. To address these problems, we proposed the DET-YOLO enhancement based on YOLOv4. Initially, we employed a vision transformer to acquire highly effective global information extraction capabilities. In the transformer, we proposed deformable embedding instead of linear embedding and a full convolution feedforward network (FCFN) instead of a feedforward network in order to reduce the feature loss caused by cutting in the embedding process and improve the spatial feature extraction capability. Second, for improved multiscale feature fusion in the neck, we employed a depth direction separable deformable pyramid module (DSDP) rather than a feature pyramid network. Experiments on the DOTA, RSOD, and UCAS-AOD datasets demonstrated that our method’s average accuracy (mAP) values reached 0.728, 0.952, and 0.945, respectively, which were comparable to the existing state-of-the-art methods. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

18 pages, 3269 KiB

Open AccessArticle

Semantic Segmentation of Hyperspectral Remote Sensing Images Based on PSE-UNet Model

by Jiaju Li, Hefeng Wang, Anbing Zhang and Yuliang Liu

Sensors 2022, 22(24), 9678; https://0-doi-org.brum.beds.ac.uk/10.3390/s22249678 - 10 Dec 2022

Cited by 10 | Viewed by 2552

Abstract

With the development of deep learning, the use of convolutional neural networks (CNN) to improve the land cover classification accuracy of hyperspectral remote sensing images (HSRSI) has become a research hotspot. In HSRSI semantics segmentation, the traditional dataset partition method may cause information [...] Read more.

With the development of deep learning, the use of convolutional neural networks (CNN) to improve the land cover classification accuracy of hyperspectral remote sensing images (HSRSI) has become a research hotspot. In HSRSI semantics segmentation, the traditional dataset partition method may cause information leakage, which poses challenges for a fair comparison between models. The performance of the model based on “convolutional-pooling-fully connected” structure is limited by small sample sizes and high dimensions of HSRSI. Moreover, most current studies did not involve how to choose the number of principal components with the application of the principal component analysis (PCA) to reduce dimensionality. To overcome the above challenges, firstly, the non-overlapping sliding window strategy combined with the judgment mechanism is introduced, used to split the hyperspectral dataset. Then, a PSE-UNet model for HSRSI semantic segmentation is designed by combining PCA, the attention mechanism, and UNet, and the factors affecting the performance of PSE-UNet are analyzed. Finally, the cumulative variance contribution rate (CVCR) is introduced as a dimensionality reduction metric of PCA to study the Hughes phenomenon. The experimental results with the Salinas dataset show that the PSE-UNet is superior to other semantic segmentation algorithms and the results can provide a reference for HSRSI semantic segmentation. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

18 pages, 15517 KiB

Open AccessArticle

An Efficient and Uncertainty-Aware Decision Support System for Disaster Response Using Aerial Imagery

by Junchi Bin, Ran Zhang, Rui Wang, Yue Cao, Yufeng Zheng, Erik Blasch and Zheng Liu

Sensors 2022, 22(19), 7167; https://0-doi-org.brum.beds.ac.uk/10.3390/s22197167 - 21 Sep 2022

Viewed by 1215

Abstract

Efficient and robust search and rescue actions are always required when natural or technical disasters occur. Empowered by remote sensing techniques, building damage assessment can be achieved by fusing aerial images of pre- and post-disaster environments through computational models. Existing methods pay over-attention [...] Read more.

Efficient and robust search and rescue actions are always required when natural or technical disasters occur. Empowered by remote sensing techniques, building damage assessment can be achieved by fusing aerial images of pre- and post-disaster environments through computational models. Existing methods pay over-attention to assessment accuracy without considering model efficiency and uncertainty quantification in such a life-critical application. Thus, this article proposes an efficient and uncertain-aware decision support system (EUDSS) that evolves the recent computational models into an efficient decision support system, realizing the uncertainty during building damage assessment (BDA). Specifically, a new efficient and uncertain-aware BDA integrates the recent advances in computational models such as Fourier attention and Monte Carlo Dropout for uncertainty quantification efficiently. Meanwhile, a robust operation (RO) procedure is designed to invite experts for manual reviews if the uncertainty is high due to external factors such as cloud clutter and poor illumination. This procedure can prevent rescue teams from missing damaged houses during operations. The effectiveness of the proposed system is demonstrated on a public dataset from both quantitative and qualitative perspectives. The solution won the first place award in International Overhead Imagery Hackathon. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

17 pages, 9910 KiB

Open AccessArticle

An Improved CenterNet Model for Insulator Defect Detection Using Aerial Imagery

by Haiyang Xia, Baohua Yang, Yunlong Li and Bing Wang

Sensors 2022, 22(8), 2850; https://0-doi-org.brum.beds.ac.uk/10.3390/s22082850 - 08 Apr 2022

Cited by 25 | Viewed by 2935

Abstract

For the issue of low accuracy and poor real-time performance of insulator and defect detection by an unmanned aerial vehicle (UAV) in the process of power inspection, an insulator detection model MobileNet_CenterNet was proposed in this study. First, the lightweight network MobileNet V1 [...] Read more.

For the issue of low accuracy and poor real-time performance of insulator and defect detection by an unmanned aerial vehicle (UAV) in the process of power inspection, an insulator detection model MobileNet_CenterNet was proposed in this study. First, the lightweight network MobileNet V1 was used to replace the feature extraction network Resnet-50 of the original model, aiming to ensure the detection accuracy of the model while speeding up its detection speed. Second, a spatial and channel attention mechanism convolutional block attention module (CBAM) was introduced in CenterNet, aiming to improve the prediction accuracy of small target insulator position information. Then, three transposed convolution modules were added for upsampling, aiming to better restore the semantic information and position information of the image. Finally, the insulator dataset (ID) constructed by ourselves and the public dataset (CPLID) were used for model training and validation, aiming to improve the generalization ability of the model. The experimental results showed that compared with the CenterNet model, MobileNet_CenterNet improved the detection accuracy by 12.2%, the inference speed by 1.1 f/s for FPS-CPU and 4.9 f/s for FPS-GPU, and the model size was reduced by 37 MB. Compared with other models, our proposed model improved both detection accuracy and inference speed, indicating that the MobileNet_CenterNet model had better real-time performance and robustness. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

18 pages, 8107 KiB

Open AccessArticle

Region-Based CNN for Anomaly Detection in PV Power Plants Using Aerial Imagery

by Michiel Vlaminck, Rugen Heidbuchel, Wilfried Philips and Hiep Luong

Sensors 2022, 22(3), 1244; https://0-doi-org.brum.beds.ac.uk/10.3390/s22031244 - 07 Feb 2022

Cited by 21 | Viewed by 4066

Abstract

Today, solar energy is taking an increasing share of the total energy mix. Unfortunately, many operational photovoltaic plants suffer from a plenitude of defects resulting in non-negligible power loss. The latter highly impacts the overall performance of the PV site; therefore, operators need [...] Read more.

Today, solar energy is taking an increasing share of the total energy mix. Unfortunately, many operational photovoltaic plants suffer from a plenitude of defects resulting in non-negligible power loss. The latter highly impacts the overall performance of the PV site; therefore, operators need to regularly inspect their solar parks for anomalies in order to prevent severe performance drops. As this operation is naturally labor-intensive and costly, we present in this paper a novel system for improved PV diagnostics using drone-based imagery. Our solution consists of three main steps. The first step locates the solar panels within the image. The second step detects the anomalies within the solar panels. The final step identifies the root cause of the anomaly. In this paper, we mainly focus on the second step comprising the detection of anomalies within solar panels, which is done using a region-based convolutional neural network (CNN). Experiments on six different PV sites with different specifications and a variety of defects demonstrate that our anomaly detector achieves a true positive rate or recall of more than 90% for a false positive rate of around 2% to 3% tested on a dataset containing nearly 9000 solar panels. Compared to the best state-of-the-art methods, the experiments revealed that we achieve a slightly higher true positive rate for a substantially lower false positive rate, while tested on a more realistic dataset. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

27 pages, 17261 KiB

Open AccessArticle

Benchmarking Domain Adaptation Methods on Aerial Datasets

by Navya Nagananda, Abu Md Niamul Taufique, Raaga Madappa, Chowdhury Sadman Jahan, Breton Minnehan, Todd Rovito and Andreas Savakis

Sensors 2021, 21(23), 8070; https://0-doi-org.brum.beds.ac.uk/10.3390/s21238070 - 02 Dec 2021

Cited by 6 | Viewed by 2206

Abstract

Deep learning grew in importance in recent years due to its versatility and excellent performance on supervised classification tasks. A core assumption for such supervised approaches is that the training and testing data are drawn from the same underlying data distribution. This may [...] Read more.

Deep learning grew in importance in recent years due to its versatility and excellent performance on supervised classification tasks. A core assumption for such supervised approaches is that the training and testing data are drawn from the same underlying data distribution. This may not always be the case, and in such cases, the performance of the model is degraded. Domain adaptation aims to overcome the domain shift between the source domain used for training and the target domain data used for testing. Unsupervised domain adaptation deals with situations where the network is trained on labeled data from the source domain and unlabeled data from the target domain with the goal of performing well on the target domain data at the time of deployment. In this study, we overview seven state-of-the-art unsupervised domain adaptation models based on deep learning and benchmark their performance on three new domain adaptation datasets created from publicly available aerial datasets. We believe this is the first study on benchmarking domain adaptation methods for aerial data. In addition to reporting classification performance for the different domain adaptation models, we present t-SNE visualizations that illustrate the benefits of the adaptation process. Full article

(This article belongs to the Special Issue Deep Learning Methods for Aerial Imagery)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Deep Learning Methods for Aerial Imagery

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI