Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
7.9
5.0
Journals
Remote Sensing
Special Issues
Multisensor Data Fusion in Remote Sensing
remotesensing-logo
Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Multisensor Data Fusion in Remote Sensing

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (30 November 2018) | Viewed by 109895

Special Issue Editors

Prof. Dr. Paul Scheunders

E-Mail Website
Guest Editor
Vision Lab, University of Antwerp (CDE), Universiteitsplein 1 (N Building), B-2610 Antwerp, Belgium
Interests: remote sensing data analysis; hyperspectral image analysis; machine learning; spectral unmixing
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Xiaoxiang Zhu

E-Mail Website
Guest Editor
Signal Processing in Earth Observation, TUM, Department Head "EO Data Science", DLR, Germany
Interests: signal processing; remote sensing; Synthetic Aperture Radar; hyperspectral imaging
Special Issues, Collections and Topics in MDPI journals
Dr. Naoto Yokoya

grade E-Mail Website
Guest Editor
RIKEN Center for Advanced Intelligence Project, Tokyo 103-0027, Japan
Interests: remote sensing; image processing; data fusion; machine learning; disaster management; environmental monitoring
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Nowadays, a large variety of sensors are available to monitor the Earth surface. Data fusion aims at the combined use of data from multiple sensors, rather than analyzing data from each of these sensors individually. Multisensor data fusion has been an active topic of research for more than 2 decades and has been recently boosted by openly available big remote sensing data, e.g. offered by the ESA’s Copernicus program.

This Special Issue aims to report the latest advances and trends in the field of multisensor data fusion for remote sensing. Both theoretical and application-oriented papers are welcome. It will cover (but will not be limited to) the following topics:

  • Multisensor and multimodal data fusion using a variety of sensors such as optical imaging, SAR, and LiDAR
  • Fusion of remote sensing data and open geospatial data including StreetView images, open GIS data, and social media data
  • Multisensor image fusion for spatial resolution enhancement such as pan-sharpening, multi/hyperspectral image fusion, and downscaling of multiresolution imagery
  • Multisensor spatio-temporal data fusion
  • Matching and co-registration of multisource data
  • New developments in estimation theory and machine learning for data fusion
  • Multisensor data fusion for specific tasks such as classification, object recognition, change detection, and biophysical parameter estimation
  • Applications of multisensor data fusion
Prof. Paul Scheunders
Prof. Xiao Xiang Zhu
Dr. Naoto Yokoya
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Multisensor image fusion
  • Decision fusion
  • Pansharpening
  • Hyperspectral image fusion
  • Spatio-temporal data fusion
  • Spatial resolution enhancement
  • Machine learning for data fusion

Published Papers (16 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Review

20 pages, 1537 KiB  
Article
Decision Fusion Framework for Hyperspectral Image Classification Based on Markov and Conditional Random Fields
by Vera Andrejchenko, Wenzhi Liao, Wilfried Philips and Paul Scheunders
Remote Sens. 2019, 11(6), 624; https://0-doi-org.brum.beds.ac.uk/10.3390/rs11060624 - 14 Mar 2019
Cited by 16 | Viewed by 4001
Abstract
Classification of hyperspectral images is a challenging task owing to the high dimensionality of the data, limited ground truth data, collinearity of the spectra and the presence of mixed pixels. Conventional classification techniques do not cope well with these problems. Thus, in addition [...] Read more.
Classification of hyperspectral images is a challenging task owing to the high dimensionality of the data, limited ground truth data, collinearity of the spectra and the presence of mixed pixels. Conventional classification techniques do not cope well with these problems. Thus, in addition to the spectral information, features were developed for a more complete description of the pixels, e.g., containing contextual information at the superpixel level or mixed pixel information at the subpixel level. This has encouraged an evolution of fusion techniques which use these myriad of multiple feature sets and decisions from individual classifiers to be employed in a joint manner. In this work, we present a flexible decision fusion framework addressing these issues. In a first step, we propose to use sparse fractional abundances as decision source, complementary to class probabilities obtained from a supervised classifier. This specific selection of complementary decision sources enables the description of a pixel in a more complete way, and is expected to mitigate the effects of small training samples sizes. Secondly, we propose to apply a fusion scheme, based on the probabilistic graphical Markov Random Field (MRF) and Conditional Random Field (CRF) models, which inherently employ spatial information into the fusion process. To strengthen the decision fusion process, consistency links across the different decision sources are incorporated to encourage agreement between their decisions. The proposed framework offers flexibility such that it can be extended with additional decision sources in a straightforward way. Experimental results conducted on two real hyperspectral images show superiority over several other approaches in terms of classification performance when very limited training data is available. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

25 pages, 3882 KiB  
Article
SDF-MAN: Semi-Supervised Disparity Fusion with Multi-Scale Adversarial Networks
by Can Pu, Runzi Song, Radim Tylecek, Nanbo Li and Robert B. Fisher
Remote Sens. 2019, 11(5), 487; https://0-doi-org.brum.beds.ac.uk/10.3390/rs11050487 - 27 Feb 2019
Cited by 6 | Viewed by 4193
Abstract
Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of [...] Read more.
Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of depth data. In this paper, we introduce a new method to fuse disparity maps from different sources, while incorporating supplementary information (intensity, gradient, etc.) into a refiner network to better refine raw disparity inputs. A discriminator network classifies disparities at different receptive fields and scales. Assuming a Markov Random Field for the refined disparity map produces better estimates of the true disparity distribution. Both fully supervised and semi-supervised versions of the algorithm are proposed. The approach includes a more robust loss function to inpaint invalid disparity values and requires much less labeled data to train in the semi-supervised learning mode. The algorithm can be generalized to fuse depths from different kinds of depth sources. Experiments explored different fusion opportunities: stereo-monocular fusion, stereo-ToF fusion and stereo-stereo fusion. The experiments show the superiority of the proposed algorithm compared with the most recent algorithms on public synthetic datasets (Scene Flow, SYNTH3, our synthetic garden dataset) and real datasets (Kitti2015 dataset and Trimbot2020 Garden dataset). Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

29 pages, 11763 KiB  
Article
A Markovian Approach to Unsupervised Change Detection with Multiresolution and Multimodality SAR Data
by David Solarna, Gabriele Moser and Sebastiano B. Serpico
Remote Sens. 2018, 10(11), 1671; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10111671 - 23 Oct 2018
Cited by 7 | Viewed by 2963
Abstract
In the framework of synthetic aperture radar (SAR) systems, current satellite missions make it possible to acquire images at very high and multiple spatial resolutions with short revisit times. This scenario conveys a remarkable potential in applications to, for instance, environmental monitoring and [...] Read more.
In the framework of synthetic aperture radar (SAR) systems, current satellite missions make it possible to acquire images at very high and multiple spatial resolutions with short revisit times. This scenario conveys a remarkable potential in applications to, for instance, environmental monitoring and natural disaster recovery. In this context, data fusion and change detection methodologies play major roles. This paper proposes an unsupervised change detection algorithm for the challenging case of multimodal SAR data collected by sensors operating at multiple spatial resolutions. The method is based on Markovian probabilistic graphical models, graph cuts, linear mixtures, generalized Gaussian distributions, Gram–Charlier approximations, maximum likelihood and minimum mean squared error estimation. It benefits from the SAR images acquired at multiple spatial resolutions and with possibly different modalities on the considered acquisition times to generate an output change map at the finest observed resolution. This is accomplished by modeling the statistics of the data at the various spatial scales through appropriate generalized Gaussian distributions and by iteratively estimating a set of virtual images that are defined on the pixel grid at the finest resolution and would be collected if all the sensors could work at that resolution. A Markov random field framework is adopted to address the detection problem by defining an appropriate multimodal energy function that is minimized using graph cuts. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

14 pages, 2829 KiB  
Article
Feature Importance Analysis for Local Climate Zone Classification Using a Residual Convolutional Neural Network with Multi-Source Datasets
by Chunping Qiu, Michael Schmitt, Lichao Mou, Pedram Ghamisi and Xiao Xiang Zhu
Remote Sens. 2018, 10(10), 1572; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10101572 - 01 Oct 2018
Cited by 63 | Viewed by 7000
Abstract
Global Local Climate Zone (LCZ) maps, indicating urban structures and land use, are crucial for Urban Heat Island (UHI) studies and also as starting points to better understand the spatio-temporal dynamics of cities worldwide. However, reliable LCZ maps are not available on a [...] Read more.
Global Local Climate Zone (LCZ) maps, indicating urban structures and land use, are crucial for Urban Heat Island (UHI) studies and also as starting points to better understand the spatio-temporal dynamics of cities worldwide. However, reliable LCZ maps are not available on a global scale, hindering scientific progress across a range of disciplines that study the functionality of sustainable cities. As a first step towards large-scale LCZ mapping, this paper tries to provide guidance about data/feature choice. To this end, we evaluate the spectral reflectance and spectral indices of the globally available Sentinel-2 and Landsat-8 imagery, as well as the Global Urban Footprint (GUF) dataset, the OpenStreetMap layers buildings and land use and the Visible Infrared Imager Radiometer Suite (VIIRS)-based Nighttime Light (NTL) data, regarding their relevance for discriminating different Local Climate Zones (LCZs). Using a Residual convolutional neural Network (ResNet), a systematic analysis of feature importance is performed with a manually-labeled dataset containing nine cities located in Europe. Based on the investigation of the data and feature choice, we propose a framework to fully exploit the available datasets. The results show that GUF, OSM and NTL can contribute to the classification accuracy of some LCZs with relatively few samples, and it is suggested that Landsat-8 and Sentinel-2 spectral reflectances should be jointly used, for example in a majority voting manner, as proven by the improvement from the proposed framework, for large-scale LCZ mapping. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

22 pages, 6202 KiB  
Article
A Prediction Smooth Method for Blending Landsat and Moderate Resolution Imagine Spectroradiometer Images
by Detang Zhong and Fuqun Zhou
Remote Sens. 2018, 10(9), 1371; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10091371 - 29 Aug 2018
Cited by 17 | Viewed by 4837
Abstract
Landsat images have been widely used in support of responsible development of natural resources, disaster risk management (e.g., forest fire, flooding etc.), agricultural production monitoring, as well as environmental change studies due to its medium spatial resolution and rich spectral information. However, its [...] Read more.
Landsat images have been widely used in support of responsible development of natural resources, disaster risk management (e.g., forest fire, flooding etc.), agricultural production monitoring, as well as environmental change studies due to its medium spatial resolution and rich spectral information. However, its availability and usability are largely constrained by its low revisit frequency. On the other hand, MODIS (Moderate Resolution Imaging Spectroradiometer) images for land studies have much more frequent coverage but with a lower spatial resolution of 250–500 m. To take advantages of the two sensors and expand their availability and usability, during the last decade, a number of image fusion methods have been developed for generating Landsat-like images from MODIS observations to supplement clear-sky Landsat imagery. However, available methods are typically effective or applicable for certain applications. For a better result, a new Prediction Smooth Reflectance Fusion Model (PSRFM) for blending Landsat and MODIS images is proposed. PSRFM consists of a dynamic prediction model and a smoothing filter. The dynamic prediction model generates synthetic Landsat images from a pair of Landsat and MODIS images and another MODIS image, either forward or backward in time. The smoothing filter combines the forward and backward predictions by weighted average based on elapsed time or on the estimated prediction uncertainty. Optionally, the smooth filtering can be applied with constraints based on Normalized Difference Snow Index (NDSI) or Normalized Difference Vegetation Index (NDVI). In comparison to some published reflectance fusion methods, PSRFM shows the following desirable characteristics: (1) it can deal with one pair or two pairs of Landsat and MODIS images; (2) it can incorporate input image uncertainty during prediction and estimate prediction uncertainty; (3) it can track gradual vegetation phenological changes and deal with abrupt land-cover type changes; and (4) for predictions using two pairs of input images, the results can be further improved through the constrained smoothing filter based on NDSI or NDVI for certain applications. We tested PSRFM to generate a Landsat-like image time series by using Landsat 8 OLI and MODIS (MOD09GA) images and compared it to two reflectance fusion algorithms: STARFM (Spatial and Temporal Adaptive Reflectance Fusion Model) and ESTARFM (Enhanced version of STARFM). The results show that the proposed PSRFM is effective and outperforms STARFM and ESTARFM both visually and quantitatively. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

28 pages, 10396 KiB  
Article
HOMPC: A Local Feature Descriptor Based on the Combination of Magnitude and Phase Congruency Information for Multi-Sensor Remote Sensing Images
by Zhitao Fu, Qianqing Qin, Bin Luo, Hong Sun and Chun Wu
Remote Sens. 2018, 10(8), 1234; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10081234 - 06 Aug 2018
Cited by 24 | Viewed by 4734
Abstract
Local region description of multi-sensor images remains a challenging task in remote sensing image analysis and applications due to the non-linear radiation variations between images. This paper presents a novel descriptor based on the combination of the magnitude and phase congruency information of [...] Read more.
Local region description of multi-sensor images remains a challenging task in remote sensing image analysis and applications due to the non-linear radiation variations between images. This paper presents a novel descriptor based on the combination of the magnitude and phase congruency information of local regions to capture the common features of images with non-linear radiation changes. We first propose oriented phase congruency maps (PCMs) and oriented magnitude binary maps (MBMs) using the multi-oriented phase congruency and magnitude information of log-Gabor filters. The two feature vectors are then quickly constructed based on the convolved PCMs and MBMs. Finally, a dense descriptor named the histograms of oriented magnitude and phase congruency (HOMPC) is developed by combining the histograms of oriented phase congruency (HPC) and the histograms of oriented magnitude (HOM) to capture the structure and shape properties of local regions. HOMPC was evaluated with three datasets composed of multi-sensor remote sensing images obtained from unmanned ground vehicle, unmanned aerial vehicle, and satellite platforms. The descriptor performance was evaluated by recall, precision, F1-measure, and area under the precision-recall curve. The experimental results showed the advantages of the HOM and HPC combination and confirmed that HOMPC is far superior to the current state-of-the-art local feature descriptors. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

19 pages, 7789 KiB  
Article
Matching Multi-Sensor Remote Sensing Images via an Affinity Tensor
by Shiyu Chen, Xiuxiao Yuan, Wei Yuan, Jiqiang Niu, Feng Xu and Yong Zhang
Remote Sens. 2018, 10(7), 1104; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10071104 - 11 Jul 2018
Cited by 8 | Viewed by 4921
Abstract
Matching multi-sensor remote sensing images is still a challenging task due to textural changes and non-linear intensity differences. In this paper, a novel matching method is proposed for multi-sensor remote sensing images. To establish feature correspondences, an affinity tensor is used to integrate [...] Read more.
Matching multi-sensor remote sensing images is still a challenging task due to textural changes and non-linear intensity differences. In this paper, a novel matching method is proposed for multi-sensor remote sensing images. To establish feature correspondences, an affinity tensor is used to integrate geometric and radiometric information. The matching process consists of three steps. First, features from an accelerated segment test are extracted from both source and target images, and two complete graphs are constructed with their nodes representing these features. Then, the geometric and radiometric similarities of the feature points are represented by the three-order affinity tensor, and the initial feature correspondences are established by tensor power iteration. Finally, a tensor-based mismatch detection process is conducted to purify the initial matched points. The robustness and capability of the proposed method are tested with a variety of remote sensing images such as Ziyuan-3 backward, Ziyuan-3 nadir, Gaofen-1, Gaofen-2, unmanned aerial vehicle platform, and Jilin-1. The experiments show that the average matching recall is greater than 0.5, which outperforms state-of-the-art multi-sensor image-matching algorithms such as SIFT, SURF, NG-SIFT, OR-SIFT and GOM-SIFT. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

22 pages, 5314 KiB  
Article
Combining Linear Pixel Unmixing and STARFM for Spatiotemporal Fusion of Gaofen-1 Wide Field of View Imagery and MODIS Imagery
by Jintian Cui, Xin Zhang and Muying Luo
Remote Sens. 2018, 10(7), 1047; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10071047 - 03 Jul 2018
Cited by 22 | Viewed by 4185
Abstract
Spatiotemporal fusion of remote sensing data is essential for generating high spatial and temporal resolution data by taking advantage of high spatial resolution and high temporal resolution imageries. At present, the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is one of the [...] Read more.
Spatiotemporal fusion of remote sensing data is essential for generating high spatial and temporal resolution data by taking advantage of high spatial resolution and high temporal resolution imageries. At present, the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is one of the most widely used spatiotemporal fusion technologies of remote sensing data. However, the quality of data acquired by STARFM depends on temporal information from homogeneous land cover patches at the MODIS (Moderate Resolution Imaging Spectroradiometer) imagery, and the estimation accuracy of STARFM degrades in highly fragmentated and heterogeneous patches. To address this problem, we developed an innovative method to improve fusion accuracy, especially in areas of high heterogeneity, by combining linear pixel unmixing and STARFM. This method derived the input data of STARFM by downscaling the MODIS data with a linear spectral mixture model. Through this fusion method, the complement effect of the advantages of remote sensing information can be realized, and the multi-source remote sensing data can be realized for visual data mining. The developed fusion method was applied in Bosten Lake, the largest freshwater lake in China, and our analysis of results suggests that (1) after introducing the linear spectral mixture model, the fusion images illustrated improved spatial details to a certain extent and can be employed to identify small objects, as well as their texture distribution information; (2) for fragmented and highly heterogeneous areas, a stronger correlation between the predicted results and the real images was observed when compared to STARFM with small bias; and (3) the predicted red band and near infrared band can generate high-precision 16-m NDVI (Normalized Difference Vegetation Index) data with advantages in both spatial resolution and temporal resolution. The results are generally consistent with the Gaofen-1 wide field of view cameras (GF-1 WFV) NDVI in the same period and therefore can reflect the spatial distribution of NDVI in detail. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

18 pages, 4574 KiB  
Article
An Enhanced Linear Spatio-Temporal Fusion Method for Blending Landsat and MODIS Data to Synthesize Landsat-Like Imagery
by Bo Ping, Yunshan Meng and Fenzhen Su
Remote Sens. 2018, 10(6), 881; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10060881 - 05 Jun 2018
Cited by 19 | Viewed by 3800
Abstract
Landsat and MODIS data have been widely utilized in many remote sensing applications, however, the trade-off between the spatial resolution and temporal frequency has limited their capacities in monitoring detailed spatio-temporal dynamics. Spatio-temporal fusion methods based on a linear model that considers the [...] Read more.
Landsat and MODIS data have been widely utilized in many remote sensing applications, however, the trade-off between the spatial resolution and temporal frequency has limited their capacities in monitoring detailed spatio-temporal dynamics. Spatio-temporal fusion methods based on a linear model that considers the differences between fine- and coarse-spatial-resolution images as linear can effectively solve this trade-off problem, yet the existing linear fusion methods either regard the coefficients of the linear model as constants or have adopted regression methods to calculate the coefficients, both of which may introduce some errors in the fusion process. In this paper, we proposed an enhanced linear spatio-temporal fusion method (ELSTFM) to improve the data fusion accuracy. In the ELSTFM, it is not necessary to calculate the slope of the linear model, and the intercept, which can be deemed as the residual caused by systematic biases, is calculated based on spectral unmixing theory. Additionally, spectrally similar pixels in a given fine-spatial-resolution pixel’s neighborhood and their corresponding weights were used in the proposed method to mitigate block effects. Landsat-7/ETM+ and 8-day composite MODIS reflectance data covering two study sites with heterogeneous and homogenous landscapes were selected to validate the proposed method. Compared to three other typical spatio-temporal fusion methods visually and quantitatively, the predicted images obtained from ELSTFM could acquire better results for the two selected study sites. Furthermore, the resampling methods used to resample MODIS to the same spatial resolution of Landsat could slightly, but did not significantly influence the fusion accuracy, and the distributions of slopes of different bands for the two study sites could all be deemed as normal distributions with a mean value close to 1. The performance of ELSTFM depends on the accuracy of residual calculation at fine-resolution and large landscape changes may influence the fusion accuracy. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

23 pages, 41039 KiB  
Article
Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network
by Jingxiang Yang, Yong-Qiang Zhao and Jonathan Cheung-Wai Chan
Remote Sens. 2018, 10(5), 800; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10050800 - 21 May 2018
Cited by 148 | Viewed by 10707
Abstract
Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, [...] Read more.
Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

18 pages, 6142 KiB  
Article
A Modified Spatiotemporal Fusion Algorithm Using Phenological Information for Predicting Reflectance of Paddy Rice in Southern China
by Mengxue Liu, Xiangnan Liu, Ling Wu, Xinyu Zou, Tian Jiang and Bingyu Zhao
Remote Sens. 2018, 10(5), 772; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10050772 - 17 May 2018
Cited by 27 | Viewed by 3916
Abstract
Satellite data for studying surface dynamics in heterogeneous landscapes are missing due to frequent cloud contamination, low temporal resolution, and technological difficulties in developing satellites. A modified spatiotemporal fusion algorithm for predicting the reflectance of paddy rice is presented in this paper. The [...] Read more.
Satellite data for studying surface dynamics in heterogeneous landscapes are missing due to frequent cloud contamination, low temporal resolution, and technological difficulties in developing satellites. A modified spatiotemporal fusion algorithm for predicting the reflectance of paddy rice is presented in this paper. The algorithm uses phenological information extracted from a moderate-resolution imaging spectroradiometer enhanced vegetation index time series to improve the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM). The algorithm is tested with satellite data on Yueyang City, China. The main contribution of the modified algorithm is the selection of similar neighborhood pixels by using phenological information to improve accuracy. Results show that the modified algorithm performs better than ESTARFM in visual inspection and quantitative metrics, especially for paddy rice. This modified algorithm provides not only new ideas for the improvement of spatiotemporal data fusion method, but also technical support for the generation of remote sensing data with high spatial and temporal resolution. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

24 pages, 14557 KiB  
Article
Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network
by Xuran Pan, Lianru Gao, Andrea Marinoni, Bing Zhang, Fan Yang and Paolo Gamba
Remote Sens. 2018, 10(5), 743; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10050743 - 11 May 2018
Cited by 55 | Viewed by 6076
Abstract
In this paper, a novel convolutional neural network (CNN)-based architecture, named fine segmentation network (FSN), is proposed for semantic segmentation of high resolution aerial images and light detection and ranging (LiDAR) data. The proposed architecture follows the encoder–decoder paradigm and the multi-sensor fusion [...] Read more.
In this paper, a novel convolutional neural network (CNN)-based architecture, named fine segmentation network (FSN), is proposed for semantic segmentation of high resolution aerial images and light detection and ranging (LiDAR) data. The proposed architecture follows the encoder–decoder paradigm and the multi-sensor fusion is accomplished in the feature-level using multi-layer perceptron (MLP). The encoder consists of two parts: the main encoder based on the convolutional layers of Vgg-16 network for color-infrared images and a lightweight branch for LiDAR data. In the decoder stage, to adaptively upscale the coarse outputs from encoder, the Sub-Pixel convolution layers replace the transposed convolutional layers or other common up-sampling layers. Based on this design, the features from different stages and sensors are integrated for a MLP-based high-level learning. In the training phase, transfer learning is employed to infer the features learned from generic dataset to remote sensing data. The proposed FSN is evaluated by using the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen 2D Semantic Labeling datasets. Experimental results demonstrate that the proposed framework can bring considerable improvement to other related networks. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

20 pages, 3596 KiB  
Article
Automatic Discovery and Geotagging of Objects from Street View Imagery
by Vladimir A. Krylov, Eamonn Kenny and Rozenn Dahyot
Remote Sens. 2018, 10(5), 661; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10050661 - 24 Apr 2018
Cited by 56 | Viewed by 12914
Abstract
Many applications, such as autonomous navigation, urban planning, and asset monitoring, rely on the availability of accurate information about objects and their geolocations. In this paper, we propose the automatic detection and computation of the coordinates of recurring stationary objects of interest using [...] Read more.
Many applications, such as autonomous navigation, urban planning, and asset monitoring, rely on the availability of accurate information about objects and their geolocations. In this paper, we propose the automatic detection and computation of the coordinates of recurring stationary objects of interest using street view imagery. Our processing pipeline relies on two fully convolutional neural networks: the first segments objects in the images, while the second estimates their distance from the camera. To geolocate all the detected objects coherently we propose a novel custom Markov random field model to estimate the objects’ geolocation. The novelty of the resulting pipeline is the combined use of monocular depth estimation and triangulation to enable automatic mapping of complex scenes with the simultaneous presence of multiple, visually similar objects of interest. We validate experimentally the effectiveness of our approach on two object classes: traffic lights and telegraph poles. The experiments report high object recall rates and position precision of approximately 2 m, which is approaching the precision of single-frequency GPS receivers. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

21 pages, 22099 KiB  
Article
Urban Land Use Mapping by Combining Remote Sensing Imagery and Mobile Phone Positioning Data
by Yuanxin Jia, Yong Ge, Feng Ling, Xian Guo, Jianghao Wang, Le Wang, Yuehong Chen and Xiaodong Li
Remote Sens. 2018, 10(3), 446; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10030446 - 12 Mar 2018
Cited by 60 | Viewed by 10581
Abstract
Land use is of great importance for urban planning, environmental monitoring, and transportation management. Several methods have been proposed to obtain land use maps of urban areas, and these can be classified into two categories: remote sensing methods and social sensing methods. However, [...] Read more.
Land use is of great importance for urban planning, environmental monitoring, and transportation management. Several methods have been proposed to obtain land use maps of urban areas, and these can be classified into two categories: remote sensing methods and social sensing methods. However, remote sensing and social sensing approaches have specific disadvantages regarding the description of social and physical features, respectively. Therefore, an appropriate fusion strategy is vital for large-area land use mapping. To address this issue, we propose an efficient land use mapping method that combines remote sensing imagery (RSI) and mobile phone positioning data (MPPD) for large areas. We implemented this method in two steps. First, a support vector machine was adopted to classify the RSI and MPPD. Then, the two classification results were fused using a decision fusion strategy to generate the land use map. The proposed method was applied to a case study of the central area of Beijing. The experimental results show that the proposed method improved classification accuracy compared with that achieved using MPPD alone, validating the efficacy of this new approach for identifying land use. Based on the land use map and MPPD data, activity density in key zones during daytime and nighttime was analyzed to illustrate the volume and variation of people working and living across different regions. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

23 pages, 20794 KiB  
Article
Matching of Remote Sensing Images with Complex Background Variations via Siamese Convolutional Neural Network
by Haiqing He, Min Chen, Ting Chen and Dajun Li
Remote Sens. 2018, 10(2), 355; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10020355 - 24 Feb 2018
Cited by 70 | Viewed by 7380
Abstract
Feature-based matching methods have been widely used in remote sensing image matching given their capability to achieve excellent performance despite image geometric and radiometric distortions. However, most of the feature-based methods are unreliable for complex background variations, because the gradient or other image [...] Read more.
Feature-based matching methods have been widely used in remote sensing image matching given their capability to achieve excellent performance despite image geometric and radiometric distortions. However, most of the feature-based methods are unreliable for complex background variations, because the gradient or other image grayscale information used to construct the feature descriptor is sensitive to image background variations. Recently, deep learning-based methods have been proven suitable for high-level feature representation and comparison in image matching. Inspired by the progresses made in deep learning, a new technical framework for remote sensing image matching based on the Siamese convolutional neural network is presented in this paper. First, a Siamese-type network architecture is designed to simultaneously learn the features and the corresponding similarity metric from labeled training examples of matching and non-matching true-color patch pairs. In the proposed network, two streams of convolutional and pooling layers sharing identical weights are arranged without the manually designed features. The number of convolutional layers is determined based on the factors that affect image matching. The sigmoid function is employed to compute the matching and non-matching probabilities in the output layer. Second, a gridding sub-pixel Harris algorithm is used to obtain the accurate localization of candidate matches. Third, a Gaussian pyramid coupling quadtree is adopted to gradually narrow down the searching space of the candidate matches, and multiscale patches are compared synchronously. Subsequently, a similarity measure based on the output of the sigmoid is adopted to find the initial matches. Finally, the random sample consensus algorithm and the whole-to-local quadratic polynomial constraints are used to remove false matches. In the experiments, different types of satellite datasets, such as ZY3, GF1, IKONOS, and Google Earth images, with complex background variations are used to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed method, which can significantly improve the matching performance of multi-temporal remote sensing images with complex background variations, is better than the state-of-the-art matching methods. In our experiments, the proposed method obtained a large number of evenly distributed matches (at least 10 times more than other methods) and achieved a high accuracy (less than 1 pixel in terms of root mean square error). Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Graphical abstract

Review

Jump to: Research

28 pages, 35856 KiB  
Review
True Orthophoto Generation from Aerial Frame Images and LiDAR Data: An Update
by Hamid Gharibi and Ayman Habib
Remote Sens. 2018, 10(4), 581; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10040581 - 09 Apr 2018
Cited by 20 | Viewed by 13560
Abstract
Image spectral and Light Detection and Ranging (LiDAR) positional information can be related through the orthophoto generation process. Orthophotos have a uniform scale and represent all objects in their correct planimetric locations. However, orthophotos generated using conventional methods suffer from an artifact known [...] Read more.
Image spectral and Light Detection and Ranging (LiDAR) positional information can be related through the orthophoto generation process. Orthophotos have a uniform scale and represent all objects in their correct planimetric locations. However, orthophotos generated using conventional methods suffer from an artifact known as the double-mapping effect that occurs in areas occluded by tall objects. The double-mapping problem can be resolved through the commonly known true orthophoto generation procedure, in which an occlusion detection process is incorporated. This paper presents a review of occlusion detection methods, from which three techniques are compared and analyzed using experimental results. The paper also describes a framework for true orthophoto production based on an angle-based occlusion detection method. To improve the performance of the angle-based technique, two modifications to this method are introduced. These modifications, which aim at resolving false visibilities reported within the angle-based occlusion detection process, are referred to as occlusion extension and radial section overlap. A weighted averaging approach is also proposed to mitigate the seamline effect and spectral dissimilarity that may appear in true orthophoto mosaics. Moreover, true orthophotos generated from high-resolution aerial images and high-density LiDAR data using the updated version of angle-based methodology are illustrated for two urban study areas. To investigate the potential of image matching techniques in producing true orthophotos and point clouds, a comparison between the LiDAR-based and image-matching-based true orthophotos and digital surface models (DSMs) for an urban study area is also presented in this paper. Among the investigated occlusion detection methods, the angle-based technique demonstrated a better performance in terms of output and running time. The LiDAR-based true orthophotos and DSMs showed higher qualities compared to their image-matching-based counterparts which contain artifacts/noise along building edges. Full article
(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-16
Back to TopTop