Depending on the number of acquired bands, remote sensing imaging technology has developed from collecting panchromatic (PAN) and color images to multispectral (MS) images, and it can now capture hyperspectral (HS) images with dozens of hundreds of bands. A PAN image with very high spatial resolution is a single-band grayscale image acquired in the visible range. It is able to obtain the shape feature of objects, but cannot distinguish colors. A color image consists of three bands which are red, green and blue, and displays the colors of objects. However, it is difficult to distinguish the features in similar colors. An MS image not only obtains spatial features, but also obtains spectral information in several bands, which is more capable of distinguishing categories of different features. However, the rough spectral resolution of MS images may not meet the requirements in some applications, and it is hard to realize fine feature detection [1
]. An HS image with a higher spectral resolution on the order of nanometers can provide finer classification [2
], which has been applied to many fields [3
] and some practical applications, such as vegetation study [8
], precision agriculture [8
], regional geological mapping [9
], mineral exploration [10
], and environment monitoring [11
]. Due to technical limitations, the spatial resolution of an HS image is low.
As both high spatial and spectral resolutions are important in practical applications, obtaining a high spatial resolution HS (HRHS) image is crucial. One effective way is to perform hyperspectral pansharpening, which fuses a high spatial resolution PAN (HRPAN) image with a low spatial resolution HS (LRHS) image. Figure 1
shows the concept of hyperspectral pansharpening.
Many hyperspectral pansharpening algorithms were developed, among which hyperspectral pansharpening methods using Bayesian and matrix factorization have been proposed in recent years. The Bayesian-based approaches include Bayesian naive Gaussian prior [12
], Bayesian sparsity promoted Gaussian prior [13
], and HySure [14
]. These algorithms utilize the posterior distribution, and are based on maximum a posteriori estimation to fuse LRHS and HRPAN images [15
]. The matrix factorization approach generates a fused HRHS image by using the nonnegative matrix factorization (NMF) under some constraints to estimate endmember and abundance matrices [16
]. The matrix factorization approach is well represented by the nonnegative sparse coding (NNSC) [17
] and constrained nonnegative matrix factorization (CNMF) [18
] methods. The main challenge in hyperspectral pansharpening is to effectively improve the spatial resolution while preserving the original spectral information. The Bayesian and matrix factorization approaches are able to achieve good results on this challenge, but have a high computational cost.
Component substitution (CS) and multi-resolution analysis (MRA) approaches are two classical hyperspectral pansharpening approaches which have simple and fast implementation. For the CS class, intensity-hue-saturation (IHS) transform [19
], principal component analysis (PCA) transform [21
], Gram–Schmidt (GS) [23
], and adaptive GS (GSA) [24
] are the most representative methods. The CS class extracts spatial details of the HS image, and replaces the extracted spatial details with the HRPAN image. Regardless of superior spatial performance, the CS class suffers from serious spectral distortion [25
]. The typical algorithms of the MRA technique are smoothing filter based intensity modulation (SFIM) [26
], Laplacian pyramid [27
], modulation transfer function generalized Laplacian pyramid (MTF-GLP) [28
], and MTF-GLP with high pass modulation (MTF-GLP-HPM) [29
]. The MRA methods generally utilize a multi-resolution decomposition to extract spatial details which are imported into the HS image. Compared with the CS methods, the MRA methods generate less spectral distortion, but usually have a larger computational burden [30
]. Recently, several algorithms based on the CS and MRA approaches have been proposed, such as the Sentinel-2A CS and MRA based sharpening algorithm [31
], the multiband Filter estimation (MBFE) algorithm [32
], and the guided filter PCA (GFPCA) algorithm [33
]. Moreover, several intelligent processing-based methods have also been proposed, and examples include deep two-branches convolutional neural network (Two-CNN-Fu) [34
], Bidirectional Pyramid Network [35
], and 3D-convolutional neural network (3D-CNN) [36
The CS and MRA approaches mostly extract the spatial information of the HRPAN image and inject it into the LRHS image, but without considering the spatial information of the LRHS image. Due to the incomplete spatial information injection, the CS and MRA approaches may result in distortion. To address this problem, we propose a novel hyperspectral pansharpening method by combining homomorphic filtering with a weighted tensor matrix. An optimized weighted tensor matrix-based method which considers the structure information of the LRHS and HRPAN images is proposed to generate more comprehensive spatial information. In addition, to extract the spatial structure information of the LRHS images, open-closing morphological operation is first used for noise removal, and homomorphic filtering is then introduced to extract the spatial details of each band. Finally, a weighted root mean squared error based method is proposed to obtain the total spatial component of the LRHS image from extracted spatial details of each band, and the Laplacian pyramid networks super-resolution algorithm is adopted to enhance the spatial resolution of the obtained spatial component. Comparative analysis was used to demonstrate the applicability and superiority of the proposed method in both spectral and spatial qualities.
As stated above, a new hyperspectral pansharpening method based on homomorphic filtering and weighted tensor matrix is proposed in this paper. The main novelties of the proposed hyperspectral pansharpening method are concluded in the following aspects.
A novel HS image spatial component extraction strategy is proposed. Open-closing morphological operation and homomorphic filtering are first introduced to remove the noise and extract the spatial details of each band of the HS image, respectively. Then, a weighted root mean squared error-based method is proposed to obtain the total spatial component of the HS image.
An optimized weighted tensor matrix-based method is proposed to integrate the spatial component of the HS image with the spatial component of the PAN image. The weighted structure tensor matrix that represents the structural information of multiple images is applied to hyperspectral pansharpening for the first time. The classical methods which mostly extract the spatial information of the PAN image inject the incomplete spatial information, and may lead to distortion. Unlike there classical methods, the proposed optimized weighted tensor matrix-based method generates the spatial information not only from the PAN image but also from the HS image, and can reduce the distortion caused by the insufficient spatial information.
The remainder of this paper is organized as follows. Section 2
describes the weighted structure tensor matrix and homomorphic filtering. In Section 3
, the proposed homomorphic filtering and weighted tensor matrix-based hyperspectral pansharpening algorithm is presented. Experimental results and discussion are provided in Section 4
, and conclusions are drawn in Section 5