An Infrared and Visible Fusion Framework Based on a Novel Decomposition Method

Abstract

Image fusion is one of the most rapidly evolving fields in image processing today, and its applications are widely expanded in various fields. In the field of image fusion, the method based on multi-scale decomposition plays an important role. However, it faces many difficult puzzles, such as the risk of over-smoothing during decomposition, blurring of fusion results, and loss of details. Aiming at these problems, this paper proposes a novel decomposition-based image fusion framework, which overcomes the problems of noise, blurring, and loss of details. Both the symmetry and asymmetry between infrared and visible images are important research hotspots in this paper. The experiments confirmed that the fusion framework outperforms other methods in both subjective observation and objective evaluation.

1. Introduction

Image fusion integrates the information of different spectral bands in the same scene. The sensors of infrared (IR) imaging can capture the thermal radiation emitted by objects and are extremely sensitive to thermal targets. However, IR images miss background texture details. The visible (VI) image sensors can obtain rich scene information, and they will be interfered with by illumination conditions, fog, occlusion, and so on. The fusion of infrared and visible images has received more and more attention to retain the target and enhance the detail and other useful information for extending the ability of visual perception [1,2,3].

There is a number of image fusion methods proposed for IR and VI images. Existing fusion methods can be classified into four categories: multi-scale transform (MST) [4,5,6,7,8,9,10,11,12,13,14,15], deep learning (DL) [16,17,18,19], sparse representation (SR) [20,21,22], and other methods [23,24,25]. In recent years, deep learning has proved superior and successful in many applications. However, there are some key issues unsolved. One is the complexity of the algorithm. DL needs strong hardware support and is difficult to deploy on embedded platforms. In particular, the training of DL needs a large number of registered image pairs, and the data set is difficult to obtain. Considering the inherent sparse representation of the human visual system, sparse representation methods have also been widely developed [20]. Other methods include total variation [15], principal components analysis (PCA) and robust PCA (RPCA) [24,25], intensity-hue-saturation (IHS) transform [26], etc. As a popular image fusion method, the multi-scale fusion theory (MST) has been successfully used in various application scenarios. The large-scale structure of an image usually contains background information and large size targets, while the small-scale structure usually represents details and texture information. A large number of algorithms have been widely proposed, such as the Laplacian pyramid (LAP) [4], discrete wavelet decomposition (DWT) [6,7], dual-tree complex wavelet transform (DTCWT) [8,9], non-subsampled contourlet transform (NSCT) [10], curvelet transform (CVT) [11], multi-resolution singular value decomposition (MSVD) [12], etc. However, these decomposition-based fusion methods have problems such as detail blurring due to excessive smoothing.

To tackle the above challenge, guided filtering (GF) was proposed by Li et al. for their image fusion framework [13]. Especially in recent years, various edge-preserving filtering methods have shown promising applications in image fusion [14,27,28,29,30,31]. For example, Ma et al. [28] used rolling guided filters to construct multi-scale images, which can separate scale information better in scale space and make the fused image have a good visual effect. Although these methods can improve the fusion results, they cannot reach the right scale separation for image features and often lose some salient information of the source image. In addition, they are still seriously troubled by low contrast and blurred details.

In the traditional filtering process, the filtering window always slides on the image. If the edge in an image is just in the center of a filter window, the filtering outputs a weighted average value of the filter window. However, the edge of the filtering operation on the image may be blurred. To avoid the problem, Yin et al. [29] proposed the method called side window filtering (SWF). Yan and Li [30] first used SWF to improve the two linear filters and applied it to the decomposition stage in image fusion, which proved the great potential of SWF. Subsequently, Yin et al. [31] applied the SWF technology to improve the guided filter called side window guided filter (SWGF), which has better edge protection performance than the improved linear filter. In this paper, the good edge-preserving property of SWF technology has been used to guide the process of image fusion.

In general, the filter based on the MST image fusion method consists of three steps. (1) The scale space is constructed to obtain the detail components with high-frequency information and the base components with low-frequency information; (2) According to the characteristics of the corresponding scale components, the fusion strategy is formulated; (3) The fusion image is reconstructed. After multi-scale decomposition, the base components mainly describe the energy distribution of the source image, without details and noise. Detail components mainly describe texture information, including detail distribution and noise. However, IR images are characterized by high contrast, strong edges, and a lack of detailed information. A VI image is characterized by uniform brightness and contains a lot of detailed information. Therefore, it is crucial to choose the right integration strategy. Most of the base components adopt simple “average” fusion rules. This rule is fast and effective, but it ignores the difference in brightness between the two bands of images.

This paper proposes a novel fusion scheme based on the SWGF, which can selectively retain the details of interest parts and effectively suppress noise. The common issue between infrared and visible images is the symmetry of visual quality for implementing practical applications. Furthermore, the asymmetry between infrared and visible images can produce complementarity to improve visual quality and system performance. Both the symmetry and asymmetry between infrared and visible images are important research hotspots in this paper. The main contributions of our work are as follows: (1) We propose a novel decomposition method using the SWF-Gaussian filter. The saliency information can be retained to the maximum extent during the image fusion process, and the halo artifact near the firm edges can be reduced. (2) We develop a brand-new framework for IR and VI image fusion. Moreover, regularization technology is employed to balance the degree of fusion.

The rest of the paper is organized as follows: Section 2 introduces the new fusion framework based on SWF technology in detail. In Section 3, the subjective and objective experiments are compared. Section 4 concludes the whole paper.

2. Materials and Methods

In this paper, we propose a new decomposition method to obtain the scale feature information of infrared and visible light images by scale decomposition, and then a new fusion framework is proposed to well reconstruct this scale feature information in the fused image. Figure 1 shows the flowchart of the proposed new fusion framework.

It is proposed to decompose the information contained in the source image into basic components and detail components, which correspond to the high-frequency and low-frequency information of the source image, respectively, as shown in Figure 2, Figure 3, Figure 4 and Figure 5. It is also demonstrated that the benefits of having fewer artifacts and halos in detail components. The guiding strategy of multi-strategy fusion is to preserve and balance the basic components of infrared and visible light images as much as possible, and to highlight the detail components in both. The purpose of regularization and nonlinearity is to remap the gray levels of infrared and visible light images, balance the gray level difference between the two, and retain large-scale information as much as possible. The control fusion ratio of IR and VI images is to provide the user with an optional parameter to adjust the background tendency of the fused image.

2.1. Multi-Scale Decomposition Based on SWF-Gaussian Filtering

Gaussian filter is a kind of simple linear filter, which is easy to use and has little calculation. However, Gaussian filter blurs all edges, which may cause halos and artifacts near the edges [28]. As an edge-preserving technology, the SWF technology was first proposed by Yin et al. The main idea of this technique is to keep the kernel function of the filter unchanged and construct several side windows around the image edge pixels in the filtering process. The output of the optimal side window is selected as the filtering result to avoid the blur caused by the filter window crossing the edge. Gaussian filtering and SWF-Gaussian filtering are respectively expressed as:

$$\mathrm{G}=\mathrm{Gaussian}\left(\mathrm{J},r_G,{\sigma }_G\right),$$

(1)

$$\text{SWF-G}=\text{SWF-Gaussian}\left(\mathrm{J},r_{\text{SWG-G}},{\sigma }_{\text{SWG-G}}\right),$$

(2)

J represents the input image. The parameters $r$ and $\sigma$ are the filtering radius and standard deviation, respectively. Their iterative updating formulas can be expressed by Equations (3) and (4), respectively.

$$J_G^{t+1}=G\left(J_G^t,r_G,{\sigma }_G\right),$$

(3)

$$J_{\text{SWF-G}}^{t+1}=\text{SWF-G}\left(J_{\text{SWF-G}}^t,r_{\text{SWF-G}},{\sigma }_{\text{SWF-G}}\right),$$

(4)

here $r_G$, $r_{\text{SWF-G}}$, ${\sigma }_G$, ${\sigma }_{\text{SWF-G}}$ are the same as those in Equations (1) and (2), $J_G^{t+1}$ and $J_{\text{SWF-G}}^{t+1}$ represent iteration t-time filtered images obtained by Gaussian filtering and SWF-Gaussian filtering, respectively. The initial input images are $J_G^1$ and $J_{\text{SWF-G}}^1$.

Figure 2a and Figure 3a are input images. Figure 2b–e shows the results of iterating the input images using a Gaussian filter and repeating the smoothing four times. It can be seen that the edge features become more and more blurred. It will cause bad results when extracting the detail components, which will be explained in the following section. Figure 3b–d show three iterations of smoothing using the SWF-Gaussian filter. They can not only blur the image but also preserve the edges of the large-scale structure. Figure 3e is the blurring image of Figure 3d with the Gaussian filter, which will be used as the base components for obtaining the detail components.

The detail components are obtained by the difference between the blurring images and the source image. In Figure 4, we can see that the Gaussian filter can output better detail components, but it has inevitable halo artifacts phenomena. For example, there is a circle of halo artifacts around the “person” in the image. The target person is not well-extracted. In contrast, Figure 5 is almost free of halos. Especially, Figure 5d shows that not only the detailed information is effectively extracted, but also the edge of the target is preserved. This effectively overcomes the halo artifacts.

Based on the above analysis, we propose a novel scheme to address halo artifacts. The scheme can obtain the base components with a combination of SWF-Gaussian filter and Gaussian filter. First, the SWF-Gaussian filter is iteratively used to produce blurred images with good edges, and then Gaussian blurring is used to produce the base components for obtaining detail components that are rich in detail but free of halos and artifacts, as follows:

$$I_{\text{SWF-G}}^{t+1}=\text{SWF-G}\left(I_{\text{SWF-G}}^t,r_{\text{SWF-G}},{\sigma }_{\text{SWF-G}}\right),$$

(5)

$$V_{\text{SWF-G}}^{t+1}=\text{SWF-G}\left(V_{\text{SWF-G}}^t,r_{\text{SWF-G}},{\sigma }_{\text{SWF-G}}\right),$$

(6)

$$B_I=G\left(I_{\text{SWF-G}}^{t+1},r_G,{\sigma }_G\right),$$

(7)

$$B_V=G\left(V_{\text{SWF-G}}^{t+1},r_G,{\sigma }_G\right),$$

(8)

where $I$ and $V$ represent IR and VI source images, respectively, and superscript represents the SWF-Gaussian filter iteratively from time t to t + 1. In our method, the IR base components $B_I$ and the VI base components $B_V$ are obtained by SWF-Gaussian filtering iteratively followed by Gaussian blurring. After that, the IR detail components $B_I$ and VI detail components $B_V$ are obtained by Equations (9) and (10).

$$D_I=I-B_{I,}$$

(9)

$$D_V=V-B_{V.}$$

(10)

The base components are not necessary to be iteratively produced through the SWF-Gaussian filtering. It is a fast and effective method to select a larger standard deviation to obtain the base components.

2.2. Base Components Fusion Base on Regularization and Nonlinearity

The decomposition components of different scale-spaces were obtained from the source image. The base components contain the general appearance structure of the source image and determine the overall brightness distribution. To keep the robustness of the base components, the imaging differences of two different bands need to be considered. The purpose of base components is to keep the large-scale structure and salient target of the two band images unchanged. The target in the infrared image is bright, while the background appears black in the IR image. Moreover, the dynamic range is extensive, so normalization was performed for better fusion.

$$B_I^{\prime }=\left(B_I-{\mu }_{B_I}\right)\frac{{\sigma }_{B_V}}{{\sigma }_{B_I}}+{\mu }_{B_V},$$

(11)

where $B_I^{\prime }$ represents the brightness of the IR base components with normalized brightness and $\mu$ is the mean. To further enlarge the distance of the effective value, $B_I^{\prime }$ is further processed.

$$L_I=\frac{B_I^{\prime }-\min \left(B_I^{\prime }\right)}{\max \left(B_I^{\prime }\right)-\min \left(B_I^{\prime }\right)},$$

(12)

where max() and min() are the maximum and minimum functions. $L_I$ reflects the characteristic distribution of IR base components. The base fusion components of IR and VI directly determine the background style of the fusion image. This paper proposes to use a nonlinear mapping function to adjust the propensity of fused image background style in IR and VI source images. Here, a nonlinear transform $S_{\lambda }()$ is introduced to further control the fusion ratio of IR and VI images:

$$S_{\lambda }\left(\alpha \right)=\frac{\arctan \left(\lambda a\right)}{\arctan \lambda },$$

(13)

where the range of $\alpha$ is (0,1), $\lambda$ represents a regularization parameter greater than 0 entered by the user. From Figure 6, we can see that when $\lambda \to 1$, the value $S_{\lambda }\left(\alpha \right)$ is close to $\alpha$, and the fusion rule tends to “average” fusion rule. When $\lambda \to \infty$, the value $S_{\lambda }\left(\alpha \right)$ is close to 1, and the fusion process tends to the “maximum absolute value” fusion. Therefore, the IR image fusion ratio can be controlled by adjusting the value of the parameter $\lambda$. Finally, the fusion weights $W_I^B$ and $W_V^B$ of the base components are obtained.

$$W_I^B=\frac{\arctan \left(\lambda L_I\right)}{\arctan \lambda }$$

(14)

$$W_V^B=1-W_I^B$$

(15)

The base components fusion can be expressed by

$$B_F=W_I^B{B}_I+W_V^B{B}_V$$

(16)

2.3. Detail Components Fusion Base on SWGF

In general, the detail components contain the texture information. However, the detail components of the two band images are very inconsistent and have complementary properties. The target of the detail components obtained by our decomposition method is prominent and the edge is good, but it also inevitably contains significant noise. Our objective is to transmit valid information and not noise. In this paper, we apply a simple and practical technique to inject bright and significant features from IR images directly into VI images, as follows:

$$P_I=\{\begin{array}{c}\left|D_I\right|-\left|D_V\right| if \left|D_I\right|-\left|D_V\right|>0\\ 0 otherwise\end{array}$$

(17)

where $P_I$ represents the salient feature of the IR detail components. $P_V$ is calculated with the same method. At the same time, the transmission of noise should be reduced. In this work, a practical optimization method based on the SWF technique improved side window guidance filter (SWGF) is proposed. SWGF can effectively reduce the interference of noise in the process of information transmission and better preserve the edges of the target. Filtering operation is denoted as follows:

$$W_I^D=\mathrm{SWGF}\left(I,P_I,r_{\mathrm{SWGF}}^D,{\sigma }_{\mathrm{SWGF}}^D\right),$$

(18)

$$W_V^D=\mathrm{SWGF}\left(V,P_V,r_{\mathrm{SWGF}}^D,{\sigma }_{\mathrm{SWGF}}^D\right),$$

(19)

where $\mathrm{SWGF}\left(·\right)$ represents the side window guided filtering function operation, and $W_I^D$ and $W_V^D$ represent the weights of the detail components of the two waveform images, respectively. $r_{\mathrm{SWGF}}^D$ and ${\sigma }_{\mathrm{SWGF}}^D$ represent the window radius and standard deviation parameters of SWGF, respectively.

Then, the detail components fusion $D_F$ is obtained from the linear weighting of the weight map. It is worth noting that the weight map of the detail components should be normalized to [0, 1].

$$D_F=W_I^D{D}_I+W_V^D{D}_V$$

(20)

2.4. Reconstruction

Generally, image reconstruction is the inverse operation of image decomposition. In the multi-scale image decomposition process proposed in this paper, the detail layer component is obtained by subtracting the base layer component from the original image. Based on this idea, the fusion image reconstruction is performed by adding the base layer fusion component and the detail layer fusion component.

Eventually, the fused image $F$ is reconstructed by simple linear addition.

$$F=B_F+D_F$$

(21)

The flow of the proposed Algorithm 1 in this paper is shown as follows:

Algorithm 1: New algorithms for fusing IR and VI images

Input: IR image $I$, VI image $V$, iteration times $t$, parameter $\lambda$.

1: Decompose the input images $(I,V)$ into base components $(B_I,B_V)$ and detail components $(D_I,D_V)$ through Gaussian filtering and SWF-Gaussian filtering.

2: Calculate the fusion weight $(W_I^B,W_V^B)$ of base components part according to Equations (11), (12), (14) and (15).

3: According to Equation (16), the fusion result $(B_F)$ of the base components part is obtained.

4: Calculate the fusion weight $(W_I^D,W_V^D)$ of detail components part according to Equations (17)–(19).

5: According to Equation (20), the fusion result $(D_F)$ of the detail components part is obtained.

6: The final fusion image $(F)$ is reconstructed by Equation (21).

Output: Fused image $(F)$.

Figure 7 shows the fusion results, where Figure 7a,b show the IR and VI source images. Figure 7c–f show the fusion results when the base components fusion coefficient $\lambda$ is set to $\lambda =1, 5, 50, 500$, respectively, and the rest of the parameters are set the same. By observing Figure 7c–f, we can learn that when $\lambda =1$, the background of the fusion image “sky” is relatively bright, and then as the value of $\lambda$ becomes larger and larger, the background of “sky” becomes darker and darker. The reason for this phenomenon is that $\lambda$ is the parameter to control the fusion ratio of the IR base components. When the $\lambda$ value is small, the VI base components occupy a larger fusion ratio, and the background tends to the VI image style as a whole. When the $\lambda$ value is large, the IR base components image occupies a large proportion of fusion, and the background as a whole tends to the IR image style. Users can adjust the proportion of fusion by setting $\lambda$ to obtain fusion images with different background styles.

3. Results

In this section, several state-of-the-art methods were employed to evaluate the performance of the proposed fusion framework. Both subjective evaluation and objective evaluation were adopted. The comparison algorithms used are as follows: LAP [4], DWT [6], NSCT [10], CVT [11], GFF [14], GTF [15], IFE [32], LPSR [20], and MSVD [12]. The TNO Image Fusion Dataset [31] was chosen for the data required for the fusion experiments.

3.1. Subjective Performance Evaluation

We selected five pairs of well-registered images, which contain various natural and artificial scenes. The fusion results are shown in Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12.

Figure 8 shows the fusion comparison results. Figure 8d–f have low contrast, which results in less prominent targets. Although Figure 8c,g,i,j have normal contrast, the background information loses a lot and the fusion effect is not uniform enough. Figure 8h becomes very blurred and a lot of the source image information is lost. Figure 8k retains much detailed information, but there are artifacts, such as people in the image. As shown in Figure 8l, our new method can achieve not only good contrast but also an obvious target and enough details.

The fusion images in Figure 9d–f,k have low contrast. In addition, the critical target information in the VI source image is dim, such as billboards and people. Figure 9c,g,h have artifacts and halos. The visual effect is not right. Figure 9h looks very blurry. The contrast and overall details of Figure 9i,l are well-preserved. However, the effect of Figure 9l is better than others. For example, the character edges of billboards in the fused images of the new method are much better.

The fusion images shown in Figure 10 have a large range of brightness, which can be used to verify the robustness of the fusion method. The fusion effect of Figure 10g,i,j is very poor for the “sky”, and the background of “cloud” in the IR image is unseen, especially the obvious halo phenomenon in Figure 10j. The IR fusion of Figure 10c,h is too high, and the VI information is seriously lost. From the images of “roof”, “floor tile”, and “car windshield”, it can be seen that Figure 10d–f,k are less ideal than Figure 10l in retaining details and edges.

Figure 11l is the result of our method, in which the edges of pedestrian and water surface reflection are transparent. There are no artifacts and halos in our method. Besides, the fused images have high contrast and less noise.

The sky area in Figure 12g,h is polluted. Figure 12d–f,k have low contrast and dark areas of the sky. Moreover, there is a halo around the pedestrian, and the target is polluted. Figure 12c,i,j highlight the details, but there are apparent artifacts on the edge of the right tree. Figure 12l shows that our method has the capability of complementing IR and VI source images and overcoming edge artifacts and halos.

In addition to the above state-of-the-art classical methods, some latest deep learning-based fusion methods were employed to compare with the proposed method in subjective performance analysis, such as DenseFuse [33], FusionGAN [16], U2Fusion [34], GANMcC [35], and DDcGAN [36]. Qualitative comparisons were performed on image pairs in the TNO dataset [37] and the RoadScene dataset [35], respectively. The qualitative results on the TNO datasets and RoadScene datasets are shown respectively in Figure 13 and Figure 14. Overall, the proposed method exhibits a better appearance than its competitors. As shown in regions rich in texture information, the competitors lost some details, e.g., the texture of the clothes, the details of the door, and the texture of the leaves, as shown in Figure 13. Furthermore, the proposed method was also applied to fuse VIS and IR images in the RoadScene. As shown in Figure 14, the details in the fusion results appear for better scene representation because the detail layer process was performed. The promising results show that the proposed method achieves a better appearance representation and less distortion, noise, or artifacts.

3.2. Objective Performance Evaluation

In addition to subjective observation, there are seven popular objective metrics for performance evaluation: information entropy (EN) [38], mutual information (MI) [38], $Q^{AB/F}$ [38], $Q^W$ [39], visual information fidelity fusion metrics (VIFF) [40], spatial frequency (SF) [38], and feature mutual information (FMI) [41]. EN is used to evaluate the amount of information on fusion results. MI reflects the quantity of effective information transfer in the process of fusion. $Q^{AB/F}$ and $Q^W$ represent the similarity between the strong edges of the target before and after the fusion process. VIFF is used to evaluate the fidelity of visual information before and after fusion. SF can represent the clarity and spatial variation of fusion results. FMI measures the feature mutual information for fusion results, and the gradient map is usually used as the evaluation feature. We used these methods to examine the fusion results of the five image pairs. Figure 15 shows the quantitative evaluation results applied to the images shown in Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12. A separate evaluation cannot comprehensively measure the image quality. Different methods perform differently in different metrics, so we could not analyze the performance of each index in isolation. Therefore, we needed to combine a variety of evaluation indicators. As shown in Figure 15, although our method was not optimal in all the indicators, the average evaluation was the best overall. Our method was optimal in both EI and $Q^W$ metrics. This fully illustrates the advantage of our method in conveying edge structure and texture information. The overall edge structure and visual effect also have good performance. Although our method was not the best one in SF, MI, $Q^{AB/F}$, and FMI, it could work very well on most of the image pairs. However, our method could overcome halo and artifact issues better than other methods.

In recent years, deep learning-based fusion methods have improved fusion performance. Some of the Representative DL-based methods were compared with the proposed method, such as DenseFuse, FusionGAN, U2Fusion, GANMcC, and DDcGAN. Quantitative comparisons were performed on image pairs in the TNO dataset, as shown in Figure 13. FMI, MI, $Q^{AB/F}$, $Q^W$, and SF metrics were used for evaluation. The comparison results are shown in

Table 1. Some of the Representative DL-based methods are compared.

Metric	Method	Img1	Img2	Img3	Img4	Img5	Img6	Img7	Rank
FMI	U2Fusion	0.89	0.92	0.83	0.79	0.92	0.87	0.89	1
	DDcGAN	0.88	0.9	0.82	0.81	0.9	0.86	0.88
	GANMcC	0.9	0.93	0.85	0.81	0.91	0.88	0.9
	DenseFuse	0.9	0.93	0.84	0.81	0.92	0.89	0.91
	FusionGAN	0.89	0.92	0.86	0.79	0.9	0.88	0.88
	Ours	0.92	0.94	0.86	0.83	0.92	0.9	0.91
MI	U2Fusion	1.38	1.86	2.07	2.16	2.41	1.95	1.83	1
	DDcGAN	1.48	1.81	1.85	2.16	1.93	1.61	1.9
	GANMcC	1.86	2.45	2.44	2.58	2.62	2.13	2.45
	DenseFuse	2.11	2.23	2.23	3.56	2.62	2.15	1.88
	FusionGAN	1.93	2.87	1.72	1.36	2.37	1.95	2.58
	Ours	2.76	2.12	2.45	2.8	2.42	2.55	2.16
$Q^{AB/F}$	U2Fusion	0.43	0.4	0.41	0.28	0.47	0.42	0.41	1
	DDcGAN	0.4	0.27	0.28	0.44	0.35	0.36	0.28
	GANMcC	0.17	0.27	0.3	0.07	0.44	0.22	0.36
	DenseFuse	0.33	0.32	0.38	0.19	0.39	0.32	0.31
	FusionGAN	0.23	0.16	0.17	0.05	0.23	0.14	0.27
	Ours	0.56	0.51	0.53	0.55	0.58	0.57	0.52
$Q^W$	U2Fusion	0.71	0.71	0.73	0.52	0.81	0.77	0.75	1
	DDcGAN	0.67	0.44	0.26	0.44	0.56	0.59	0.49
	GANMcC	0.49	0.58	0.59	0.32	0.73	0.55	0.68
	DenseFuse	0.71	0.66	0.66	0.49	0.68	0.68	0.6
	FusionGAN	0.47	0.45	0.27	0.21	0.37	0.41	0.49
	Ours	0.88	0.77	0.71	0.62	0.8	0.81	0.6
SF	U2Fusion	9.88	9.6	9.37	23.46	7.45	10.17	9.22	3
	DDcGAN	12.24	7.72	11.29	26.02	7.52	11.46	8.66
	GANMcC	4.73	5.61	5.05	10.98	5.62	5.15	5.46
	DenseFuse	6.58	5.25	5.19	16.45	3.35	6.33	4.08
	FusionGAN	6.32	4.39	4.19	10	3.29	4.66	5.45
	Ours	11.85	8.1	7.28	28.24	5.8	10.68	6.4

. On the TNO dataset, the proposed method achieves the best performance on FMI, MI, $Q^{AB/F}$, and $Q^W$. The results on SF follow behind U2Fusion and DDcGAN by 2.82 and 2.26, respectively. These results show that in the proposed model, the similarity and correlation between the fusion image and source images are higher and have less distortion and larger gradient amplitude. A bold number indicates the performance of the best method among all methods in terms of a particular metric.

The efficiency of the algorithm is also an important metric.

Table 2. Running time (seconds) is compared.

LAP	DWT	NSCT	CVT	GFF	GTF	IFE	LPSR	MVSD	Proposed
0.2294	0.6356	0.8087	0.3890	0.2207	0.8208	0.4776	0.1210	0.3265	0.2021

reports the running time of the fusion algorithm shown in Figure 9. All algorithms were running on a computer of i7-8700 CPU and 8 G RAM, using MATLAB 2018b (MathWorks company, Natick, MA, USA).

4. Conclusions

This paper presents a novel decomposition-based image fusion framework, which overcomes the problems of noise, blurring, and loss of details. Firstly, the decomposition method of basic components and detail components based on SWF-Gaussian was proposed. More importantly, we developed a base components fusion strategy through normalization and nonlinear processing, which could control the fusion ratio by adjusting the regularization parameters. Furthermore, SWGF was applied to optimize the fusion process of detail components, suppressing the noise transfer effect, and obtaining better visual perception. The quantitative results show that the method has the advantages of simplicity, effectiveness, and robustness.