Improved Procedure for Multi-Focus Image Quality Enhancement Using Image Fusion with Rules of Texture Energy Measures in the Hybrid Wavelet Domain

Abstract

Feature extraction is a collection of the necessary detailed information from the given source, which holds the information for further analysis. The quality of the fused image depends on many parameters, particularly its directional selectivity and shift-invariance. On the other hand, the traditional wavelet-based transforms produce ringing distortions and artifacts due to poor directionality and shift invariance. The Dual-Tree Complex Wavelet Transforms (DTCWT) combined with Stationary Wavelet Transform (SWT) as a hybrid wavelet fusion algorithm overcomes the deficiencies of the traditional wavelet-based fusion algorithm and preserves the directional and shift invariance properties. The purpose of SWT is to decompose the given source image into approximate and detailed sub-bands. Further, approximate sub-bands of the given source are decomposed with DTCWT. In this extraction, low-frequency components are considered to implement Texture Energy Measures (TEM), and high-frequency components are considered to implement the absolute-maximum fusion rule. For the detailed sub-bands, the absolute-maximum fusion rule is implemented. The texture energy rules have significantly classified the image and improved the output image’s accuracy after fusion. Finally, inverse SWT is applied to generate an extended fused image. Experimental results are evaluated to show that the proposed approach outperforms approaches reported earlier. This paper proposes a fusion method based on SWT, DTCWT, and TEM to address the inherent defects of both the Parameter Adaptive-Dual Channel Pulse coupled neural network (PA-DCPCNN) and Multiscale Transform-Convolutional Sparse Representation (MST-CSR).

1. Introduction

In this digital era, image processing plays a vital role, and it is essential to study the various approaches to it. Initially, texture analysis concentrates on statistical measures of texture measured from the Grey Level Co-occurrence Matrix (GLCM) [1,2]. Texture is one of the visually significant characteristics to enrich the source information for further analysis. Image recognition has many applications, including remote sensing, content-based image representation, automatic inspection of data, and so on. Texture is a feature extraction that identifies structural, geometrical, statistical, and model-based representations [3]. The traditional image fusion approaches include PA-DCPCNN and MST-CSR. The performance of the PA-DCPCNN method is limited due to the inbuilt demerits of the Non-subsampled Contourlet Transform (NSCT) and Pulse Coupled Neural Network (PCNN). The effective fusion rule based on MST-CSR can be developed to achieve better fusion performance. To analyze and characterize the information, wavelets are most widely used. In this proposed method, precise and unified framework activity is constructed along with hybrid wavelets with the combination of SWT [4,5] and DTCWT [6,7,8,9]. Further, for efficient information extraction, texture features are applied [10,11,12]. This procedure for framework activity is described in Section 2.

Several approaches to MIF have been proposed over the years. For example, Mahesh et al. proposed the usage of Spatial Pyramid Zernike Moments (SPZM) and Law’s Textures Feature Set (LTFS)-based facial expression recognition system to describe the patterns of facial region changes for movement in the muzzles of the face. This work is constructed with multilayer perception and feedforward neural networks. An experiment is carried out to demonstrate concatenating feature vectors of facial expression with better accuracy and a low error rate [13]. Ganasala et al. proposed nonsubsampled shearlet transform (NSST) and TEM to extract helpful information that is practical and valuable related to the study of human body structure. In this work, the luminance, in-phase, quadrature (YIQ) colour model is implied for functional information confinement. The visual analysis is done with five quality metrics such as spatial frequency (SF), edge-intensity (EI), mutual information (MI), visual information fidelity (VIF), and structural similarity index measure (SSIM) [14]. In medical imaging, clinical procedures essential to the diagnosis of disease and surgery using image fusion. Medical procedures demand high accuracy. This work is implemented with the combination of SWT and TEM by Ganasala et al. The superiority of the work is proved with the help of fusion quality metrics, and practical analysis tasks are studied [15]. In multi-resolution (MR) fusion, to describe the pattern of an image, Gaussian Markov random fields (GMRF) and Gibbs random fields (GRF) play vital roles in characterizing the texture. This work was implemented with Law’s mask descriptor for MR images and a k-nearest neighborhood classifier for classifying each texture pattern into appropriate classes by Dash et al. [16]. Setiawan et al. concentrate on TEM laws for feature extraction and artificial neural network (ANN) for classification. The superiority of this work is proved by comparing it with other procedures in terms of accuracy [17]. The approaches cited in the literature are compared to the proposed methodology both qualitatively and quantitatively [4,18,19,20,21,22]. From the above literature study, it is identified that the following are weak points: lack of spatial distortion, image blurring, low efficiency of the method, results related to visual and quantitative quality needing improvement, lack of shift-invariance, and poor directional selectivity.

Numerous imaging fusion approaches are now being used in various applications, including security, remote sensing, machine vision, and medical imaging, thanks to recent technological breakthroughs. It is still necessary to investigate a variety of difficulties related to image fusion, specifically in biomedical image fusion. Additionally, a suitable, accurate and consistent fusion technique is needed for the various image types for diverse domains, which should be simple to understand to get better results. In addition, misregistration is a significant problem discovered when fusing images. Hence image fusion solutions must be resilient against unpredictable acquisition conditions or reasonable computing time in real-time systems. The critical idea of the proposed method is to investigate the suitability of the hybrid wavelet transforms using texture energy rules for a wide range of standard test images, including biomedical images. Further, it is well known that in the case of wavelet transform-based methods, the image is transformed into another space (using a transformation function) so that the texture is more easily distinguishable in the new space.

The directional selectivity and shift invariance of the fused image are two factors that significantly impact its quality. As a result of their weak directionality and shift invariance, conventional wavelet-based transforms, on the other hand, result in ringing distortions and aberrations. In order to address the shortcomings of the conventional wavelet-based fusion technique and maintain the directional and shift invariance qualities, the hybrid wavelet fusion algorithm uses DTCWT in combination with SWT. High-frequency components in this extraction are thought to execute the absolute-maximum fusion rule, while low-frequency components are thought to implement TEM. The absolute-maximum fusion rule is applied to the specific sub-bands. The accuracy of the output image after the fusion process is considerably increased by the texture energy rules classification of the image.

The proposed method aims to eliminate distortion for a better-quality fused image. Most multi-focus image fusion (MIF) algorithms suffer from blurring because of low spatial resolution. The SWT and DTCWT used with the TEM method considerably affect composite images. The proposed method successfully improves the quality of fused images by increasing their resolution, decreasing additive noise, minimizing distortion, and preserving edge detail and other important values such as the contrast in high-contrast regions of the image.

The key contributions of the proposed work are as follows:

• Multi-focus images are fused using a hybrid technique, i.e., Hybrid Wavelet with TEM;
• In the case of transform wavelet-based methods, the image is transformed into another space (using a transformation function) so that the texture is more easily distinguishable in the new space;
• The procedure plays a dominating role in the fusion of several source images to create a new image with enhanced quality, directionality, shift invariance, visual quality, and information retention over the source images;
• Using TEM Laws, various image characteristics are extracted. Using these attributes for fusion results in a fused image with much information;
• A process of analysing has been done both quantitatively and qualitatively;
• Feature extraction is more efficient than traditional methodologies.

The continuing parts are arranged as ensues: The proposed fusion concept is discussed in Section 2. Section 3 outlines the experimental results. Section 4 discussed qualitative comparison. Section 5 gives the conclusions.

2. Proposed Fusion Scheme

In this work, TEM laws [15] are used to predict the features of the image. The input image data is passed through with SWT [4,5], which quantifies the information into approximate and detailed bands. Subsequently, the approximate information is put into detailed and approximate bands again. Further, TEM laws are applied on the low contrast part of the approximate bands, and the absolute-maximum fusion rule is implemented on approximate bands of DTCWT [6,7,8,9]. Further, the inverse DTCWT procedure is applied to the output of the low contrast part and absolute maximum fusion rule. The absolute-maximum fusion rule is implemented in the case of SWT detailed bands. Further, inverse SWT is applied on the output of detailed sub-bands of SWT and output of IDTCWT to generate the final image, as shown in Algorithm 1. The proposed method can provide both good shift invariance and directional selectivity, enhances the resolution of fused images, eliminates additive noise, and reduces distortion, all of which increase image quality. Figure 1 represents the fusion scheme of the proposed method.

Algorithm 1: Hybrid Wavelet (i.e., SWT and DTCWT) with laws of TEM

Input:  Multi-focus Images Output:  Features Extracted Image Steps:  Apply SWT to generate sub-band images of given multi-focus images. Apply DTCWT for LL sub-bands of multi-focus images. TEM laws are applied on the low contrast part of the approximate bands, and the absolute-maximum fusion rule is implemented on approximate bands of DTCWT. Apply inverse DTCWT procedure on the output of low contrast part and absolute- maximum fusion rule. The absolute-maximum fusion rule is implemented in the case of SWT detailed bands. Apply inverse SWT on the output of detailed sub-bands of SWT and output of IDTCWT to generate the final image, i.e., features extracted image.

The following significant steps of the proposed scheme involve SWT decomposition, DTCWT decomposition, and implementation of TEM, which are discussed in the following subsections.

2.1. Decomposition Procedure Using SWT

Down-sampling and up-sampling of two primary operations proposed in wavelet transformations. Regarding other wavelet transformations, SWT [4,5] implements an up-sampling procedure that up-samples the filters at each level of decomposition. Instead of down-sampling, it removes odd values. With this procedure, sub-bands are created. The sub-bands of the SWT wavelet have the same size as the source. The second degree of SWT sub-bands is represented by four levels: LL (low-low), LH (low-high), HL (high-low), and HH (high-high). Here, LL is referred to as approximation sub-bands and others are detailed sub-bands. The flow diagram of SWT is represented in Figure 2.

The source image is processed to low and high-pass extracts with SWT to get approximation and detailed coefficients. Ref. [23] illustrates the SWT at the jth level for the source image (M, N).

$$\mathrm{Low}\text{\_}{\mathrm{Low}}_{\mathrm{K}+1}\left(\mathrm{r}+\mathrm{c}\right)={\displaystyle \sum }_{\mathrm{p}}{\displaystyle \sum }_{\mathrm{q}}{\mathrm{low}}_{\mathrm{p}}^{\mathrm{k}}{\mathrm{low}}_{\mathrm{q}}^{\mathrm{k}}\mathrm{Low}\text{\_}{\mathrm{Low}}_{\mathrm{K}}\left(\mathrm{r}+\mathrm{p},\mathrm{c}+\mathrm{q}\right)$$

(1)

$$\mathrm{Low}\text{\_}{\mathrm{High}}_{\mathrm{K}+1}\left(\mathrm{r}+\mathrm{c}\right)={\displaystyle \sum }_{\mathrm{p}}{\displaystyle \sum }_{\mathrm{q}}{\mathrm{high}}_{\mathrm{p}}^{\mathrm{k}}{\mathrm{low}}_{\mathrm{q}}^{\mathrm{k}}\mathrm{Low}\text{\_}{\mathrm{Low}}_{\mathrm{K}}\left(\mathrm{r}+\mathrm{p},\mathrm{c}+\mathrm{q}\right)$$

(2)

$$\mathrm{High}\text{\_}{\mathrm{Low}}_{\mathrm{K}+1}\left(\mathrm{r}+\mathrm{c}\right)={\displaystyle \sum }_{\mathrm{p}}{\displaystyle \sum }_{\mathrm{q}}{\mathrm{low}}_{\mathrm{p}}^{\mathrm{k}}{\mathrm{high}}_{\mathrm{q}}^{\mathrm{k}}\mathrm{Low}\text{\_}{\mathrm{Low}}_{\mathrm{K}}\left(\mathrm{r}+\mathrm{p},\mathrm{c}+\mathrm{q}\right)$$

(3)

$$\mathrm{High}\text{\_}{\mathrm{High}}_{\mathrm{K}+1}\left(\mathrm{r}+\mathrm{c}\right)={\displaystyle \sum }_{\mathrm{p}}{\displaystyle \sum }_{\mathrm{q}}{\mathrm{high}}_{\mathrm{p}}^{\mathrm{k}}{\mathrm{high}}_{\mathrm{q}}^{\mathrm{k}}\mathrm{Low}\text{\_}{\mathrm{Low}}_{\mathrm{K}}\left(\mathrm{r}+\mathrm{p},\mathrm{c}+\mathrm{q}\right)$$

(4)

where r equals 1, 2, 3, 4, … M and c equals 1, 2, 3, 4, … N and the low and high values indicate the low_pass and high_pass extracts. The low-frequency sub-bands at level k + 1 are denoted by the notation Low_Low_K+1, while Low_Low_K denotes those at level k. Low_High_K+1 represents the signal’s horizontal-detail coefficient, High_Low_K+1 represents the signal’s vertical-detail coefficient, and High_High_K+1 represents the signal’s diagonal-detail coefficient. SWT improves shift invariance and low computational time because the algorithm does not need to down-sample. In SWT, filters are applied first to rows and secondly to columns. Due to these advantages, SWT is selected for the proposed method.

2.2. DTCWT Based Decomposition

In this subsection, concerning the base of SWT approximate sub-bands, DTCWT [6,7,8,9] is implemented for further decomposition, decomposing the information into two bands, LL and HH, respectively. TEM laws are applied for the bands of LL, which is discussed in Section 2.3. When implementing the DTCWT, extract the sub = bands using Equation (5).

$$\left\{\left(L_t^X{H}_{l,d}^X\right)|X=A,B;levels=1,2,3;t-orientation=1, 2;d-orientation=1,2,3,4,5,6\right\}$$

(5)

where $L_t^x$ stands for the L-level sub-bands at low frequencies (t orientation) and $H_{l, d}^x$ for the H-level sub-bands at high frequencies (d orientation); Low and High are constructed.

2.3. Implementation of TEM Laws for Final Fusion Process

In general, approximate sub-bands consist of much data from a source image. It is essential to transfer this information into a fused image, extracting features such as edges, ripples, waves, and sparks [15]. The following three primary vectors are used. $L=\left[1, 2,1\right]$, $E=\left[-1, 0, 1\right]$, $S=\left[-1, 2,-1\right]$. With these three vectors, nine convolution matrixes are listed as follows:

$${\mathit{matrix}}_1=\left[\begin{array}{ccc}1& 2& 1\\ 2& 4& 2\\ 1& 2& 1\end{array}\right]$$

(6)

$${\mathit{matrix}}_2=\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]$$

(7)

$${\mathit{matrix}}_3=\left[\begin{array}{ccc}-1& 2& -1\\ -2& 4& -2\\ -1& 2& -1\end{array}\right]$$

(8)

$${\mathit{matrix}}_4=\left[\begin{array}{ccc}1& 0& -1\\ 0& 0& 0\\ -1& 0& -1\end{array}\right]$$

(9)

$${\mathit{matrix}}_5=\left[\begin{array}{ccc}1& -2& 1\\ 0& 0& 0\\ -1& 2& -1\end{array}\right]$$

(10)

$${\mathit{matrix}}_6=\left[\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ 1& 2& 1\end{array}\right]$$

(11)

$${\mathit{matrix}}_7=\left[\begin{array}{ccc}1& -2& 1\\ -2& 4& -2\\ 1& -2& 1\end{array}\right]$$

(12)

$${\mathit{matrix}}_8=\left[\begin{array}{ccc}-1& -2& -1\\ 2& 4& 2\\ -1& -2& -1\end{array}\right]$$

(13)

$${\mathit{matrix}}_9=\left[\begin{array}{ccc}1& 0& -1\\ -2& 0& 2\\ 1& 0& -1\end{array}\right]$$

(14)

The mask of matrix₁ is non-zero-sum, and the remaining are zero-sum. The mask matrix₁ produces average energy. matrix₂ and matrix₃ produce first and second-order vertical edges. matrix₆ gives horizontal edges. matrix₄ refers to waves. matrix₅, matrix₇, and matrix₉ represents ripples. The proposed image fusion process uses 3 × 3 masks, providing the fused image with detailed information. Using these 3 × 3 texture masks, the coefficient saliency measurement for the approximation sub-band is created. Assume that ASX represents the source image X’s approximate sub-band. Further, TEM, represented as TEM₁ to TEM₉, relates to convolution matrices: matrix₁ to matrix₉ are used in the following Equations (15)–(23).

$${\mathit{TEM}}_1\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_1\left(m+2,n+2\right)$$

(15)

The convolution matrix₁ should be applied on the approximate sub-band corresponding block; the result is stored in the TEM₁(r, c). Similarly, the remaining convolution matrices will be applied to the corresponding blocks of approximate sub-bands.

$${\mathit{TEM}}_2\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_2\left(m+2,n+2\right)$$

(16)

$${\mathit{TEM}}_3\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_3\left(m+2,n+2\right)$$

(17)

$${\mathit{TEM}}_4\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_4\left(m+2,n+2\right)$$

(18)

$${\mathit{TEM}}_5\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_5\left(m+2,n+2\right)$$

(19)

$${\mathit{TEM}}_6\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_6\left(m+2,n+2\right)$$

(20)

$${\mathit{TEM}}_7\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_7\left(m+2,n+2\right)$$

(21)

$${\mathit{TEM}}_8\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_8\left(m+2,n+2\right)$$

(22)

$${\mathit{TEM}}_9\left(r,c\right)={{\displaystyle \sum }}_{m=-1}^1{{\displaystyle \sum }}_{n=-1}^{1}ASX{\left(r+m,c+n\right)}^{2}matri{x}_9\left(m+2,n+2\right)$$

(23)

Absolute values of TEMs are evaluated by using the following formula. Let the normalized TEMs be NTEMX₁, NTEMX₂, NTEMX₃, NTEMX₄, NTEMX₅, NTEMX₆, NTEMX₇, NTEMX₈, NTEMX₉

$$\mathit{NTEM}{X}_i =\mathrm{normalized} \left(\right|\mathit{TEM}{X}_i\left|\right)$$

(24)

Saliency of approximate sub-bands is calculated using Equation (25).

$$\mathit{SASX}={{\displaystyle \sum }}_{i=1}^{9}NTEM{X}_i$$

(25)

In a similar way, the saliency measure SASY is determined for the approximate sub-bands ASY of the original image Y. The first fusion decision map (DM) is gathered by comparing the saliency values of the two source images at each position. Fusion decision maps for saliency measures are initially developed as

$$\mathit{DM}(r, c)={\{}_{0 Otherwise}^{1 if SASX\left(r,c\right)\ge SASY\left(r,c\right)}$$

(26)

A structural element (SE), which fills in any minor gaps smaller than the SE, is used to perform additional morphological operations in closed as well as open procedures. Given in Equation (27), it is indicated by CD. The action of closing an image is denoted by the symbol “●”, and on a closed decision map, the morphological opening operation is carried out to remove any tiny white items that are smaller than the SE. OD, as given in Equation (28), designates the decision map that results. The picture closure action is represented by the operator “o” in this case.

CD = DM●SE 

(27)

OD = CD o SE 

(28)

Isolated white patches are removed by using the following Equation (29).

AD = bwareaopen (OD, N) 

(29)

In order to eliminate black patches, the following Equation (30) is used

FD = bwareaclose (AD, N) 

(30)

Finally, approximate coefficients fused information is represented by using Equation (31).

$$\mathit{ACF}(r, c)={\{}_{ASY\left(r,c\right) if FD\left(r,c\right)=0}^{ASX\left(r,c\right) if FD\left(r,c\right)=1}$$

(31)

In the case of the detailed sub-band rule, fused information is obtained by using the following Equation (32).

$$\mathit{DSF}(r, c)={\{}_{DSY\left(r,c\right) Otherwise}^{DSX\left(r,c\right) if \left|DSX\left(r,c\right)\ge DSY\left(r,c\right)\right|}$$

(32)

2.4. Measuring the Performance of the Proposed Method

In this section, the proposed strategy’s effectiveness is evaluated subjectively and objectively compared to state-of-the-art approaches. The qualitative evaluation of the visual quality of the fused image is referred to as the subjective assessment. On the other hand, comparing the results of several image fusion efficiency measures is how objective evaluation, also known as quantitative evaluation, is carried out. This quantitative approach, known as objective analysis, is based on mathematical modeling. It assesses the spectral and spatial similarity of the fused image to the input images. There are two approaches to conducting quantitative analysis: with or without a reference image [24]. In this paper, the proposed fusion method experiments are conducted with and without reference images.

In this study, a total of eighteen well-known metrics, including entropy (E(F)) [20,24,25], average-gradient (AG) [20,25], correlation-coefficient (CC) [20,24,25], total-fusion-performance (Q^AB/F) [20,25], structural-similarity (SSIM) [20,24,26], edge-dependent-fusion-quality (Q_E) [20,27], standard-deviation (SD) [24,25], spatial-frequency (SF) [24,25], weighted-fusion-quality (Qw) [28], Feature-Mutual-Information (FMI) [24], Fusion-Loss (L^AB/F) [25], Fusion-Artifacts (N^AB/F) [25], Root-Mean-Square-Error (RMSE) [24], Peak-Signal-to-Noise-Ratio (PSNR) [24], Edge-Intensity (EI) [27], Gray-Mean-Value (GM) [29,30], Image-Definition (ID) [29,31], Percentage-Fit-Error (PFE) [24], and UIQI (Q⁰) [29,32] are used to assess the interpretations of dissimilar fusion approaches quantitatively.

3. Experimental Results

A hybrid wavelet (SWT and DTCWT) and TEM are proposed in this paper. The proposed algorithm’s quality was evaluated using quality metrics, including correlation-coefficient (CC), entropy (E(F)), total-fusion-performance (Q^AB/F), average-gradient (AG), structural-similarity (SSIM), standard-deviation (SD), spatial-frequency (SF), weighted-fusion-quality (Q_W), edge-dependent-fusion-quality (Q_E), feature-mutual-information (FMI), root-mean-square-error (RMSE), fusion-loss (L^AB/F), peak-signal-to-noise-ratio (PSNR), fusion-artifacts (N^AB/F), edge-intensity (EI), gray-mean-value (GM), image-definition (ID), percentage-fit-error (PFE), and universal quality index (Q⁰). The proposed strategy is contrasted with previously published methods using these metrics. These criteria are used to assess the comparison and robustness of the fused images against distortions. Source images that are utilized for comparison are familiar with multi-focus image fusion. Numerous images from various datasets are used in experiments [33]. The proposed method produces successful outcomes for these images. However, the approaches that are contrasted with the proposed methodology do not yield results for these images, so they are not included in the publication. This paper compares approaches in the literature with [4,18,19,20,21,22] using the desk, book, clock, flower, lab, leaf, flowerpot, Pepsi, balloon, grass, leopard, calendar, wine, craft, and remote sensing images. Additionally, the results of the proposed method for a few tested source images are shown. The images come in a range of sizes and quality. The proposed approach can be used with any multi-focus image, not just in this study.

3.1. Outcomes of the Proposed Methodology for a Few Specified Images

Several kinds of grayscale images are utilized to apply the proposed method. Analyses such as E(F), SD, AG, QE, CC, SSIM, Q^AB/F, RMSE, PFE, PSNR, and SF are applied to these images. Many different kinds of images are used and analysed. Images in Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 are the visual outcomes for the balloon, leopard, craft, wine, grass, and calendar images.

Table 1. The performance of the proposed methodology for a couple of trailing images.

Input Images	E(F)	AG	CC	QAB/F	SSIM	QE	SD	SF	RMSE	PFE	PSNR
Book	7.4138	12.3380	0.9898	0.9339	0.9230	0.9196	60.1162	24.5811	9.4618	9.2919	38.4051
Clock	7.4149	5.2647	0.9811	0.9874	0.9875	0.9068	51.2894	8.5571	3.1392	2.8573	43.1966
Desk	7.3534	8.1919	0.9626	0.9960	0.9895	0.9284	47.5574	15.6158	3.3421	3.0798	42.9247
Flower	7.2059	14.3306	0.9651	0.9970	0.9907	0.9069	38.8063	21.9121	4.1448	3.7621	41.9898
Lab	7.1336	6.6539	0.9776	0.9962	0.9922	0.9261	47.6548	13.0163	2.6898	2.0455	43.8676
Leaf	7.4216	25.6660	0.9237	0.9745	0.8225	0.8478	46.1852	30.8838	9.9976	8.1367	38.1658
Balloon	7.4656	9.8594	0.9844	0.9996	0.9997	0.9555	48.3050	20.8516	0.6925	0.5604	49.7605
Pepsi	7.1289	8.1156	0.9823	0.9983	0.9889	0.9475	45.5193	14.0604	2.3387	2.1737	44.4751
Leopard	7.4275	13.2577	0.9898	0.9995	0.9993	0.9524	65.7919	19.7912	0.7950	0.7018	49.1613
Flowerpot	7.4446	13.3991	0.9746	0.9943	0.9902	0.9163	52.5217	23.6082	3.3136	2.6395	42.9618
Wine	7.6381	35.0934	0.9497	0.9894	0.9278	0.8742	71.9911	51.5020	7.3466	5.3524	39.5039
Craft	6.4911	5.9747	0.9634	0.9988	0.9930	0.8880	30.8352	12.5696	1.5758	1.0967	46.1897

demonstrates the outcomes of the proposed process for a selection of trialed images.

Table 2. The methodologies from the literature have been compared to a number of images.

Images	Moushmi et al. [18]		Aymaz et al. [4]		Li et al. [19]		Proposed Method
	RMSE	PSNR	RMSE	PSNR	RMSE	PSNR	RMSE	PSNR
Flower	X	X	4.99	76.22	7.84	X	4.14	41.99
Clock	4.51	X	5.85	40.5	X	X	3.14	43.20
Book	7.04	X	X	X	X	X	9.46	38.40
Lab	X	X	X	X	4.65	X	2.69	43.87

shows the RMSE and PSNR values, two subjective measurements, for a selection of multi-focus images. In Table 2, the proposed method is compared to methods in the literature that use subjective criteria. The letter ‘X’ indicates that image specifications for the item are not available. Images of a flowerpot, a clock, a bottle of Pepsi, a cameraman, a desk, a book, a lab, and a flower are used to compare the proposed method to those already in use. The most favourable results are highlighted here. These metrics are practiced to evaluate the resistance of the proposed method to distortion. Conclusions suggest the proposed method fares well in subjective evaluations.

3.2. Comparison of Clock Image Fusion with the Literature

The clock, depicted in Figure 9, is used to evaluate the first multi-focus image. In Figure 9, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 9b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. An image that is right-focused but not left-focused is one that is dominated by the right. The approach is put into practice, and the result is the all-in-focus image, as seen in Figure 9d. The following calculations are used to assess the effectiveness of the proposed methodology: E(F), AG_F, Q_E, CC, Q^AB/F, and SSIM. The evaluation of the proposed methodology is then compared to other strategies previously described in the literature. The comparison’s findings are displayed in

Table 3. The results and comparisons for clock image ([20]).

MIF-Methods	QE	AGF	QAB/F	E(F)	CC	SSIM
PA-DCPCNN	0.8538	6.0719	0.8968	7.3854	0.9808	0.9031
Proposed Method	0.9068	5.2647	0.9874	7.4149	0.9811	0.9875

of the report. The proposed method outperforms those approaches, according to the literature [20], and the best results of the methods are highlighted.

3.3. Comparison of Desk Image Fusion with the Literature

The desk, depicted in Figure 10, is used to evaluate the second multi-focus image. In Figure 10, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 10b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. A right-focused image has only its right side considered; the left side has not been considered. Following a successful strategy implementation, Figure 10d depicts the procedure for producing the entire in-focus image. The following parameters are computed to measure the effectiveness of the given technique: E(F), AG_F, CC, Q^AB/F, SSIM, and Q_E. The proposed strategy’s performance is then compared to other approaches already published in the literature. The comparison’s findings are displayed in

Table 4. The results and comparisons for desk image ([20]).

MIF-Methods	QE	QAB/F	E(F)	CC	AGF	SSIM
PA-DCPCNN	0.8669	0.8958	7.346	0.9644	8.215	0.8693
Proposed Method	0.9284	0.9960	7.3534	0.9626	8.1919	0.9895

of the report. The best results of methods are highlighted in bold, and the literature [20] indicates that the proposed strategy is more effective than existing methods.

3.4. Comparison of Book Image Fusion with the Literature

The book, depicted in Figure 11, is used to evaluate the third multi-focus image. In Figure 11, in (a), the original image is depicted. Images with left and right focus are shown in Figure 11b,c, respectively. The phrase “left focused image” describes an image whose left side is focused but not right. While the left of the image is out of focus, the right side is. The method’s creation of the all-in-focus image is depicted in Figure 11d after it has been successfully applied. The following parameters are computed to assess the efficiency of the proposed methodology: E(F), AG_F, SSIM, CC, Q^AB/F, and Q_E. Finally, the effectiveness of the proposed strategy is contrasted with other approaches previously reported in the literature.

Table 5. The results and comparisons for book image ([20]).

MIF-Methods	SSIM	QAB/F	AGF	CC	E(F)	QE
PA-DCPCNN	0.9539	0.9145	13.7059	0.9825	7.2957	0.8838
Proposed Method	0.9850	0.9339	12.3380	0.9898	7.4138	0.9196

of the report includes the comparison results. The proposed method is more effective than most of the other methods according to the literature [20] and the best results of the methods are noted in bold.

3.5. Comparison of Flower Image Fusion with the Literature

The flower in Figure 12 is used to evaluate the fourth multi-focus image. In Figure 12, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 12b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. An image that is right-focused but not left-focused is one that is dominated by the right. The approach is put into practice, and the result is the all-in-focus image, as seen in Figure 12d. To evaluate the performance of the proposed methodology, calculations are made for SSIM, E(F), CC, AG_F, Q^AB/F, and Q_E. The performance of the proposed strategy is then contrasted with that of other approaches previously reported in the literature. The comparison’s findings are shown in

Table 6. The results and comparisons for flower image ([20]).

MIF-Methods	QE	QAB/F	E(F)	CC	AGF	SSIM
PA-DCPCNN	0.8617	0.8869	7.2212	0.9687	14.3156	0.9477
Proposed Method	0.9069	0.9970	7.2059	0.9651	14.3306	0.9907

of the report. The best outcomes are given in bold, and the research [20] indicates that the proposed strategy is more effective than the alternatives.

3.6. Comparison of Lab Image Fusion with the Literature

The lab, depicted in Figure 13, is used to evaluate the sixth multi-focus image. In Figure 13, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 13b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image.” A right-focused image has only its right side considered; the left side has not been taken into consideration. After the technique has been used effectively, Figure 13d illustrates the process of producing the all-in-focus image. The following parameters—E(F), Q_E, AG_F, SSIM, CC, and Q^AB/F—are computed to assess the proposed technique performance. The performance of the proposed strategy is then contrasted with that of other approaches previously reported in the literature. The comparison’s findings are displayed in

Table 7. The results and comparisons for lab image ([20]).

MIF-Methods	SSIM	CC	QAB/F	AGF	E(F)	QE
PA-DCPCNN	0.9122	0.9791	0.8996	6.6468	7.1178	0.8675
Proposed Method	0.9922	0.9776	0.9962	6.6539	7.1336	0.9261

of the report. The best results are highlighted in bold, and the literature [20] indicates that the proposed strategy is more effective than existing methods.

3.7. Comparison of Leaf Image Fusion with the Literature

In Figure 14, a leaf is used to evaluate the sixth multi-focus image. In Figure 14, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 14b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. The image is concentrated on its right side but not on its left. Following a successful strategy implementation, Figure 14d depicts the procedure for producing the entire in-focus image. To evaluate the performance of the proposed technique, the following parameters are computed: Q_E, E(F), SSIM, AG_F, Q^AB/F, and CC. The proposed strategy’s performance is then compared to other approaches already published in the literature. The comparison’s findings are displayed in

Table 8. The results and comparisons for leaf image ([20]).

MIF-Methods	QE	CC	E(F)	QAB/F	AGF	SSIM
PA-DCPCNN	0.8122	0.9366	7.4064	0.8874	19.1755	0.7587
Proposed Method	0.8478	0.9237	7.4216	0.9745	25.6660	0.8225

of the report. The best results are highlighted in bold, and the literature [20] indicates that the proposed strategy is more effective than existing methods.

3.8. Comparison of Pepsi Image Fusion with the Literature

Pepsi, which is depicted in Figure 15, is the seventh multi-focus image that is being evaluated. In Figure 15, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 15b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. The right dominates an image that is right-focused but not left-focused. The approach is put into practice, and the result is the all-in-focus image, as seen in Figure 15d. The proposed methodology’s performance is evaluated using the calculations AG_F, and Q^AB/F. The proposed strategy’s performance is then compared to that of other approaches already published in the literature. The comparison’s findings are displayed in

Table 9. The results and comparisons for Pepsi image ([4]).

MIF-Methods	QAB/F	AG
SWT-PCA-without SR	0.81	6.01
SWT-PCA with SR	0.93	15.06
Proposed Method	0.99	8.12

of the report. The best results are highlighted in bold, and the literature [4] indicates that the proposed strategy is more effective than existing methods.

3.9. Comparison of Flowerpot Image Fusion with the Literature

The flowerpot, depicted in Figure 16, is the ninth multi-focus image being evaluated. In Figure 16, the original image is depicted in (a). Images with a left focus and a right focus, respectively, are shown in Figure 16b,c. Left-side focus is referred to as a “left focused image”, while the right-side focus is referred to as a “right unfocused image”. A right-focused image has only its right side considered; the left side has not been taken into consideration. After the technique has been used effectively, Figure 16d illustrates the process of producing the all-in-focus image. The proposed methodology’s performance is evaluated using the following parameters: Q_W and Q_E. The proposed strategy’s performance is then compared to that of other approaches already published in the literature. In

Table 10. The results and comparisons for flowerpot image ([21]).

MIF-Methods	Qw	QE
CNN-DWT Domain	0.9281	0.8748
Proposed Method	0.9211	0.9163

of the report, the comparison’s outcomes are displayed. The best results are highlighted in bold, and the literature [21] indicates that the proposed strategy is more successful than the existing methods.

3.10. Comparison on More Multi-Focus Image Pairs

Optimal subjective and objective results cannot be achieved with a single method for all possible image pairs. In order to demonstrate the average performance of different methods, the flowing experiment uses eight and ten multi-focus image pairings, respectively (Figure 17 and Figure 18). Image pairs shown in Figure 17 and Figure 18 were processed using the proposed method to create the fused images shown in Figure 19 and Figure 20. As can be seen in Figure 19 and Figure 20, the proposed technique yields satisfactory fusion results for all the examined image pairs.

Table 11. The average objective evaluation of different techniques on multi-focus image pairs ([20]).

Evaluation Metric	QE	QAB/F	AGF	CC	E(F)	SSIM
PA-DCPCNN	0.8336	0.898	15.1203	0.9736	7.2766	0.872
Proposed Method	0.9098	0.9603	16.2755	0.97	7.2607	0.9755

and

Table 12. The average objective evaluation of different techniques on multi-focus image pairs ([22]).

MIF-Methods	QE	AG	Q0	EI	QAB/F	GM	ID	SD
DTCWT—CSR	0.7433	7.6828	0.8152	77.7976	0.7745	103.8605	9.7117	52.0522
CVT—CSR	0.712	7.6924	0.8051	77.8838	0.742	104.4969	9.732	52.1355
NSCT—CSR	0.7186	7.6807	0.8327	77.7429	0.7571	104.6126	9.7074	52.2404
DWT—CSR	0.6941	7.7793	0.8168	78.6239	0.736	104.4005	9.8798	52.323
Proposed Method	0.9234	12.6504	0.9202	67.7652	0.9891	104.1459	8.6857	52.9553

provide the average objective evaluation by various methods for the image pairs in Figure 17 and Figure 18. Table 11 and Table 12 display the comparison results. Compared to other methods detailed in the literature [20,22], the proposed method achieves better results, and the most beneficial aspects of each method are highlighted in bold.

3.11. Test on a Medical Image Pair

In the medical image experiment, we used one image pair from medical image fusion trial to determine how well these methods generalize to other image fusion use cases. In

Table 13. The results and comparisons for medical image ([20]).

MIF-Methods	E(F)	CC	AGF	SSIM	QE	QAB/F
PA-DCPCNN	6.4646	0.9255	14.6353	0.7027	0.813	0.8704
Proposed Method	4.9199	0.9102	14.9477	0.9601	0.8296	0.8652

, the fusion results for the “Medical” image pairings are displayed, while in Figure 21, the corresponding findings for the “Medical” image pairs are visualized. The parameters SSIM, E(F), CC, AG_F, Q^AB/F, and Q_E are calculated to assess the effectiveness of the proposed methodology. Finally, we compare the proposed method to those already described in the literature. Table 13 presents the comparison results for the report. The best results of each methodology are highlighted in bold, supporting the conclusion that the proposed strategy is superior to the alternatives presented in the literature [20].

3.12. Test on a Remote Sensing Image Pair

In the remote sensing image experiment, we used one image pair from remote sensing image fusion trial to determine how well these methods generalize to other image fusion use cases. In

Table 14. The results and comparisons for remote sensing image ([20]).

MIF-Methods	E(F)	CC	AGF	SSIM	QE	QAB/F
PA-DCPCNN	7.195	0.5219	12.6765	0.6679	0.6085	0.8229
Proposed Method	7.2322	0.4575	12.6270	0.8009	0.7213	0.8988

, the fusion results for the “Remote sensing” image pairings are displayed, while in Figure 22, the corresponding findings for the “Remote sensing” image pairs are visualized. The parameters SSIM, E(F), CC, AG_F, Q^AB/F, and Q_E are calculated to assess the effectiveness of the proposed methodology. Finally, we compare the proposed method to those already described in the literature. Table 14 presents the comparison results for the report. The best results of each methodology are highlighted in bold, supporting the conclusion that the proposed strategy is superior to the alternatives presented in the literature [20].

4. Qualitative Comparison

Quantitative comparison is performed with E(F), Q_E, Q^AB/F, SSIM, CC, AGF Q⁰, SD, EI, ID, and GM. The entropy E(F) measures a fused image’s information content. When the entropy of the combined image is high, there is much information there. Figure 23 shows the value of this measure for several image fusion techniques. The average improvement of the proposed method concerning entropy is 0.39% with reference to PA-DCPCNN [20].

Q_E quantifies the edge-dependent fusion quality index of the fused image. A higher Q_E indicates a more effective fusion strategy. Figure 24 illustrates the significance of this statistic across several image fusion approaches. The average improvement of the proposed method concerning Q_E is 5.32% with reference to PA-DCPCNN [20].

The amount of edge information (Q^AB/F) carried by the fused image within the source images is calculated by total-fusion performance. A larger value of Q^AB/F is desired. Figure 25 illustrates this metric’s value for several image fusion techniques. The average improvement of the proposed method concerning Q^AB/F is 8.56% with reference to PA-DCPCNN [20].

The structural similarity index measure (SSIM) is used to evaluate the degree to which the reference and fused images have comparable pixel intensities patterns locally. If the value is 1, it means that the reference image and the fused image are analogous. Figure 26 illustrates the relative importance of this metric in comparison with other image fusion techniques. The average improvement of the proposed method concerning SSIM is 7.32% with reference to PA-DCPCNN [20].

The correlation coefficient (CC) measures the degree to which the reference and fused images have the same spectral characteristics. Higher values of CC mean that the fusion method is better. Figure 27 illustrates the relative importance of this metric in comparison with other image fusion techniques. For this parameter, the resultant average value is less than PA-DCPCNN because this indicates that variables move in the opposite direction [20].

The average gradient (AG_F) quantitatively represents the fused image’s detail and texture components. The fused image has increased sharpness and clarity proportional to the value of the AG_F. Figure 28 illustrates the relative performance of various image fusion techniques based on this metric’s value. The average improvement of the proposed method concerning AG_F is 5.95% with reference to PA-DCPCNN [20].

The evaluation of different fusion methods for different parameters based on their average values is depicted in Figure 29. Concerning Q_E, the proposed method is 19.50%, 22.89%, 22.17%, and 24.83% superior compared to methods 1, 2, 3, and 4, respectively. Concerning AG, the proposed method is 39.26%, 39.19%, 39.28%, and 38.50% superior compared to methods 1, 2, 3, and 4, respectively. Concerning Q⁰, the proposed method is 11.41%, 12.50%, 9.50%, and 11.23% superior compared to methods 1, 2, 3, and 4, respectively. Concerning Q^AB/F, the proposed method is 21.69%, 24.98%, 23.45%, and 25.58% superior compared to methods 1, 2, 3, and 4, respectively. Concerning GM, the proposed method is 0.27% superior compared to method 1. Concerning SD, the proposed method is 1.70%, 1.54%, 1.35%, and 1.19% superior compared to methods 1, 2, 3, and 4, respectively [22].

Computational Efficiency

All of the experiments are carried out in MATLAB R2018b on a computer equipped with an Intel(R)-Core (TM) i5-11320H CPU operating at 3.20 GHz and 16 gigabytes of random-access memory (RAM). Comparing the average amounts of time these various methods take to compute results gives an evaluation of the computational burdens imposed by both existing and proposed approaches.

Table 15. Average running time (in seconds) of different methods.

Image Pairs	NSCT-PCNN	IM	DSIFT	MST-SR	BF	BRW-TS	PA-DCPCNN	Proposed Method
Gray-Scale	201.4214	6.4078	16.0172	49.1306	1.7311	2.4192	153.6215	0.6790

contains information on the typical amount of time required to complete these approaches. Since the proposed method integrates SWT, DTCWT, and TEM, the amount of time needed to complete the computations is significantly reduced. As a result, the proposed method has a shorter amount of computing time than existing methods [20].

5. Conclusions

In this paper, TEM laws are used in the hybrid wavelet domain combination of SWT and DTCWT to overcome the drawbacks of the PA-DCPCNN and MST-CSR based methods. The proposed work is implemented on standard images. As a result of using a hybrid algorithm with TEM, the proposed fused image can achieve higher visual quality when compared to input images because of its good directionality and a high degree of shift invariance. It is compared with recently published image fusion methods: PA-DCPCNN, and MST-CSR (i.e., DWT-CSR, DTCWT-CSR, CVT-CSR, and NSCT-CSR). The quantitative comparison with fusion quality metrics shows that the proposed method is better than the state-of-the-art methodologies. The detailed representation of the fusion outcome shows that the proposed method gives more information. The use of the fused images makes it easier to perform operations and understand the results. Further, the proposed method may combine deep learning algorithms with different colour spaces.