Low-Light Image Enhancement Method Based on Retinex Theory by Improving Illumination Map

Abstract

Recently, low-light image enhancement has attracted much attention. However, some problems still exist. For instance, sometimes dark regions are not fully improved, but bright regions near the light source or auxiliary light source are overexposed. To address these problems, a retinex based method that strengthens the illumination map is proposed, which utilizes a brightness enhancement function (BEF) that is a weighted sum of the Sigmoid function cascading by Gamma correction (GC) and Sine function, and an improved adaptive contrast enhancement (IACE) to enhance the estimated illumination map through multi-scale fusion. Specifically, firstly, the illumination map is obtained according to retinex theory via the weighted sum method, which considers neighborhood information. Then, the Gaussian Laplacian pyramid is used to fuse two input images that are derived by BEF and IACE, so that it can improve brightness and contrast of the illuminance component acquired above. Finally, the adjusted illuminance map is multiplied by the reflection map to obtain the enhanced image according to the retinex theory. Extensive experiments show that our method has better results in subjective vision and quantitative index evaluation compared with other state-of-the-art methods.

1. Introduction

With the rapid development of modern computer and multimedia information technology, digital image processing systems have been widely used in industrial production [1], scientific research [2], intelligent transportation [3], and other fields. However, the process of image acquisition is complex and affected by many factors, resulting in various image defects. Insufficient lighting conditions, such as indoor, cloudy, or night conditions, often lead to a serious decline in image quality [4].

Researchers have pursued various improvements from both the hardware and software perspectives. The former is the development of image acquisition equipment [5,6], such as using professional low light level cameras (e.g., Sony, SiOnyx, and Photonis), but they are expensive and not suitable for daily life. The latter is to improve the collected image through digital image processing. The main research direction of this paper was software-based low-light image enhancement.

Enhancing the visual quality of images captured under weak light conditions is a difficult problem. These images usually appear with low contrast, low resolution, high noise, and over-exposure, due to insufficient light and gain, which affect our visual effects seriously. Consequently, low-light image enhancement technology, which aims to improve the overall brightness, local contrast, and detail information of raw images, has been widely studied [7,8,9,10].

Directly increasing the image brightness is the most intuitive and simplest method. Typical algorithms use histogram equalization (HE) and grayscale transformation, which enhance pixel values through mathematical functions. HE and its derivative algorithms [11,12,13,14,15] are widely used due to their simplicity and effectiveness. Grayscale transformation uses a mathematical function to change grayscale values, and gamma correction (GC) is one of the most widely used methods [16,17,18]. These methods enhance the entire image to the same degree regardless of whether the area is dark or not, which easily causes overexposure. Retinex theory [19,20,21,22,23,24,25] assumes that an image can be decomposed into an illumination component and a reflection component, and the latter is used as the final enhanced image. Single-scale retinex (SSR) [20], multi-scale retinex (MSR) [21], and multi-scale retinex with color restoration (MSRCR) [22] are typical algorithms based on retinex theory. However, these methods enhance the image by eliminating illumination, resulting in loss of information.

Image fusion-based enhancement algorithms have attracted the attention of scholars because they do not require any physical model but only fuse useful information from multiple images [26,27,28,29]. Fu et al. [27] introduced a fusion method based on a variety of mature image proposing technologies to improve image quality (MF). Unlike the classical retinex method, this method improves shimmering images by maintaining and enhancing image illumination. However, for the enhancement of image brightness, an inverse tangent function is still used, and the enhancement degree of global brightness is the same, leading to a certain degree of overexposure. Dai et al. [28] proposed a fractional-order mask and the fusion framework model (FFM) for low-light image enhancement. However, the model produces an overall darker and less visually appealing result.

Recently, deep learning-based methods have been proposed, which extract numerous image features from datasets and hence can effectively improve image quality [30,31,32,33]. Ren et al. [30] proposed a trainable hybrid network that consists of two distinct streams to obtain a clear image. Park et al. [32] present a dual autoencoder network model to enhance low-light images and reduce noise by combining the stacked and convolutional autoencoders. Chen et al. [33] designed a so-called Retinex-Net (Ret-Net) network, which includes a decom-net for decomposition and an enhancement-net for illumination adjustment. However, these algorithms rely on many natural images for training the network, heavily depend on the number of input images and are computationally complex and time-consuming.

Although many researchers have proposed numerous low-light image enhancement algorithms, several problems still need to be addressed. For instance, some methods may not be able to completely extract contents at very dark regions, some may pose the risk of excessive enhancement, and some may blur detail information.

In this paper, we propose a retinex theory-based algorithm to enhance images well and solve the above problems. The proposed method enhances the bright and dark areas to varying degrees and comprises three parts: (1) estimation of illumination component and reflection component from raw image; (2) blending two input images obtained by BEF and IACE to enhance the illumination component by Gaussian Laplacian pyramid; and (3) deriving the final enhanced image based on retinex theory. The general process of the proposed model is shown in Figure 1, and the main innovations are marked in red font and explained in detail in Section 3.2.

Our main contributions can be summarized as follows.

(1)

We designed a neighborhood-weighted illuminance map estimation algorithm by considering the correlation among pixels. The estimated illuminance map carries important information about objects’ structure in the visual scene.

(2)

We designed two functions to acquire input images for fusion, including a brightness enhancement function (BEF) and an improved adaptive contrast enhancement (IACE). The BEF can not only highlight the contents within dark areas but also retain the details within bright areas, and the IACE can enhance contrast while avoiding noise amplification.

(3)

We found that significant details exist in regions with large gradients, and large variances are rich in information. Hence, we designed a detail weight for the fusion procedure by considering the gradient and variance of local regions.

(4)

Compared with previous algorithms, the proposed method has several advantages. Firstly, the brightness of the illumination map is differentially enhanced (i.e., the light and dark areas of the image are enhanced to different degrees), so that not only is the content of the dark areas displayed, but also the details of the bright areas are not blurred. Then, the high-frequency component in contrast enhancement tends to amplify the noise, and in this paper, the image is enhanced in the form of a piecewise function to reduce noise. Finally, the weight based on gradient and variance enriches the image details.

The rest of this paper is organized as follows. Section 2 discusses the relevant work. The proposed method is detailed in Section 3. The performance of the proposed method is evaluated both quantitatively and qualitatively in Section 4. Section 5 highlights some limitations of the proposed work and future research directions. Section 6 concludes the paper.

2. Related Works

In this section, we focus on reviewing and analyzing some basic contents required for the proposed model. Specifically, we first introduce the retinex theory, then give the formula and corresponding curve graphs used in the two fused images, and finally explain the importance of gradient and variance for designing the weight in the subsequent work.

2.1. Retinex Theory

Retinex theory [19] divides an image into a reflection component and an illumination component:

$$S^c=R^c\times L,$$

(1)

where $S$, $R$ and $L$ are original image and reflection image and illumination image, respectively; and $c$ denotes three color channels of RGB (red, green, blue). The algorithm based on retinex theory aims to enhance the illuminance component to improve the quality of low-light images [27].

2.2. Gamma Correction, Sigmoid Function and Sine Function

GC and sigmoid function are classical methods to enhance image brightness. The formula for GC is

$$E=\sqrt[\gamma ]{S},$$

(2)

where $E$ is the enhanced image and $\gamma$ is a constant. The current and subsequent are normalized images taking values between [0, 1]. sigmoid function hardly changes outside the interval [−10, 10], but the pixel values of the images are all greater than 0. Therefore, we intercept the definition domain of the sigmoid function between [0, 10] and adjust the formula accordingly, the final formula is shown below:

$$E=\frac{1}{1+e^{-10\times S}}.$$

(3)

In order to significantly improve the brightness of the image, we need to intercept the part of the sigmoid function within [0, 1] and further normalize it. Hence, the specific formula becomes:

$$E_1=\left(\frac{1}{1+e^{-10\times S}}-0.5\right)\times 2.$$

(4)

2.3. Correlation Study of Gradient and Variance

The gradient of an image measures the change rate of pixels in the horizontal and vertical directions and is commonly used to provide more detailed information for edge detection and reconstruction [34,35,36,37,38,39]. The variance reflects the high-frequency component. The larger the variance, the better the visual effect is, and the easier different objects are to distinguish.

Figure 2 shows two images, either of which is divided into 36 small regions of the same size. The number in Figure 2b is the corresponding number of each region. Figure 2c illustrates the regions with the maximum gradient ($Max\_g$) and the minimum gradient ($Min\_g$) and the maximum ($Max\_v$) and minimum variance ($Min\_v$). From Figure 2, it can be seen that the regions with $Max\_g$ and $Max\_v$ contain rich information, while the regions with $Min\_g$ and $Min\_v$ only have plain contents. So, the regions with $Max\_g$ and $Max\_v$ are vital for detail processing.

3. The Proposed Algorithm

The objective of the proposed method is the enhancement of a low-light image by enhancing the brightness and contrast of the illuminance map. Firstly, a neighborhood-based approach is proposed to estimate the illuminance component, and then obtain the reflection component through the retinex theory. Secondly, differential enhancement is applied to each region in the illuminance component, that is, the luminance is substantially increased in darker regions and hardly enhanced in brighter regions. Contrast enhancement uses the form of piecewise function to update the image pixel by pixel to reduce noise. Then, the two enhanced images are used to obtain the adjusted illumination map using the Gaussian–Laplacian pyramid fusion algorithm. Finally, the improved illumination map is multiplied by the reflection component to obtain the final enhancement result. Figure 3 shows the flowchart of the algorithm in this paper. As follows, we introduce our method in this section with all necessary details.

3.1. Estimation of Illumination Component and Reflection Component

In this section, an illuminance map estimation (IME) algorithm is proposed based on the neighborhood information of pixels. Specifically, the pixel’s value is updated using a weighted sum with the neighboring pixels, followed by the morphological closure algorithm and guided filter for refinement. The specific steps are as follows.

Let $N_i$ be the i-th pixel’s neighborhood, $M_i$ the maximum value of R, G, and B at the position $i$, j the pixel’s position inside $N_i$, $w_j$ the weight value of position j. The neighborhood-based illuminance map is derived from:

$$I_i=0.5\times M_i+0.5\times {\displaystyle {\sum }_j(N_i}\left\{j\right\}\times w_j).$$

(5)

Considering the fact that the greater the difference among pixel values, the lower the weight that should be constructed, or conversely, the greater the weight value at the position $j$, the smaller the difference between $N_i$ and $M_i$ is, we designed the following weight:

$$w_j=\{\begin{array}{l}\frac{1}{\left|N_i\left\{j\right\}-M_i\right|}, if \left|N_i-M_i\right|\ne 0 \\ \min \left(M+0.1,1\right) , else\end{array},$$

(6)

where $M$ is the maximum weight in the neighborhood. We normalize the weights to obtain the final $w_j$. The above idea is similar to the inverse distance weight (IDW) method [40,41,42].

Then, we use the morphological closure algorithm [27,43] and Guided filter [44] to refine the illumination map. The neighborhood-based illuminance map from Equation (5) is preliminarily refined by:

$$I_{mor}=\frac{I•P}{255},$$

(7)

where $P$ denotes the structuring element and $•$ is the closing operation. Dividing by 255 is used to map the range to [0, 1] for the downstream operations. The final refined illumination map is obtained by:

$$I_{end}={\displaystyle {\sum }_{j\in {\omega }_i}{W}_{\left\{i,j\right\}}\left(g\right)\times I_{mor}\left(m,n\right)},$$

(8)

where $g$ is the value component of the guidance image in HSV space and $W$ is the filter kernel:

$$W_{\left\{i,j\right\}}\left(g\right)=\frac{1}{{\left|{\omega }_i\right|}^2}{\displaystyle {\sum }_{\left\{i,j\right\}\in {\omega }_i}\left(1+\frac{\left(g_i-m_i\right)\left(g_j-m_i\right)}{{\sigma }_i^2+\epsilon }\right)}.$$

(9)

Here ${\omega }_i$ is the window centered at the i-th pixel and $\left|{\omega }_i\right|$ is the number of pixels within ${\omega }_i$; $m_i$ and ${\sigma }_i^2$ are the mean and variance of guidance image within ${\omega }_i$, respectively, and $\epsilon$ is a regularization parameter.

According to Equation (1), the reflection component is obtained by:

$$R=\exp \left(\log S-\log I_{end}\right)$$

(10)

3.2. Illumination Map Enhancement Based on Gaussian Laplacian Pyramid Fusion

The initial illumination component and reflection component were obtained in Section 3.1. This section mainly improves the illumination component in terms of both brightness and contrast. For the problem of image brightness enhancement, the existing image enhancement algorithms have limited ability to enhance the darker regions. These include methods such as the bio-inspired multi-exposure fusion framework (BIMEF) [45], MF, structure-revealing low-light image enhancement via robust retinex model (SR) [46], and low-light image enhancement with semi-decoupled decomposition (SDD) [47], with results shown in Figure 4. The brighter areas are over-enhanced, resulting in blurred details with some methods, such as BIMEF, low-light image enhancement via illumination map estimation (LIME) [48], low-light image enhancement using the camera response model (LECARM) [49], and SDD, with results shown in Figure 5.

To solve the above problems, we designed the BEF function in two directions, as detailed in Section 3.2.1. The adaptive contrast enhancement (ACE) method improves the contrast by combining the high-frequency information of the image, but the high-frequency component is often doped with noise. Therefore, we devised a form of piecewise function to improve the image contrast pixel by pixel, as detailed in Section 3.2.2. The design of the fusion weight and Gaussian Laplacian pyramid are explained in Section 3.2.3 and Section 3.2.4, respectively.

3.2.1. Brightness Enhancement of Illuminance Map

This section designs the BEF function by analyzing several brightness enhancement functions given in Section 2.2. Figure 6 shows two graphs for the different transformation functions, and the values of Equations (2) and (4) at any point are higher than the identity line, indicating that the pixel value at any point of the enhanced image is higher than that of the original image.

Figure 7 displays one raw low-light image and its counterparts enhanced by different transform functions. Both GC (2) and truncated normalized sigmoid (4) can enhance the brightness of the image. However, Equation (4) has limited enhancement ability for dark regions marked by red boxes, as shown in Figure 7c. Observing the curves of each formula in Figure 6a, it is found that GC (2) has the greatest enhancement in the region of low pixel value. Therefore, we can hopefully enhance the brightness details simultaneously for dark regions by cascading GC (2) and sigmoid (4) as follows:

$$y_1=\sqrt[\gamma ]{E_1}.$$

(11)

Figure 7d shows the image enhanced by Equation (11), which is better than Equations (2) and (4). However, for the window area in the red box of the image, Equation (11) produces overexposure, resulting in complete loss of image content. This is due to the fact that in the high-pixel value region, the curve of Equation (11) is much higher than the identity line in the high-pixel value region, as shown in Figure 6a. Inspired by the characteristics of the sine function (SF), in which the values are greater than 0 between [0, 0.5] and less than 0 between [0.5, 1], as shown in Figure 6b, we accordingly considered adding the SF function to the original image to suppress the bright areas in the image. We designed the following transform function:

$$s_i=S_i+SF\left(S_i\right),$$

(12)

where

$$SF\left(S_i\right)=\sin \left(S_i\times 2\pi \right)/2\pi$$

It is obvious that we have $SF\left(S_i\right)\ge 0$ for $0\le S_i\le 0.5$, while $SF\left(S_i\right)\le 0$ for $0.5\le S_i\le 1$.

Obviously, the enhancement result of Equation (12) can retain more details as shown in Figure 8a, but the whole image is dark. In order to further improve the visual quality, we combine Equations (11) and (12) to obtain the first image for fusion:

$$I_1=0.5\times \left(y_1+s\right).$$

(13)

So far, we have completed the luminance improvement of the illuminance map through Equation (13).

The corresponding plot of Equation (13) is shown in Figure 6a, where the entire curve is almost always above the identity line with the first part of the curve being significantly higher than the identity line, due to the need for greater enhancement in the low pixel value region. However, it almost coincides with the line near 1, indicating that there is almost no enhancement here, so that over-enhancement can be avoided. Figure 8b shows the enhanced result with Equation (13). As can be seen, the resulting image not only improved the brightness, but also was not overexposed. It is clearly better than that obtained with Equation (12).

3.2.2. Contrast Enhancement of Illuminance Map

Improving image contrast is also an important step to enhance low-illumination images. The adaptive contrast enhancement algorithm (ACE) to enhance the high-frequency component is a classical method [50]. The specific formula is:

$$I_i=m_i+G_i\left(S_i-m_i\right) ,$$

(14)

where $S_i$ and $I_i$ are the i-th pixel of the original image and the corresponding enhanced image.

$$G_i=alpha\times \left(m_i/{\sigma }_i^2\right), 0<alpha<1$$

where $m_i$ and ${\sigma }_i^2$ are the mean and variance of the local windows ${\omega }_i$, respectively. We set $alpha=0.3$ in this paper.

As is known to all, the noise is regarded as a point whose pixel value is significantly different from other pixels in the local area, which belongs to the high-frequency part. Therefore, the ACE algorithm can easily amplify the noise while improving contrast. To solve this problem, we sought to determine whether any pixel is noisy or not and then update its value in two corresponding ways. Specifically, when a pixel is not a noisy point, its value is still updated according to ACE (14), and conversely, its value is replaced with the mean and variance of the local area after removing all noisy points. The whole updating rule for improved ACE (IACE) is as follows:

$$I_{\mathrm{i}}=\{\begin{array}{l}{m}_i+G_i\left(S_i-m_i\right) , if S_i is a non-nosiy point \\ {\mu }_i+v_i , else\end{array},$$

(15)

where ${\mu }_i$ and $v_i$ are the mean and variance of the i-th window ${\omega }_i$ after removing noisy points, respectively. The mean ${\mu }_i$ ensures the overall information of the local area, and the variance $v_i$ takes into account the difference information between pixels This simple criterion maintains the difference value between pixels and does not blur the image. The final image obtained from (15) is the second input image ($I_2$) for fusion, where $n_1=\#\left\{\left|S_i-S_j\right|\le 0.05\right\}, S_j\in {\omega }_i$, $\#$ represents the number of elements in the set. For the local area with the size of $3\times 3$, the experiment shows that if $n_1\le n_2$ is defined as the noise point, the experimental effect is the best. Of course, $n_2$ can be adjusted according to the size of the local area; in this paper, $n_2=3$.

Figure 9 shows the contrast enhancement results achieved with Equations (14) and (15). For the noisy image shown in the first row, we can see that IACE can eliminate most of noise compared to ACE. For the noise-free images shown in the second row, we can see that IACE can improve contrast and sharpness. In other words, IACE can significantly improve contrast and eliminate noise at the same time.

3.2.3. Design of Fusion Weights

In order to combine the advantages of the two images, the Gaussian–Laplacian pyramid algorithm is used to fuse the two input images obtained above, and the fusion weights are composed of brightness weight and detail weight. This section mainly introduces two kinds of weights.

In order to have a natural brightness, we adopt a global brightness weight for the k-th input image as in [27]:

$$W_{k,1}=\exp \left(-\frac{{\left(I_k-0.5\right)}^2}{2\times {0.25}^2}\right),$$

(16)

As pointed out in Section 2, the gradient and variance are important for image details. In order for the enhanced image to contain rich content, the detail weight is designed as follows:

$$W_{k,2}=\lambda \frac{\ln \left(1+10\times a_k\right)}{\ln 10},$$

(17)

$\lambda$ is the normalization factor, where

$$a_k={\sigma }_k+gra{d}_k,$$

(18)

$$gra{d}_k=\left|{\nabla }_x{I}_k\right|+\left|{\nabla }_y{I}_k\right|,$$

(19)

where ${\nabla }_x$ and ${\nabla }_y$ are the gradients of the local patch in $x$ direction and $y$ direction, respectively, and ${\sigma }_k$ is the variance of the local region. $a_k$ is the summation of the variance and the gradient, indicating the amount of information in the local area. To ensure that the value of $W_{k,2}$ is between [0, 1], $a_k$ needs to be normalized. So, the larger the $a_k$, the higher the weight value and information contained in the image.

The final weight is obtained by:

$${\overline{W}}_k=\frac{W_k}{{\displaystyle {\sum }_k{W}_k}},$$

(20)

where

$$W_k=W_{k,1}+W_{k,2}$$

(21)

3.2.4. Gaussian Laplacian Pyramid Fusion

In order to obtain a refined illumination map, we use the Gaussian–Laplacian Pyramid method [51] to fuse two input images obtained from Equations (13) and (15) with the fusion weights from Equation (20). We decompose either input image into a Laplacian pyramid to extract image features and decompose either normalized weight ${\overline{W}}_k$ into a Gaussian pyramid to smooth strong transitions [27]. Then, we fuse these pyramids by mixing each layer, and the final enhanced illumination map is obtained by summing up-sampling on the fused pyramids:

$$I_{final}={\displaystyle {\sum }_l{U}_d\left({\displaystyle {\sum }_{k=1}^2{G}_l\left\{{\overline{W}}_k\right\}\times L_l\left\{I_k\right\}}\right)},$$

(22)

where $U_d$ is the up-sampling operator with factor $d=2^l-1$ ($l$ is the number of layers of the pyramid; we select $l=6$), and $L_l\{·\}$ and $G_l\{·\}$ are the Laplacian decomposition and Gaussian decomposition, respectively.

3.3. Image Enhancement Based on the Retinex Theory

According to retinex theory, the adjusted illumination map is multiplied by the reflection component from Equation (10) to obtain the final enhanced image:

$$S_{en}=R\times I_{final} .$$

(23)

4. Experiments and Results

In this section, we verified the performance of the proposed method. All experiments were conducted using Matlab R2018a on a PC with a 3.60 GHz Intel Pentium Dual Core Processor and 16.0 G RAM.

We performed qualitative and quantitative comparisons with the state-of-the-art algorithms. They included BIMEF [45], MF [27], LIME [48], SR [46], Ret-Net, FFM [28], LECARM [49], and SDD [47]. Several datasets such as MF [27], Fusion [29], LOL dataset [33], LIME [48], NPE [52], DICM [53], Synthetic dataset [33,54], and SRIE [55].

For evaluating the proposed method’s performance, five no reference image quality assessment (NIQA) metrics, including the natural image quality evaluator (NIQE) [56], the blind/reference-less image spatial quality evaluator (BRISQUE) [57], the colorfulness-based patch-based contrast quality index (CPCQI) [58], the integrated local NIQE (IL-NIQE) [59], and the no-reference free energy-based robust metric for contrast distortion (NIQMC) [60], and one full reference IQA metric, that is, the patch-based contrast quality index (PCQI) [61], were used for quantitative evaluation.

4.1. Subjective Evaluation of Experimental Results

In this section, we analyzed the effectiveness of the proposed method from a subjective perspective for the darker and lighter areas of the image. As shown in Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15, six images, i.e., “Bridge”, “House”, “Window and Eave”, “Plant and Shadow”, “Tree”, and “Town” were used to illustrate that the proposed method can commendably reveal dark areas. As shown in Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21 and Figure 22, another seven images, i.e., “Girl”, “River”, “Shoes”, “Street”, “Window curtain”, “Hand”, and “Church” were used to illustrate that the proposed method does not produce overexposure, while increasing overall brightness and retaining more details.

4.1.1. Testing the Effectiveness of the Proposed Method in Dark Regions

The overall pixel values of shimmering images were small, and there were often completely dark areas in the images, such as the bridge in Figure 10, the wooden stakes in the corner of the house in Figure 11, the eave in Figure 12, and so on. As shown in Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15, all methods improved the brightness of the image to a certain extent, but the red box areas of the image were enhanced differently.

For instances, for the red frame region in Figure 11, some methods (i.e., MF, LIME, FFM, and LECARM) had less visible enhancement, while some methods (i.e., BIMEF, SR, and SDD) had little or no effect. The dark region enhanced by Ret-Net could be clearly seen, but its details were also blurred. Our proposed method had a significant effect for recognition of the wooden stake. This was because addition of GC to the BEF function improved the enhancement of dark areas. Similarly for Figure 15, for the roof in the red frame, some methods (e.g., BIMEF, SR, Ret-Net, FFM, SDD) blurred details, whereas LIME and LECARM led to different degrees of over-enhancement. The enhancement effects of MF and the proposed method were obviously better than those of the other algorithms. However, in the area of the square box, the proposed method showed relatively clear content.

4.1.2. Testing the Effectiveness of the Proposed Method in Bright Regions

For the highlighted areas close to the light sources in the images, the pixel value was generally high. Examples include the girl’s dress in Figure 16, the river in Figure 17, the hand in Figure 21, etc. These areas were inherently highly brightness and did not require much enhancement. Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21 and Figure 22 show the enhancement results of all comparison algorithms and the proposed method on different images. For high pixel value areas, each comparison algorithm caused a different degree of blurring, while the proposed model retained more complete information.

For example, in Figure 16, for the skirt in the red box area, BIMEF, MF, LIME, SR, FFM, LECARM and SDD adopted the same strategy for all areas when enhancing the image, regardless of the pixel values of the image itself, resulting in excessive exposure and blurred image details. The BEF transform function of our proposed method enhanced the brighter region slowly or not, and was able to improve the image brightness while maintaining the detail information, so the content in the region could be preserved completely. In Figure 22, the window above the church had a high pixel value and, therefore, did not require much enhancement. However, BIMEF, MF, LIME, SR, FFM, LECARM and SDD were enhanced for every pixel value of the whole image, thus losing some information. The method proposed in this paper retained the original content of the window area to a large extent.

4.2. Objective Quality Assessments

To quantitatively evaluate the performance of the proposed method, we performed a full-reference evaluation and a non-reference evaluation. Specifically, we used 104 images from different datasets for comparative experiments. These images were composed of two parts, a total of 13 images from Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21 and Figure 22 and 91 other images, respectively. The average values of the corresponding quantitative indicators are shown in

Table 1. Average value of six quantitative metrics of 104 images under different algorithms.

	NIQE (↓)	BRISQUE (↓)	CPCQI (↑)	IL-NIQE (↓)	NIQMC (↑)	PCQI (↑)
BIMEF [45]	4.242	19.686	1.030	27.523	4.820	1.036
MF [27]	4.346	20.181	1.078	26.076	5.072	1.079
LIME [48]	4.621	23.323	1.007	26.783	5.472	1.011
SR [46]	5.798	35.398	0.903	33.877	4.839	0.919
Ret-Net [33]	5.086	24.126	0.784	24.993	4.660	0.802
FFM [28]	4.436	23.720	0.980	28.030	4.508	0.986
LECARM [49]	4.255	18.865	1.059	27.003	5.118	1.061
SDD [47]	5.115	30.359	0.985	30.464	4.893	1.000
Proposed	4.192	19.140	1.080	25.916	4.913	1.080

, and the best indicators are shown in bold. Where, (↑) means that the larger the value, the better, and (↓) is the opposite.

(1) Non-Reference Evaluation: We employed five non-reference metrics, that is, NIQE, BRISQUE, CPCQI, IL-NIQE, and NIQMC, for low-light image quality evaluation.

NIQE constructs a “quality-aware” collection of statistical features based on the Natural Scene Statistics (NSS) model in the spatial domain. BRISQUE uses NSS with locally normalized luminance coefficients to quantify the possible loss of “naturalness” in order to assess the overall image quality. CPCQI uses a regression module to generate visual quality metrics by analyzing 17 features such as contrast, sharpness, and luminance. IL-NIQE learns multivariate Gaussian models of image blocks by integrating statistical features from multiple natural images. Additionally, the Bhattacharyya-like distance is used to measure the quality of each image block, and the overall quality score is obtained by averaging pooling. NIQMC combines local and global determination of which image has greater contrast and better quality between two images. The lower the values of NIQE, BRISQUE, and IL-NIQE, the higher the image quality is, while higher scores for CPCQI and NIQMC mean that a better contrast is obtained than the reference image.

As shown in Table 1, the proposed method had the optimal NIQE and CPCQI values, with the BRISQUE value and IL-NIQE ranking second and NIQMC ranking fourth. This means that the results of the proposed method were more consistent with human vision, and the contrast was in the middle position.

(2) Full-Reference Evaluation: We conducted a full-reference evaluation using the PCQI metric. PCQI decomposes the image block into mean intensity, signal intensity and signal structure components and evaluates the perceived distortion of these components separately. A higher PCQI denotes that the result has higher visual effect and quality. As shown in Table 1, our proposed method stood out as the best performer.

This part verified the effectiveness of our method from two aspects: full reference and non-reference. Although not every index had the best results, the model outperformed other algorithms overall.

5. Limitations and Discussion

There are still some deficiencies in the model proposed in this paper. (1) When selecting the $\gamma$ value of Equation (11), we found that the larger the $\gamma$ was, the more obvious the dark area detail was, as shown in Figure 23. The quantitative index in Figure 24 almost reached the optimal value when $\gamma =1$, but it could not display the content of the dark area. Therefore, how to balance the selection of $\gamma$ and quantitative indicators is a problem to be studied in the future. (2) With the continuous development of deep learning, using a convolutional neural network to improve low-light images will be our key research direction.

6. Conclusions

In this paper, we presented a simple yet efficient retinex-based method for low-light images. The research mainly included the following steps: firstly, an initial illumination map estimation algorithm based on neighborhood information was proposed. Then, in order to further enhance brightness and local contrast of the initial illumination map, two input images, which were obtained by BEF and IACE, respectively, were fused by means of the Gaussian–Laplacian pyramid. Finally, the enhanced image was obtained based on retinex theory. Images from different datasets were used to verify the effectiveness of our method from both subjective and objective aspects, that is, our method can not only improve the brightness, but also retain more detailed information. In the future, we aim to enhance low light images by extracting rich image features in conjunction with the currently hotter deep learning.