1. Introduction
Image deblurring aims to get a clean, sharp image from a noisy, blurred image. Blurs can be observed in many fields such as out-of-focus blur in X-ray imaging because of the poor localization of the point spread function and motion blur caused by the movement of the person. Generally, the image blurring process can be modeled as the convolution of an original clear image with a shift-invariant blur kernel plus additive Gaussian white noise, i.e.,
where
is the original clean image,
K is the convolution operator, and
n is Gaussian noise with zero mean and variance
. According to whether the blur kernel
K is prior knowledge, the problem of image deblurring can be divided into non-blind deblurring and blind deblurring. When
K is known exactly, the problem is obtaining a clean image
u from the observed image
f and prior knowledge
K; when
K is unknown, the problem is estimating the blur kernel
K first and then obtaining the clear image
u from image
f.
For the restoration of a blurred image, scientists take advantage of some prior knowledge of the unknown image
u by adding a regularization term, which can be modeled as follows:
where
is called the fidelity term and
is called the regularization term, and
is a tuning parameter to balance the weight between the fidelity term and the regularization term. There are many methods that have been proposed for image restoration. Blind deconvolution image restoration [
1,
2] performs image restoration based on degradation models and prior knowledge, so it can adapt to different types and degrees of image degradation. However, blind deconvolution image restoration usually requires complex models and algorithms to model and process the degradation process of images, which makes it difficult to apply in practice. In addition, the unsuitability of inverse problems makes blind deconvolution image restoration unable to guarantee the uniqueness of solutions. The PSF restoration method [
3,
4] aims to reduce blur and noise in the image by calculating and estimating PSF, thus restoring the sharpness and detail of the image. However, it is sensitive to noise in the image, which may be amplified and affect the quality of the restored image. An image recursive filter [
5,
6] is an effective image smoothing and denoising filter, which calculates the value of the output pixel by the weighted average of the current pixel and its neighbors. However, the design and optimization of the image recursive filter is relatively complex, and there is a poor effect of noise suppression, which is easily introduces artifacts or distortion. In the field of variation image restoration, the most famous model in image restoration is the TV model, which is as follows:
Actually, TV was originally proposed in [
7] for image denoising, and then it was extended to image deblurring in [
8]. The TV model is a popular method that can effectively reduce the noise and blur in the image, and also preserve the sharp edge and texture detail of the image. However, TV regularization tends to get a piecewise constant solution, and, therefore, it easily causes a staircase effect. In order to efficiently suppress the staircase artifacts, numerous models with improved regularization terms have been proposed. This includes high-order partial differential equations [
9] and higher-order TV methods (HOTV) [
10], total variation regularization methods [
11,
12], sparsity regularization models [
13,
14], and fractional-order TV (FOTV) models [
15,
16]. In addition, nonlocal total variation (NLTV) [
17,
18] and block-matching 3-D (BM3D) [
19,
20] have been the most promising deblurring methods for recovering texture. Although NLTV and BM3D have shown good performance in image restoration, the NLTV functional minimization problem has always been a difficult optimization problem because of its high computation complexity and the non-differentiability, and the BM3D method has limited effectiveness in processing high-noise images and motion-blurred images.
Another method to overcome the staircase effect is the total generalized variation (TGV) regularization, which was firstly proposed by Bredies et al. as a penalty function in [
21]. As an extension of TV regularization, TGV has good properties such as rotational invariance, lower semi-continuity, and convexity. The results show that the TGV regularization method can preserve the details of image edges and textures and suppress the staircase effect. In addition, the scalar weight
of the second-order TGV regularization model has multiple parameters, and better image restoration results can be achieved by adjusting the parameters, so the TGV model has been extensively studied in image restoration [
22,
23,
24,
25] and medical imaging [
26]. It can be formulated as:
Although the TGV regularization model has many advantages, it tends to amplify the noise while restoring the image detail, creating artifacts or distortions, and due to the sensitivity of parameters, it may lead to poor deblurring results or excessive smoothing results (see [
27,
28]).
In this paper, aiming at achieving a good performance for image restoration, we propose a fractional-order fidelity-based total generalized variation model (FTGV) for image deblurring. The objective function takes the TGV as the regularization term that can suppress staircase effect, preserve edges and a fractional-order gradient fidelity term to preserve more details, and get a trade-off between edge preservation and blur removal by adjusting the regularization parameters. Then, based on the non-smoothing of the regularization term of the variation model and the non-convexity of the fractional fidelity term, we propose two optimization algorithms based on the primal-dual (PD) and alternating direction multiplier (ADMM), which transform the fractional problem into subproblems with less computation by introducing auxiliary variables, and finally solve the minimization problem by alternate iteration strategy. By precisely adjusting step size parameters and penalty parameters, the two algorithms are insensitive to the weights , non-integer order , and balance parameters in the variational model, and run faster, thus making the model more robust and efficient.
The rest of this paper is organized as follows. In
Section 2, we present a brief introduction of TGV model, and then give the new deblurring model and the discrete form for objection functional. We provide the numerical scheme based on the PD algorithm and ADMM algorithm to solve the proposed model and analyze the convergence in
Section 3. Numerical experiments are shown to illustrate the performance of the proposed model in
Section 4.
4. Numerical Experiments
In this section, we will test the performance of the proposed model (
2) with the PD algorithm and ADMM algorithm, and also compare with some efficient visual and analytical methods for image deblurring such as TGV [
21], APE-TGV [
28], D-TGV [
33], BM3D [
19], and NLTV [
18]. To evaluate the restoration results, we use these quantitative measures, including the peak signal to noise ratio (PSNR), the mean square error (MSE), and the structural similarity (SSIM) metric, which are commonly used in image processing. The better quality image will have higher PSNR and SSIM, but lower MSE.
In experiments, we consider three common blur scenarios: motion blur, disk blur, and average blur. The motion blur, disk blur, and average blur are generated by the MATLAB built-in functions fspecial(‘motion’, 20, 50), fspecial(‘disk’, 5) and ones
, respectively. Except for the blur, we also add the Gaussian noise with standard deviation
to the blurry image. For illustration, six test images are presented in
Figure 2 with different sizes. In all experiments, we terminate two algorithms when
or Maxiter = 2000.
4.1. Comparison of Proposed Algorithms
In this subsection, we compare the efficiency of the two proposed algorithms, FTGV-ADMM and FTGV-PD, by minimizing the same objective function (
10). We optimize the algorithmic parameters for each algorithm to achieve higher PSNR improvement, which are listed in
Table 1. The visual results are provided in
Figure 3,
Figure 4 and
Figure 5, and
Table 2 and
Table 3 report the restoration results in terms of PSNR and SSIM values, along with other contrastive models. In addition, we plot the energy and MSE values with respect to the iteration in
Figure 6 and
Figure 7, and numerically demonstrate the convergence of each algorithm.
FTGV-ADMM does not satisfy the convergence of ADMM and, hence, for any penalty parameter, is not available. Actually, different choices about could influence the convergent speed of the algorithms. In our experiments, we set a satisfying rule of thumb for deblurring based on experience .
In
Table 2 and
Table 3, we find that FTGV-ADMM has the highest PNSR value and competitive SSIM value for the texture image, butterfly image, brain image, and man image. FTGV-PD achieved slightly lower results than FTGV-ADMM in the PSNR value for disk blur and average blur. Unfortunately, we notice that FTGV-PD achieves bad results on motion blur for all test images, whether it is the texture image or natural images. In the visualization aspect, FTGV-ADMM achieves better recovery results than FTGV-PD overall.
The results provided in
Figure 6 show that FTGV-ADMM has the lowest MSE value and the fewest iterations for all test images. On the contrary, FTGV-ADMM requires more iterations to obtain a smaller MSE. We observe that FTGV-ADMM is the fastest algorithm to minimize the energy because it costs less iterations. In terms of the disk blur presented in
Figure 7, FTGV-PD owns the lowest MSE and less iterations for the texture image than FTGV-ADMM, but FTGV-ADMM has the fastest rate to obtain the lowest MSE for other three images. In addition, FTGV-PD is the fastest and best algorithm to minimize the energy. Although FTGV-ADMM has brief oscillations occurring during the descent process of energy, it did not affect the convergence result. For average blur, we get the same results as disk blur, so we omit it.
Next, we illustrate how the fractional order
affects the image restoration referred to in
Figure 8, which plots the largest PSNR value as a function of fractional order
. The experimental results show that the fractional order
in the fidelity term can avoid the staircase effect and get more detailed structures by choosing suitable orders.
From
Figure 8, we can notice that for motion blur, the optimal order occurs at
{0.1–0.8}; it is obvious that we get the lowest PSNR value when
for all test images. For the disk blur, the optimal order occurs at
, where test images have the highest PSNR value. The trend of curve of the average blur is roughly the same as with the disk blur, so it is omitted. Thus, we use the fractional orders
or
in our experiments to obtain a higher PSNR through simple adjustments.
Overall, as FTGV-ADMM involves the least number of parameters, and is suitable for all blur types and images compared to the FTGV-PD, we will use it for the rest of the experiments.
4.2. Comparison of Other TGV-Based Methods
To demonstrate that the model based on the fractional-order fidelity term has better capability in texture restoration than other fidelity-based models for images with rich texture and nature images, we compare the proposed model (
2) with TGV [
21], APE-TGV [
28], and D-TGV [
33] for three different blur kernels and the standard deviation
Gaussian noise. To fairly compare, the parameters of the comparison models are selected according to the recommendations of the corresponding paper through adjusting them appropriately to get better results and the best PSNR; the choice of parameters is listed in
Table 1. We provide results in
Figure 3,
Figure 4 and
Figure 5 and
Table 2 and
Table 3.
As we can see in
Figure 3b,
Figure 4b, and
Figure 5b, TGV could not eliminate blur completely. The images restored by APE-TGV in
Figure 3c,
Figure 4c and
Figure 5c were over-smoothed, in which the texture structure was lost. The D-TGV model has advantages in maintaining structures, but it tends to lose some texture details and create some artifacts. From
Figure 3,
Figure 4 and
Figure 5, it is easy to find that the restoration result of D-TGV is visually inferior to APE-TGV for the texture image. This means that D-TGV and APE-TGV are imperfect under certain circumstances. However, the proposed model with the ADMM algorithm (FTGV-ADMM) overcome these drawbacks and gets a better balance between deblurring completely and restoring more image details; see
Figure 3e,
Figure 4e and
Figure 5e.
Table 2 and
Table 3 report that FTGV-ADMM is comprehensively superior to other models in terms of PSNR. For example, it is 8.4 db, 2.65 db, and 1.69 db higher than the TGV, APE-TGV, and D-TGV for the texture image damaged by motion blur. This demonstrates the superiority of FTGV-ADMM.
4.3. Comparison with BM3D and NLTV
In this subsection, we demonstrate the performance of FTGV-ADMM with the famous blockmatching and 3D filtering (BM3D) method and the nonlocal TV (NLTV) for solving the image deblurring problem. In this paper, we use the preconditioned Bregmanized operator splitting (PBOS) method proposed by Zhang and Burger et al. [
18]. Other parameters of the method are selected as suggested by the authors, but the regularization parameter
is adjusted to obtain the highest PSNR value.
From
Table 4, we can see that FTGV-ADMM achieves the best PSNR and SSIM values, which are far higher than the BM3D and NLTV methods. The visual quality of the deblurred results by the NLTV method is too over-smooth, and it produces obvious artifacts; see enlarged zoom
Figure 9i,
Figure 10i,
Figure 11i and
Figure 12i. The BM3D method is one of the best existing deblurring methods for Gaussian blurs and out-of-focus blurs, but the restoration image is still a little smooth and many details have been lost, which can be seen in the enlarged, zoomed
Figure 9h,
Figure 10h,
Figure 11h and
Figure 12h. Most importantly, FTGV-ADMM overcomes these difficulties and achieves higher visual quality.