Dual-Fusion Active Contour Model with Semantic Information for Saliency Target Extraction of Underwater Images

Abstract

Underwater vision research is the foundation of marine-related disciplines. The target contour extraction is significant for target tracking and visual information mining. Aiming to resolve the problem that conventional active contour models cannot effectively extract the contours of salient targets in underwater images, we propose a dual-fusion active contour model with semantic information. First, the saliency images are introduced as semantic information and salient target contours are extracted by fusing Chan–Vese and local binary fitting models. Then, the original underwater images are used to supplement the missing contour information by using the local image fitting. Compared with state-of-the-art contour extraction methods, our dual-fusion active contour model can effectively filter out background information and accurately extract salient target contours. Moreover, the proposed model achieves the best results in the quantitative comparison of MAE (mean absolute error), ER (error rate), and DR (detection rate) indicators and provides reliable prior knowledge for target tracking and visual information mining.

1. Introduction

In recent years, the development and utilization of the ocean have gradually become an important development direction. Since underwater vision research is the basis of marine-related disciplines, the rapid development of underwater image processing technology is inevitable [1,2]. Image segmentation is a basic method of target extraction, which aims to partition an image into several constituent regions. Each region has coherent intensities, colors, and textures [3]. Furthermore, image segmentation can provide technical support for target tracking, image restoration [4,5,6], and other tasks.

Now, some results have been achieved in underwater image segmentation. Liu et al. [7] proposed an improved level set algorithm based on the gradient descent method and applied it to segment underwater biological images. Wei et al. [8] improved the K-means algorithm to segment underwater image backgrounds and addressed the issue of improper K value determination. Moreover, this algorithm can minimize the impact of the initial centroid position of a grayscale image. SM et al. [9] used a canny edge detection algorithm to segment underwater images, whereas background noise greatly affected the canny edge detection algorithm. Sun et al. [10] and Li et al. [11] used fuzzy C-means to segment underwater images. Rajeev et al. [12] used the K-means algorithm to segment underwater images. However, the clustering algorithms mentioned above are greatly affected by the local gray unevenness of underwater images. Moreover, clustering algorithms contain local convergence errors and are only suitable for underwater images with a single background gray level.

Some investigators have segmented underwater images based on their optical properties and achieved good results. For example, Chen et al. [13] proposed an optical feature extraction, calculation, and decision method to identify the collimated region of artificial light and employed a level set method to segment the objects within the collimated region. This method could better identify the target region, but the level set method could not filter out background noise when the target region contained background information. Xuan [14] et al. proposed an RGB(red, green, blue) color channel fusion segmentation method for underwater images. The proposed method obtained the grayscale image with high foreground-background contrast and employed the thresholding segmentation method to conduct fast segmentation. However, the disadvantage of this method is that when the color of the background region is similar to the foreground region, the target cannot be segmented.

Active contour models have also been used for underwater image segmentation. Zhu et al. [15] used the cluster-based algorithm for co-saliency detection and highlighted salient regions in the underwater images. Then, the local statistical active contour model was used to extract the target contours of underwater images. Qiao et al. [16] proposed an improved method based on the active contour model. The method used the RGB color space and contrast limited adaptive histogram equalization (CLAHE) to increase the contrast of the sea cucumber thorns and body. Then, the method extracted the edge of the sea cucumber thorns using the active contour model. Li et al. [17] improved the traditional level set methods by avoiding the calculation of the signed distance function (SDF) to segment underwater images. The improved method could speed up the computational complexity without re-initialization. Bai et al. [18] proposed a method based on morphological component analysis (MCA) and adaptive level set evolution to segment underwater images. The MCA decomposes the image into texture and cartoon parts sparsely. The new adaptive level set evolution method combines the threshold piecewise function with the evariable right coefficient and halting speed function and was used to obtain the edges of the cartoon part. Shelei et al. [19] segmented underwater grayscale images by fusing the geodesic active contour model (GAC) and the Chan–Vese (CV) model. However, this method required the target region of the underwater image to have a uniform grayscale. Chen et al. [20] integrated the transmission map and the saliency map into a unified level set formulation to extract the salient target contours of the underwater images. Antonelli et al. [21] proposed some spatially varying regularization methods by using local image features (such as textures, edge, noise, etc.) to prevent excessive regularization in smooth regions and preserve spatial features in nonsmooth regions. However, it is difficult for this method to obtain accurate regularization parameters for underwater images with low contrast and blurred texture information due to blurred local features, such as textures and edges. Nawal et al. [22] presented an efficient approach for unsupervised segmentation based on image feature extraction to segment natural and textural images. Unfortunately, this method is unsuitable for natural underwater images with weak texture information.

As a new technology of image processing, the neural network has also been used for underwater image segmentation. O’Byrne et al. [23] proposed the use of photorealistic synthetic imagery to train deep encoder-decoder networks. This method synthesized virtual underwater images, and each rendered image had a corresponding ground truth per-pixel label map. The mapping relationship between the underwater images and the segmented images was established by training the encoder-decoder network. Zhou et al. [24] proposed a deep neural network architecture for underwater scene segmentation. The architecture extracted features by pre-training VGG-16 and learned to expand the lower resolution feature maps using the decoder. The neural network achieved certain results in underwater image segmentation, but the lack of underwater data sets with corresponding functions is still a problem.

Most of the existing underwater image segmentation methods are used to segment images with high foreground-background contrast and single background grayscale. When the underwater images with varying background grayscale and the targets have complex textures, the segmentation results of the above methods are not satisfactory. To address the above problem, we propose a novel dual-fusion model with semantic information for salient object segmentation of underwater images with complex backgrounds. In summary, the contributions of our model are as follows:

• We introduce saliency maps as semantic information to segment foreground information and background information;
• The dual-fusion energy equation is proposed to extract the contours of saliency targets by integrating the local and global intensity fitting term;
• For the missing saliency target information, we propose the correction module to correct the saliency target contour error by introducing the original image contour information.

This paper is organized as follows: Section 1 reviews related works. In Section 2, we introduce in detail the derivation process of the dual-fusion model. Section 3 shows the experimental process, and we compare the proposed method with state-of-the-art segmentation methods, and the results demonstrate the superiority of the proposed methods. The conclusion of this paper is shown in Section 4.

2. Related Works

2.1. The Chan–Vese (CV) Model

The Chan–Vese (CV) model [25] was initially derived from the Mumford–Shah (MS) functional [26]. The MS functional aims to find an optimal contour $C$ that divides the image domain into disjoint subregions and a piecewise smooth approximation image $I:{\mathsf{\Omega }}_i\subset R$ from the original image $I_0:\mathsf{\Omega }\subset R^2$, and the energy functional of MS can be expressed as follows:

$$E^{\mathrm{MS}}(I,C)={\displaystyle {\int }_{\mathrm{\Omega }}{\left(I_0-I\right)}^2}dx+\mu {\int }_{\mathrm{\Omega }\backslash C}|\nabla I{|}^{2}dx+v|C|$$

(1)

where $\mu ,v\ge 0$ are positive weighting constants, $\left|C\right|$ is the length of the contour $\left|C\right|$, and $\nabla I$ is the derivative of the function $I$. However, the non-convexity of the above energy functional makes it difficult to be minimized. Besides, the MS functional is restricted to piecewise constant equations, so the CV model has been proposed to simplify and modify the MS functional. The basic idea of the CV model is to find a particular partition to separate a given image into foreground and background. The energy functional of the CV model can be defined as follows:

$$E^{\mathrm{CV}}\left(C,c_1,c_2\right)={\lambda }_1{\displaystyle {\int }_{\mathrm{in}(C)}{\left(I_0-c_1\right)}^2}dx+{\lambda }_2{\displaystyle {\int }_{\mathrm{out} (C)}{\left(I_0-c_2\right)}^2}dx+v\cdot \mathrm{len}(C)+\mu \cdot \mathrm{area}(\mathrm{in}(C))$$

(2)

where $\mu ,v,{\lambda }_1,{\lambda }_2\ge 0$ are positive parameters, $\mathrm{in}(C)$ and $\mathrm{out}(C)$ represent the region inside and outside of the contour $C$, and $c_1$ and $c_2$ are two constants that approximate the image intensity in $\mathrm{in}(C)$ and $\mathrm{out}(C)$, respectively. The Euclidean length term $\mathrm{len}(C)$ is used to regularize the contour. The first 2 terms in Equation (2) are the global binary fitting energy. This energy can be represented by a level set formulation, and then the energy minimization problem can be converted by solving a level set evolution equation. The evolution equation can be expressed as follows:

$$c_1=\frac{{\displaystyle {\int }_{\mathrm{\Omega }}{I}_0}\cdot H(\varphi )dx}{{\displaystyle {\int }_{\mathrm{\Omega }}H}(\varphi )dx},c_2=\frac{{\displaystyle {\int }_{\mathrm{\Omega }}{I}_0}\cdot (1-H(\varphi ))dx}{{\displaystyle {\int }_{\mathrm{\Omega }}(1-H(}\varphi ))dx}$$

(3)

$$\frac{\partial \varphi }{\partial t}=\delta (\varphi )\left[v\mathrm{div}\left(\frac{\nabla \varphi }{|\nabla \varphi |}\right)-\mu -{\lambda }_1{\left(I_0-c_1\right)}^2+{\lambda }_2{\left(I_0-c_2\right)}^2\right]$$

(4)

where $H(\cdot )$ is the Heaviside function, and $\delta (\cdot )$ is the Delta function, which is derivative of the Heaviside function. For Equation (4), $v$ is a scaling parameter. If $v$ is small enough, small targets are likely to be extracted while if $v$ is large, large targets can be detected.

Whereas if the image intensities are inhomogeneous, the global fitting will not be accurate. Therefore, the CV model is not suitable for inhomogeneous images, and the segmentation results are affected by the position of the initial level set [3]. However, the CV model has better robustness to noise.

2.2. The Local Image Fitting(LIF) Model

Different from the CV model’s segmentation of the foreground and background by evolving the level set curve, the local image fitting (LIF) [27] utilizes the local image information to construct a local image fitting energy functional, which can be viewed as a constraint of the differences between the fitting image and the original image. So, the LIF model can ignore the influence of the intensity inhomogeneity, and its energy functional is defined as follows:

$$\frac{\partial \varphi }{\partial t}=\left(I(x)-I^{\mathrm{LFI}}(x)\right)\left(m_1-m_2\right){\delta }_{\epsilon }(\varphi ),$$

(5)

where $I^{\mathrm{LFI}}(x)$ is a local fitted image:

$$I^{\mathrm{LFI}}(x)=m_1{H}_{\epsilon }(\varphi (x))+m_2\left(1-H_{\epsilon }(\varphi (x))\right),$$

(6)

where $m_1$ and $m_2$ are the averages of the image intensities of the Gaussian window inside and outside the contour, respectively. $m_1$, $m_2$ can be expressed as follows:

$$\left\{\begin{array}{l}{m}_1=\mathrm{mean}\left(I\in \left(\{x\in \mathrm{\Omega }|\varphi (x)<0\}\cap W_k(x)\right)\right)\hfill \\ m_2=\mathrm{mean}\left(I\in \left(\{x\in \mathrm{\Omega }|\varphi (x)>0\}\cap W_k(x)\right)\right)\hfill \end{array}\right.,$$

(7)

where $W_k(x)$ is a truncated Gaussian window or a constant window.

Then, the LIF model uses the variation calculus and the steepest descent method to minimize $E^{\mathrm{LIF}}(\varphi )$, and the level set evolution equation can be expressed as follows:

$$\frac{\partial \varphi }{\partial t}=\left(I(x)-I^{\mathrm{LFI}}(x)\right)\left(m_1-m_2\right){\delta }_{\epsilon }(\varphi ),$$

(8)

3. Dual-Fusion Active Contour Model

In this section, we propose a dual-fusion active contour model with semantic information to extract the target contours of underwater images. The existing methods cannot extract the target contour from the background without semantic information. So, it is necessary to introduce semantic information and roughly extract the saliency target contour from the complex background. To avoid the extraction error of the saliency target, we introduce the original image contour to correct and supplement the missing contour information. The proposed model can accurately extract the saliency target contour from the complex background using the semantic information and correction module.

3.1. Saliency Image Fitting Energy

This paper uses the pyramid feature attention network [28] to acquire the saliency images. However, due to the low contrast of underwater images, there were some errors in the saliency detection results, such as a local inhomogeneous intensity, background noise, and missing contour information. In view of the local inhomogeneous intensity of the saliency images, we preliminarily employ the local binary fitting to construct the energy functional $E_{\mathrm{sal}}$:

$$E_{\mathrm{sal} }\left(C,f_1(x),f_2(x)\right)={\lambda }_1{\displaystyle {\int }_{\mathrm{in}(C)}{\left(S-f_1(x)\right)}^2}dx+{\lambda }_2{\displaystyle {\int }_{\mathrm{out} (C)}{\left(S-f_2(x)\right)}^2}dx,$$

(9)

where $S$ is the saliency images, $C$ is a contour in the image domain $\mathrm{\Omega }$, and $f_1$ and $f_2$ are the image local fitting intensities near the point $x$. The local fitting intensities $f_1$, $f_2$ can be expressed as follows [29,30]:

$$f_1(x)=\frac{K_{\sigma }(x)\cdot \left[H_{\epsilon }(\varphi (x))S\right]}{K_{\sigma }(x)\cdot H_{\epsilon }(\varphi (x))}$$

(10)

$$f_2(x)=\frac{K_{\sigma }(x)\cdot \left[\left(1-H_{\epsilon }(\varphi (x))\right)S\right]}{K_{\sigma }(x)\cdot \left[1-H_{\epsilon }(\varphi (x))\right]}$$

(11)

where $K_{\sigma }(x)$ is the Gaussian kernel, $S$ is the saliency images, and $H_{\epsilon }$ is the Heaviside function $H(\cdot )$ can be expressed as:

$$H_{\epsilon }(x)=\frac{1}{2}\left[1+\frac{2}{\pi }\arctan \left(\frac{x}{\epsilon }\right)\right],$$

(12)

However, the local binary fitting may introduce some local minimums and is sensitive to noise. Affected by the accuracy of saliency detection, the saliency map of underwater images will inevitably have background noise. Moreover, the initialization curve greatly affects the segmentation results. To solve the aforesaid problems, we introduce the global fitting term from the CV model into the energy functional $E_{\mathrm{sal}}$. The local-global fitting intensities can be expressed as follows:

$$\left\{\begin{array}{l}{I}_1=\omega c_1+(1-\omega )f_1\hfill \\ I_2=\omega c_2+(1-\omega )f_2\hfill \end{array}\right.$$

(13)

where $I_1$ and $I_2$ are the mixed intensity, $c_1$ and $c_2$ are two constants derived from Equation (3), and $\omega$ is a weight coefficient $(0\le \omega \le 1)$. According to the test images in this paper, the value of $\omega$ can be taken from 0.5 to 0.9. Moreover, the more inhomogeneous the image intensity, the smaller the value of $\omega$.

With the level set representation, the energy functional can be expressed as follows:

$$E_{\mathrm{sal} }\left(\varphi ,I_1(x),I_2(x)\right)={\lambda }_1{\displaystyle {\int }_{\mathrm{\Omega }}{\left(S-I_1(x)\right)}^2}H(\varphi (x))dx+{\lambda }_2{\displaystyle {\int }_{\mathrm{\Omega }}{\left(S-I_2(x)\right)}^2}(1-H(\varphi (x)))dx$$

(14)

The improved fitting energy $E_{\mathrm{sal}}$ not only takes the local intensity information into account but also avoids the local minimization. Therefore, for the saliency images of underwater images, the improved energy functional can extract the contour of the inhomogeneous images more accurately.

3.2. Original Image Fitting Energy

The local inhomogeneous intensity and noise problems can be solved by fusing the local intensity fitting and CV model. However, the missing contour information of the saliency image still needs to be solved. Therefore, the original underwater images are used to make up the missing contour information.

In this paper, we use the local image fitting model (LIF) [27] to extract the contour of original underwater images. The energy functional $E_{\mathrm{org}}$ can be expressed as:

$$E_{\mathrm{org}}(\varphi )=\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}{\left|I(x)-I^{\mathrm{LFI}}(x)\right|}^2}dx,x\in \mathrm{\Omega },$$

(15)

where $I^{\mathrm{LFI}}(x)$ is a local fitted image, as shown in Equation (6). Although the models, such as LBF [29,30], LGIF [31], and RMPCM [3] can extract the target contours of underwater images very well, as shown in Figure 1, the LIF model has higher efficiency. This higher efficiency is because the energy functional of the LIF model does not include a kernel function. Moreover, the LIF model can fit the original image well while reducing the noise significantly by minimizing the difference between the fitted image and the original image.

As shown in Figure 1, the LBF, LGIF, and LIF models could extract the target contour better, but LBF was more sensitive to the initial contour curve (as shown in the green dashed area). The energy functional of LGIF and RMPCM both involved the kernel function. The kernel function performs more than one convolution operation for each iteration step, so the evolution speed is slow. The running time of the above models is shown in

Table 1. Iterations and CPU time (in seconds).

	LBF	LGIF	LIF	RMPCM
	$1239\times 731 \mathrm{pixels}$
Iterations	200	200	200	200
Time (s)	93.2969	55.5938	38.5469	63.3052

.

Figure 1 and Table 1 intuitively show that the LIF model has advantages regarding both the speed and contour extraction results. So, we use the LIF model to extract the original image contour to correct the contour information of the salient target.

3.3. Dual-Fusion Active Contour Model

To use less fitting energy at the target contours than at other locations, we use an edge indicator function [32,33] to indicate target contours. The function can be expressed as follows:

$$g\triangleq \frac{1}{1+{\left|\nabla G_{\sigma }\cdot I\right|}^2},$$

(16)

Then, we define the dual-fusion intensity fitting energy functional as follows:

$$E^{\mathrm{DFIF}}(\varphi )=g\left[{\omega }_1{E}_{\mathrm{org}}+\left(1-{\omega }_1\right)E_{\mathrm{sal}}\right],$$

(17)

where ${\omega }_1$ is a weight coefficient $\left(0\le {\omega }_1\le 1\right)$, and $E_{\mathrm{org}}$ and $E_{\mathrm{sal}}$ are the original images’ fitting energy functional and the saliency images’ fitting energy functional, respectively.

Finally, the dual-fusion intensity fitting energy functional $E^{\mathrm{DFIF}}(\varphi )$ can be obtained by combining Equations (14)–(17):

$$\begin{array}{c}{E}^{\mathrm{DFIF}}\left(\varphi ,I_1,I_2\right)=g\left[{\omega }_1{E}_{\mathrm{org} }+\left(1-{\omega }_1\right)E_{\mathrm{sal} }\right]=\\ g\left[{\omega }_1\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}{\left|I(x)-I^{LFI}(x)\right|}^2}dx+\left(1-{\omega }_1\right)\left({\lambda }_1{\displaystyle {\int }_{\mathrm{\Omega }}{\left(S-I_1(x)\right)}^2}{H}_{\epsilon }(\varphi (x))dx\right.\right.+\\ \left.\left.{\lambda }_2{\displaystyle {\int }_{\mathrm{\Omega }}{\left(S-I_2(x)\right)}^2}\left(1-H_{\epsilon }(\varphi (x))\right)dx\right)\right]\end{array}$$

(18)

Then, we minimize $E^{\mathrm{DFIF}}\left(\varphi ,I_1,I_2\right)$ with respect to $\varphi$ to obtain the corresponding gradient descent flow [29,30,31]:

$$\frac{\partial \varphi }{\partial t}=g{\delta }_{\epsilon }(\varphi )\left[{\omega }_1{e}_1+\left(1-{\omega }_1\right)e_2\right],$$

(19)

where:

$$\left\{\begin{array}{c}{e}_1=\left(I-m_1{H}_{\epsilon }(\varphi (x))-m_2\left(1-H_{\epsilon }(\varphi (x))\right)\right)\left(m_1-m_2\right)\\ e_2=-{\lambda }_1{\left(S-I_1(x)\right)}^2+{\lambda }_2{\left(S-I_2(x)\right)}^2\end{array}\right.,$$

(20)

where $I$ and $S$ are the original images and the saliency images, respectively. $I_1(x)$ represents the integrated local and global intensities, and $m_1$ and $m_2$ are averages of the image intensities in a Gaussian window inside and outside the contour.

3.4. Regularize the Level Set Function

As pointed out by Zhang’s method [27], Gaussian filtering can replace the traditional regularized term to regularize the level set function. Therefore, the smoothing process of the level set function can be expressed as:

$${\varphi }^{k+1}=G_{\eta }\cdot {\varphi }^k,\eta >\sqrt{\Delta t},$$

(21)

where $\eta$ is the standard deviation, and $\Delta t$ is the time-step.

In fact, the smoothing effect of the level set function by Gaussian filtering is slightly worse than the traditional regularized term and is greatly affected by the time-step. However, the computing efficiency of Gaussian filtering is much higher than the traditional regularized term.

4. Results and Discussion

This section tested the proposed method on intensity-heterogeneous underwater images captured from underwater videos downloaded from the NATURE FOOTAGE website and Fish Dataset. Moreover, the method was compared with some state-of-the-art contour extraction methods in terms of its efficiency and accuracy. All contour extraction results were produced on the same computer to ensure fairness. The computer was configured as Intel(R) Core(TM) i7-8650U CPU @ 2.11 GHz, 16.00 GB memory, Windows 10 system, and x64 processor. MatlabR2017a was the software platform. We used the same parameters ${\eta }^2=6,\sigma =2,\epsilon =1,{\lambda }_1=3,{\lambda }_2=1$ and time-step $\Delta t=0.1$. The initial level set function was defined by:

$$\varphi (x,t=0)=\left\{\begin{array}{ll}-c_0,\hfill & x\in \mathrm{in}(C)\hfill \\ 0,\hfill & x\in C\hfill \\ c_0,\hfill & x\in \mathrm{out}(C)\hfill \end{array}\right.,$$

(22)

where $c_0>0$ is a constant (in our experiments, $c_0=1$) and $\mathrm{in}(C)$ and $\mathrm{out}(C)$ represent the region inside and outside of the contour $C$, respectively.

Moreover, the parameter ${\omega }_1$ is a constant, which controls the influence of the saliency image fitting energy and original image fitting energy. When the missing information of the saliency target contour is severe, ${\omega }_1$ should be relatively larger; otherwise, ${\omega }_1$ should be a small value. Moreover, $\omega$ should be smaller when the intensity inhomogeneity of the saliency image is severe. This is because the local intensity fitting can better segment the target in the intensity inhomogeneity region, and the results of contour extraction rely on the local intensity fitting. Otherwise, $\omega$ should be larger to suppress the noise interference. In the experiment, we need to choose appropriate values for $\omega$ and ${\omega }_1$ according to the degree of inhomogeneity and the degree of saliency detection deviation. In the experiment of this paper, the value of $\omega$ was from 0.5 to 0.9, and the value of ${\omega }_1$ was from 0.1 to 0.8.

4.1. The Benefits of Local-Global Intensity Fitting

A comparative experiment was performed to prove the effectiveness of the local-global intensity fusion in Section 3.1. We conducted different experiments, as shown in

Table 2. The comparative experiment of the local-global intensity.

Experiments	local Intensity	Global Intensity
A	✓
B		✓
C (our fusion intensity)	✓	✓

.

In experiment A, the fitting intensity of the energy functional is the local intensity. In experiment B, the fitting intensity of the energy functional is the global intensity. Moreover, the energy functional with the fusion local-global intensity is shown in Experiment C. The contour extraction results of the experiments are shown in Figure 2.

As shown in Figure 2, experiment A extracted the target contour in the intensity inhomogeneity region, but the result was greatly affected by the initial contour curve (blue circled area) and was sensitive to noise (green circled area). Moreover, the method of experiment A also extracted the contours of the non-boundary regions. Experiment B could extract the target contour in the intensity homogeneity region and was not disturbed by noise, but the target contour in the intensity inhomogeneity region could not be extracted. By comparing the segmentation results in Figure 2a–c, it can be seen that the fused energy functional (experiment C) can not only effectively eliminate the influence of the initial curve and noise interference but also effectively segment intensity-inhomogeneous regions.

4.2. The Effect of Original Image Correction

Figure 3 shows the result of our method in the underwater image segmentation. As shown in Figure 3, the coordinate points $\{[\mathrm{X},\mathrm{Y}]:[77,41]\}$ and $\{[\mathrm{X},\mathrm{Y}]:[152,57]\}$ are located at the saliency target edge in Figure 3b. However, in Figure 3c, the coordinate point at the same position is inside the target instead of on the target edge. This error is caused by the deviation in the saliency detection. Therefore, it is necessary to use the original image to supplement the missing information. This paper used the local image fitting method to extract the contour information of the original image. It then used the contour information to correct the deviation caused by saliency detection. The result of the correction is shown in Figure 3e. As shown in Figure 3e, the missing contour information of the saliency image is accurately supplemented, and the background information is filtered out.

4.3. Performance of the Dual-Fusion Active Contour Model

Figure 4 shows the performance of our method. It can be seen from Figure 4d that our method can filter out the background information and accurately extract the target contour. Figure 4b shows the saliency images of the original underwater images. The red circles represent the intensity inhomogeneity region, the yellow circles represent the noise region, and the green circles represent the missing region of the target. For the regions of the intensity inhomogeneity and noise, our method can still extract the target contour well using the local-global intensity fitting term. Moreover, the saliency image of the first image obviously lacks part of the target information (green circle region). Our method can still extract the complete target contour by integrating the original image contour information.

4.4. Qualitative Comparison

4.4.1. Comparison of the Segmentation Results with Other Models

To verify the effectiveness of the proposed method, we compared the segmentation results with other classic models, such as LBF [29,30], LGIF [31], LIF [27], and RMPCM [3], respectively. The comparison results are shown in Figure 5.

It can be seen from Figure 5 that the LBF model is limited by the initial contour curve and cannot completely extract the target contour. The LGIF model is minimally affected by local background noise due to the fusion of the global intensity fitting, but it still cannot accurately extract the target contour. The LIF and RMPCM models can completely extract the target contour, but they are greatly affected by background noise and local target features. Our model introduces semantic information to filter out background noise very well. Moreover, because of the global-local intensity fitting, our method can handle local inhomogeneous regions without interfering in the local target features. In addition, the target contour of the original image perfectly complements the missing semantic information.

Furthermore, we compared the segmentation results with Zhu’s method [15] and Chen’s method [20], which also introduced saliency images as semantic information. Since we could not obtain the source codes of Zhu’s method [15] and Chen’s method [20], to ensure the fairness of the comparison results, we adapted the segmentation images from Zhu’s method [15] (2017,IEEE) and Chen’s method [20] (2019,IEEE) as the comparison images. The comparison results are shown in Figure 6.

As can be seen in Figure 6, even though our method, Zhu’s method [15], and Chen’s method [20] all introduce semantic information, our method can extract the target contour more accurately than Zhu’s method [15] and Chen’s method [20]. As shown in the blue circle region of Figure 6a, our method extracted the target contour in the detail region more accurately. This is because we added the local-global fitting term to better extract the contours of local inhomogeneous regions, and the original image correction module can correct the errors in semantic information. As shown in the green circle region of Figure 6b, our method can filter out background noise better than Chen’s method [20] and is more robust.

4.4.2. Comparison of the Saliency Segmentation Results with Other Models

To further verify the superiority of the proposed method, we also compared the contour extraction results of the underwater image with the saliency image as the input of several classic models. To test the robustness of the proposed method, we only selected low-quality saliency images (inhomogeneous local intensity and incomplete saliency information) for the comparison experiments. As shown in Figure 7, the segmentation results of LBF are severely affected by the initial contour curve and are disturbed by the inhomogeneous regions inside the target. The LGIF model can avoid the influence of the initial contour curve but cannot extract complete contour information, as shown in the green dotted region in Figure 7(2). The LIF model can extract the target contour relatively completely, but it easily falls into the local optimum and is also affected by the initial contour curve. The RMPAM model avoids the local optimum error, but it also has the problem that the contour information cannot be extracted completely, as shown in the green dotted region in Figure 7(2). Our method can effectively avoid the local optimum and supplement the missing contour information through the original image. Hence, the results of our method are more accurate and complete than other methods.

4.5. Quantitative Comparison

In the following experiment, we compare the proposed method with the aforementioned methods using several evaluation indexes to conduct a quantitative analysis. Here, three evaluation indicators, namely the mean absolute error (MAE), the error rate (ER), and the detection rate (DR), are employed for quantitative comparison. The MAE, ER, and DR can be expressed by the following equations:

$$\mathrm{MAE}=\frac{1}{m\times n}{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^n\left|De{t}_{(x,y)}-g{t}_{(x,y)}\right|}},$$

(23)

$$\mathrm{ER}=\frac{1}{m\times n}{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^n\left|De{t}_{(x,y)}-g{t}_{(x,y)}\right|}}/\frac{1}{m\times n}{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^{m}De{t}_{(x,y)}}}\cdot g{t}_{(x,y)},$$

(24)

$$\mathrm{DR}=\frac{1}{m\times n}{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^{m}De{t}_{(x,y)}}}\cdot g{t}_{(x,y)}/\frac{1}{m\times n}{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^m\left(De{t}_{(x,y)}+g{t}_{(x,y)}\right)}},$$

(25)

where $m$ and $n$ represent the length and width of the image, $Det$ is the result of image segmentation, and $gt$ is the hand-crafted ground truth. So, $De{t}_{(x,y)}\cdot g{t}_{(x,y)}$ represents the contour that is accurately extracted by the model and $De{t}_{(x,y)}+g{t}_{(x,y)}$$De{t}_{(x,y)}+g{t}_{(x,y)}$ represents the union of the image segmentation result and the ground truth. The larger the $De{t}_{(x,y)}\cdot g{t}_{(x,y)}$, the more contour that is correctly extracted. $De{t}_{(x,y)}-g{t}_{(x,y)}$ represents the pixels that are incorrectly extracted, so the larger $De{t}_{(x,y)}-g{t}_{(x,y)}$ is, the more pixels are incorrectly extracted. So, the smaller the value of $\mathrm{ER}$, the more accurate the result of contour extraction. The smaller the value of MAE, the more accurate the contour extraction result. Moreover, a large value of $\mathrm{DR}$ can indicate that the contour extraction result of the model is accurate. The evaluation results of the aforementioned 5 methods are shown in

Table 3. The MAE results of LBF, LGIF, LIF, RMPCM, and our method in Figure 5.

	(i)	(ii)	(iii)	(iv)	(v)	(vi)	(vii)	(viii)
LBF	9.5723	3.6588	3.0737	12.8131	2.1789	4.4882	6.9886	5.3132
LGIF	7.9119	3.7481	3.4620	9.6148	2.2546	4.1299	7.4210	6.3514
LIF	6.0811	4.0945	2.5343	7.4206	2.9782	4.2036	10.2057	4.9874
RMPCM	10.5081	4.3048	5.1594	12.9181	2.3604	6.0406	7.9683	3.9875
Our	2.3695	3.2604	1.9161	4.5302	1.2455	2.7715	5.9417	2.9702

,

Table 4. The ER results of LBF, LGIF, LIF, RMPCM, and our method in Figure 5.

	(i)	(ii)	(iii)	(iv)	(v)	(vi)	(vii)	(viii)
LBF	0.7434	0.5195	0.0510	0.7863	0.2585	0.2769	0.3403	0.2386
LGIF	1.0012	0.5978	0.1414	0.7304	0.1237	0.2283	0.2092	0.3109
LIF	0.8355	0.4713	0.0761	0.3578	0.1209	0.2938	0.5013	0.1483
RMPCM	1.1478	0.3300	0.0962	1.0167	0.1167	0.2137	0.3200	0.9286
Our	0.2709	0.2649	0.0452	0.3411	0.0776	0.2007	0.2060	0.0571

and

Table 5. The DR results of LBF, LGIF, LIF, RMPCM, and our method in Figure 5.

	(i)	(ii)	(iii)	(iv)	(v)	(vi)	(vii)	(viii)
LBF	1.3148	1.7739	13.9989	1.2499	3.5065	3.4029	2.7005	3.9709
LGIF	0.9778	1.5494	6.0806	1.3423	6.7686	4.0834	4.1945	3.0709
LIF	1.1707	1.9462	10.1875	2.6965	6.9244	3.2240	1.8872	6.1722
RMPCM	0.8569	2.7046	8.6031	0.9690	7.0274	4.2727	2.8823	1.0500
Our	3.4725	3.3199	15.3967	2.7586	10.3096	4.6516	4.3053	14.0682

.

A smaller value of MAE represents a higher contour extraction accuracy. According to Table 3, the contour extracted by the proposed model obtained the smallest MAE value, which shows that the proposed model can extract target contours more accurately than the other 4 models. Table 4 shows the error rate (ER) of the 5 methods. The ER values between the target contour extracted by the proposed method and ground truth are the smallest, so the proposed method has the highest accuracy. Table 5 shows the detection rates of the above five methods. The detection rate represents how many contour pixels are correctly extracted. Therefore, our model with the highest detection rate can extract the target contour more accurately.

5. Conclusions

Aiming to resolve the problem that conventional active contour models cannot effectively extract the contours of the salient object in underwater images, we proposed a dual-fusion active contour model with semantic information. The proposed method extracted the saliency target contour by fusing the local and global intensity and extracted the original image contour information by the local image fitting model to correct the saliency information deviation. We verified the superiority of the dual-fusion active contour model with semantic information by qualitative comparison and quantitative comparison. The qualitative comparison results show that the proposed model can effectively suppress the interference of noise and can more accurately extract the contour of the intensity inhomogeneity region. Moreover, we also verified that the missing saliency target contour can be effectively corrected by the contour information of the original image. The quantitative comparison results show that the salient object contour extraction results of the proposed model achieve the highest accuracy and lowest error. Therefore, the proposed model can effectively solve the problem that conventional active contour models cannot extract the salient object contour due to the lack of semantic information and provides support for underwater target tracking, underwater image restoration, and other technologies.