Leak Detection in Water Pipes Based on Maximum Entropy Version of Least Square Twin K-Class Support Vector Machine

Abstract

Numerous novel improved support vector machine (SVM) methods are used in leak detection of water pipelines at present. The least square twin K-class support vector machine (LST-KSVC) is a novel simple and fast multi-classification method. However, LST-KSVC has a non-negligible drawback that it assigns the same classification weights to leak samples, including outliers that affect classification, these outliers are often situated away from the main leak samples. To overcome this shortcoming, the maximum entropy (MaxEnt) version of the LST-KSVC is proposed in this paper, called the MLT-KSVC algorithm. In this classification approach, classification weights of leak samples are calculated based on the MaxEnt model. Different sample points are assigned different weights: large weights are assigned to primary leak samples and outliers are assigned small weights, hence the outliers can be ignored in the classification process. Leak recognition experiments prove that the proposed MLT-KSVC algorithm can reduce the impact of outliers on the classification process and avoid the misclassification color block drawback in linear LST-KSVC. MLT-KSVC is more accurate compared with LST-KSVC, TwinSVC, TwinKSVC, and classic Multi-SVM.

1. Introduction

Water supply pipelines are important infrastructure in cities, and maintaining the stable operation of water supply pipelines has significant economic, sanitary, and environmental worth. Therefore, the real-time monitoring of pipeline operation status and detection of suspected leak risks are significant for maintaining the safe operation of pipe network, avoiding water resource waste, and realizing sustainable production [1].

As a vital technology in the machine learning field, the support vector machine (SVM) [2] and its improved versions are widely utilized in pipeline leak detection and localization. To achieve greater efficiency in leak detection in water pipes, a novel improved multi-class SVM algorithm is herein proposed, called maximum entropy [3] (MaxEnt) version of LST-KSVC [4] (MLT-KSVC). This paper is organized as follows. Various leak detection and location methods proposed in recent years are summarized in Section 2. The theoretical explanation of LST-KSVC and MLT-KSVC is presented in Section 3. Experimental setup and data processing are presented in Section 4. Finally, the conclusions are offered in Section 5.

2. Related Work

In recent years, several leak detection and location methods based on artificial intelligence algorithms have been proposed. This paper divides these technologies into two categories: (1) leak recognition or detection method based on machine learning algorithms [5,6,7,8] and (2) leak recognition or detection method based on deep learning algorithms [9,10,11,12]. These two methods collect leak acoustic signal, pressure signal, flow signal, or transient water hammer wave signal of the water pipes to build leak dataset.

In the leak detection system based on machine learning algorithms, Bohorquez [13] presented a methodology that uses artificial neural networks (ANNs) to predict the presence of leak features in a pipeline. This methodology demonstrated the potential of the combined use of both fluid transient pressure waves and ANNs to detect leak features in pipelines. Pérez [14] also proposed an improved ANNs leak diagnosis for fluid transport pipelines. In this methodology, the pressure and flow rate were acquired as original data for ANNs, and the pipe friction factor was used as an input to estimate the leak point. Gong [15] proposed a pipe leak detection method based on acoustic emission (AE) data and neural networks. In this detection method, a leak classifier was built based on a backpropagation neural network after leak feature extraction and analysis from AE signals. Diao [16] combined particle swarm optimization (PSO) algorithm, MaxEnt, and variational mode decomposition (VMD) to remove background noise from leak AE signals. Then, the SVM was employed to complete leak recognition for de-noised leak data. Quy [17] used the spectral portrait method to pre-process pipe leak AE signals. Next, a multi-class SVM classifier was used for leak detection.

In the leak detection system based on deep learning algorithms, Kang [18] combined a one-dimensional convolutional neural network (CNN) and SVM to solve computational and time cost problems in an online pipe leak detection. Cody [19] combined CNN with variational autoencoder to detect small leaks in buried water pipelines. Lang [20] recommended a method to detect small leak apertures in pipelines. The method was composed of wavelet packet analysis (WPA) and deep belief network (DBN) with independent component regression (ICR).

However, these methods fail to remove the interference of noise. When SVM and its improved algorithms are used in pipeline leak monitoring, outliers often disturb the classification processing. Such outliers are distant from the data sample center and they are mainly brought about by interference noise. To overcome this shortcoming, this paper introduces a novel algorithm, the MLT-KSVC algorithm, which is based on the recently presented LST-KSVC algorithm. Unlike the LST-KSVC algorithm, which assigns the same classification weights to all data samples, MLT-KSVC uses the MaxEnt model to build two classification weight matrices for leak samples. In these matrices, outliers are assigned small values, minimizing their negative effect on classification, and effectively solving the problem of outlier interference to the classification algorithm. Some researchers have combined entropy theory with SVMs algorithm. Zhang [21] used sample entropy to extract features from multichannel electroencephalography (EEG) signals; the extracted features were used for the classification of classic SVM. Xie [22] combined optimized variational mode decomposition, permutation entropy, and normalized Spearman correlation coefficient to extract features from ship-radiated noise (S-RN) signals. Then, these features were classified by a multi-class SVM algorithm. However, most of these combinations are not directly aimed at outliers, and their effect on outliers is very limited.

3. Theory

3.1. Background of LST-KSVC

LST-KSVC is a novel multi-class classification algorithm that uses the "one-versus-one-versus-rest" strategy [23] to evaluate all training samples with ternary output {−1, 0, +1}. In this section, we use $D=\left\{\left({\mathit{x}}_1,y_1\right),\left({\mathit{x}}_2,y_2\right),\dots ,\left({\mathit{x}}_m,y_m\right)\right\}$ as the training data set. Where ${\mathit{x}}_i$ represents the input sample in the m-dimensional real space $R_m$ and $y_i\in {\mathit{N}}^q$ is the q-class outputs $i=1,\dots ,m$. In the LST-KSVC classification, the formulas for two non-parallel hyperplanes are as follows:

$$\{\begin{array}{l}{w}_1^{+}\mathit{x}+b_{+}=0\\ w_2^{-}\mathit{x}+b_{-}=0\end{array}$$

(1)

where $w_1^{+}\mathrm{and}{w}_2^{-}\in {\mathit{R}}^{\mathrm{n}}$ are the normal matrices of the hyperplane, but $b_{+}$ and $b_{-}\in R$ are two constants. The decision functions of LST-KSVC are obtained by the following two optimization functions:

$$\underset{w_1^{+},b_{+},\mathit{\delta },\mathit{\xi },\mathit{\eta }}{\min }\frac{1}{2}{\mathit{\delta }}^T\mathit{\delta }+\frac{c_1}{2}{\mathit{\xi }}^T\mathit{\xi }+\frac{c_2}{2}{\mathit{\eta }}^T\mathit{\eta }$$

(2)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}\mathit{A}{w}_1^{+}+{\mathit{e}}_{+}{b}_{+}=\mathit{\delta }\\ -{\mathit{e}}_{-}-\left(\mathit{B}{w}_1^{+}+{\mathit{e}}_{-}{b}_{+}\right)=\mathit{\xi }\\ {\mathit{e}}_0\left(\epsilon -1\right)-\left(\mathit{C}{w}_1^{+}+{\mathit{e}}_0{b}_{+}\right)=\mathit{\eta }\end{array}$$

and

$$\underset{w_2^{-},b_{-},{\mathit{\delta }}^{*},{\mathit{\xi }}^{*},{\mathit{\eta }}^{*}}{\min }\frac{1}{2}{\mathit{\xi }}^{*T}{\mathit{\xi }}^{*}+\frac{c_3}{2}{\mathit{\delta }}^{*T}{\mathit{\delta }}^{*}+\frac{c_4}{2}{\mathit{\eta }}^{*T}{\mathit{\eta }}^{*}$$

(3)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}\mathit{B}{w}_2^{-}+{\mathit{e}}_{-}{b}_{-}={\mathit{\xi }}^{*}\\ {\mathit{e}}_{+}-\left(\mathit{A}{w}_2^{-}+{\mathit{e}}_{+}{b}_{-}\right)={\mathit{\delta }}^{*}\\ {\mathit{e}}_0\left(1-\epsilon \right)-\left(\mathit{C}{w}_2^{-}+{\mathit{e}}_0{b}_{-}\right)={\mathit{\eta }}^{*}\end{array}$$

In Equations (2) and (3), $\mathit{\delta }$ and ${\mathit{\delta }}^{*}$, $\mathit{\xi }$ and ${\mathit{\xi }}^{*}$, and $\mathit{\eta }$ and ${\mathit{\eta }}^{*}$ belong to the l₁-dimensional real space, l₂-dimensional real space, and l₃-dimensional real space, respectively. $\mathit{A},\mathit{B},\mathit{C}\in {\mathit{R}}^{l_i\times n}\left(i=1,2,3\right)$, $c_i\left(\mathrm{i}=1,\dots ,4\right)$, and $\epsilon$ are positive real factors. e₁ and e₂, e₀ are vectors of appropriate dimensions. The final classification decision functions of LST-KSVC in the linear case are determined as:

$$f\left(\mathit{x}\right)=\{\begin{array}{l}+1, \mathrm{if}{\mathit{x}}^T{w}_1^{+}+b_{+}>-1+\epsilon \\ -1, \mathrm{if} {\mathit{x}}^T{w}_2^{-}+b_{-}<1-\epsilon \\ 0, oth\mathit{e}rwis\mathit{e} \end{array}$$

(4)

The decision functions of LST-KSVC in the nonlinear case are determined as:

$$f\left(x\right)=\{\begin{array}{l}+1, \mathrm{if} k\left({\mathit{x}}^{\mathrm{T}},{\mathit{D}}^{\mathrm{T}}\right)w_1^{+}+{\mathrm{b}}_{+}>-1+\epsilon \\ -1, \mathrm{if} k\left({\mathit{x}}^{\mathrm{T}},{\mathit{D}}^{\mathrm{T}}\right)w_2^{-}+{\mathrm{b}}_{-}<1-\epsilon \\ 0, oth\mathit{e}rwis\mathit{e} \end{array}$$

(5)

3.2. MLT-KSVC

3.2.1. Background of MaxEnt Model

The principle of MaxEnt is to find the largest entropy model from the probability model set that satisfies the known constraint conditions. Given a data set ${\left\{x_i,y_i\right\}}_{i=1}^N$, the feature function of the data set is $f_i\left(x,y\right),i=1,2,\dots n$, and the constraints of the MaxEnt model are obtained according to the empirical distribution condition:

$${\displaystyle \sum }_{x,y}\tilde{P}\left(x\right)p\left(y|x\right)f\left(x,y\right)={\displaystyle \sum }_{x,y}\tilde{P}\left(x,y\right)f\left(x,y\right)$$

(6)

Assume that all sets $C$ satisfying the constraints are:

$$C=E_P\left(f_i\right)=E_{\tilde{P}}\left(f_i\right),i=1,2,\dots ,n$$

(7)

The conditional entropy is defined on the conditional probability distribution $P\left(Y|X\right)$ is:

$$H\left(P\right)=-{\displaystyle \sum }_{x,y}P\left(y,x\right)\lg P\left(y|x\right)=-{\displaystyle \sum }_{x,y}\tilde{P}\left(x\right)P(y|x)\lg P(y|x)$$

(8)

The goal is to find the corresponding $P(y|x)$ when $H\left(P\right)$ is the largest. Here, a minus sign is added to $H\left(P\right)$ to find the extreme minimum value. To make $-H\left(P\right)$ a convex function, it is convenient to use the convex optimization method to find the extreme value. Therefore, the loss function for MaxEnt is:

$$P^{*}=arg\underset{P\in C}{\max }H\left(P\right)$$

(9)

The objective function of the MaxEnt model is:

$$\underset{P\in C}{\min }{\displaystyle \sum }_{x,y}\tilde{P}\left(x\right)P(y|x)\lg P(y|x)$$

(10)

$$\mathrm{s}.\mathrm{t}.E_p\left(f_i\right)=E_{\tilde{p}}\left(f_i\right),{\displaystyle \sum }_{y}P\left(y|x\right)=1$$

(11)

The objective function of the MaxEnt model is an optimization problem with constraints. Based on the principle of Lagrangian duality, this problem can be transformed into an unconstrained optimization problem. First, we introduce a series of Lagrangian multipliers ${\omega }_0,{\omega }_1,\dots ,{\omega }_n$, and define the Lagrangian function $L\left(P,\omega \right)$ corresponding to this objective function:

$$L\left(P,\omega \right)=-H\left(P\right)+{\omega }_0\left[1-{\displaystyle \sum }_{y}P\left(y|x\right)\right]+{{\displaystyle \sum }}_{i=1}^n{\omega }_i\left(E_{\tilde{P}}\left(f_i\right)-E_P\left(f_i\right)\right)$$

(12)

Next, the optimization problem is transformed into $\underset{P\in C}{\min }L\left(P,\omega \right)$, where the Lagrangian function $L\left(P,\omega \right)$ must meet the constraints to obtain the extreme minimum value. After satisfying the constraints, $L\left(P,\omega \right)=\max L\left(P,\omega \right)$ is obtained, and then the optimization problem is transformed into an extreme minimum-maximum solution problem that is convenient for Lagrangian dual calculation:

$$\underset{P\in C}{\min }\underset{\omega }{\max }L\left(P,\omega \right)$$

(13)

Since $L\left(P,\omega \right)$ is a convex function with respect to $P$, according to the Lagrangian duality, the extreme minimum-maximum problem of $L\left(P,\omega \right)$ is equivalent to the extreme maximum-minimum problem:

$$\underset{P\in C}{\min }\underset{\omega }{\max }L\left(P,\omega \right)=\underset{\omega }{\max }\underset{P\in C}{\min }L\left(P,\omega \right)$$

(14)

Next, we find the extreme minimal problem $\underset{P\in C}{\min }L\left(P,\omega \right)$ of $\underset{\omega }{\max }\underset{P\in C}{\min }L\left(P,\omega \right)$, and $\underset{P\in C}{\min }L\left(P,\omega \right)$ is solved to obtain the function of $\omega$, denoted as $\Psi \left(\omega \right)$:

$$\Psi \left(\omega \right)=\underset{P\in C}{\min }L\left(P,\omega \right)=L\left(P_{\omega },\omega \right)$$

(15)

The solution $P_{\omega }$ of the above formula is:

$$P_{\omega }=arg\underset{P\in C}{\min }L\left(P,\omega \right)=P_{\omega }\left(y|x\right)={\left[Z_{\omega }\left(x\right)\right]}^{-1}\exp \left({\displaystyle \sum }_{i=1}^n{\omega }_i{f}_i\left(x,y\right)\right)$$

(16)

$$Z_{\omega }\left(x\right)={\displaystyle \sum }_{y}exp\left[{\displaystyle \sum }_{i=1}^n{\omega }_i{f}_i\left(x,y\right)\right]$$

(17)

In Equation (17), $f_i\left(x,y\right)$ represents the feature function and ${\omega }_i$ is the weight value of the feature function, and thus $P_{\omega }\left(y|x\right)$ is the MaxEnt model. The minimization problem is solved to obtain the weight value function of $\omega$, and the solved optimal solution is recorded as ${\omega }^{*}$:

$${\omega }^{*}=arg\underset{\omega }{\max }\left[{\displaystyle \sum }_{x,y}\tilde{P}\left(x,y\right){{\displaystyle \sum }}_{i=1}^n{\omega }_i{f}_i\left(x,y\right)+{\displaystyle \sum }_x\tilde{P}\left(x\right)\lg Z_{\omega }\left(x\right)\right]$$

(18)

A series of obtained weight values ${\omega }_i^{*}$ are filled into the matrix ${\mathit{W}}_{1+}$. Then, ${\omega }_i^{*}$ are reversed to obtain $-{\omega }_i^{*}$ and filled into matrix ${\mathit{W}}_{2-}$. Because outliers are distant from the data center, their probability $\left(P\right)$ value belonging to the data set is the lowest among all sample points. Hence, the weight value corresponding to outliers is much smaller than other normal sample points.

3.2.2. Linear MLT-KSVC

Similar to LST-KSVC, MLT-KSVC also has two hyperplanes. The two hyperplanes in linear MLT-KSVC are defined as follows:

$$\{\begin{array}{l}{w}_1^{+}\mathit{x}+{\mathit{b}}_{+}=0\\ w_2^{-}\mathit{x}+{\mathit{b}}_{-}=0\end{array}$$

(19)

where $w_1^{+}$ and $w_2^{-}$ are normal matrices of the hyperplane, $w_1^{+},w_2^{-}\in {\mathit{R}}^{\mathrm{n}}$, $b_{+}$ and $b_{-}$ belong to $R$ real number space. Next, ${\mathit{W}}_{1+}$ and ${\mathit{W}}_{2-}$ matrices are introduced into the objective functions of MLT-KSVC. This operation makes MLT-KSVC avoid the negative impact of outliers to the greatest extent. The obtained objective functions of MLT-KSVC are as follows:

$$\underset{w_1^{+},b_{+},\mathit{\alpha },\mathit{\gamma },\mathit{\eta }}{\min }\frac{1}{2}{\mathit{\alpha }}^T\mathit{\alpha }+\frac{c_1}{2}{\mathit{\gamma }}^T\mathit{\gamma }+c_2{\mathit{\lambda }}^T\mathit{\eta }$$

(20)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}{\mathit{W}}_{1+}\left(\mathit{A}{w}_1^{+}+{\mathit{e}}_{+}{b}_{+}\right)=\mathit{\alpha }\\ {\mathit{W}}_{2-}\left\{-{\mathit{e}}_{-}-\left(\mathit{B}{w}_1^{+}+{\mathit{e}}_{-}{b}_{+}\right)\right\}=\mathit{\gamma }\\ {\mathit{e}}_0\left(\mathit{\gamma }-1\right)-\left(\mathit{C}{w}_1^{+}+{\mathit{e}}_0{b}_{+}\right)=\mathit{\eta }\end{array}$$

and

$$\underset{w_2^{-},b_{-},{\mathit{\alpha }}^{*},{\mathit{\gamma }}^{*},{\mathit{\eta }}^{*}}{\min }\frac{1}{2}{\mathit{\gamma }}^{*T}{\mathit{\gamma }}^{*}+\frac{c_3}{2}{\mathit{\alpha }}^{*T}{\mathit{\alpha }}^{*}+c_4{\mathit{\lambda }}^{*T}{\mathit{\eta }}^{*}$$

(21)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}{\mathit{W}}_{2-}\left(\mathit{B}{w}_2^{-}+{\mathit{e}}_{-}{b}_{-}\right)={\mathit{\gamma }}^{*}\\ {\mathit{W}}_{1+}\left\{{\mathit{e}}_{+}-\left(\mathit{A}{w}_2^{-}+{\mathit{e}}_{+}{b}_{-}\right)\right\}={\mathit{\alpha }}^{*}\\ {\mathit{e}}_0\left(1-\mathit{\gamma }\right)-\left(\mathit{C}{w}_2^{-}+{\mathit{e}}_0{b}_{-}\right)={\mathit{\eta }}^{*}\end{array}$$

where $c_i\left(i=1,\dots ,4\right)$ is positive real number factor, ${\mathit{W}}_{1+}$ and ${\mathit{W}}_{2-}$ are obtained from the MaxEnt model, $\mathit{\alpha }$ and ${\mathit{\alpha }}^{*}$ are two vectors belonging to the l₁-dimensional real number space, $\mathit{\gamma }$ and ${\mathit{\gamma }}^{*}$ are two vectors belonging to the l₂-dimensional real number space, and $\mathit{\eta }$ and ${\mathit{\eta }}^{*}$ are two vectors belonging to the l₃-dimensional real number space. The matrices $\mathit{A},\mathit{B},\mathrm{and}\mathit{C}$ all belong to the real number space ${\mathit{R}}^{l_i\times n}\left(i=1,2,3\right)$, e₀ and e₁, e₂ are three adjustment vectors. $\mathit{\lambda }$ and ${\mathit{\lambda }}^{*}$ also belong to the l₃-dimensional real number space. They are calculated by the least-squares linear loss function and can be used to avoid the local convergence phenomenon of the objective function. Next, constraint conditions in Equations (20) and (21) are substitute into the objective functions so that the objective functions are optimized under the constraint conditions. The new objective functions obtained are as follows:

$$\begin{gathered} \underset{w_1^{+},b_{+},\mathit{\alpha },\mathit{\gamma },\mathit{\eta }}{\min }\frac{1}{2}\Vert {\mathit{W}}_{1+}{\left(\mathit{A}{w}_1^{+}+{\mathit{e}}_{+}{b}_{+}\right)\Vert }^2+\frac{c_1}{2}\Vert {\mathit{W}}_{2-}{\left\{-{\mathit{e}}_{-}-\left(\mathit{B}{w}_1^{+}+{\mathit{e}}_{-}{b}_{+}\right)\right\}\Vert }^2 \\ +c_2{\mathit{\lambda }}^T\left\{{\mathit{e}}_0\left(\mathit{\gamma }-1\right)-\left(\mathit{C}{w}_1^{+}+{\mathit{e}}_0{b}_{+}\right)\right\} \end{gathered}$$

(22)

and

$$\begin{gathered} \underset{w_2^{-},b_{-},{\mathit{\alpha }}^{*},{\mathit{\gamma }}^{*},{\mathit{\eta }}^{*}}{\min }\frac{1}{2}\Vert {\mathit{W}}_{2-}{\left(\mathit{B}{w}_2^{-}+{\mathit{e}}_{-}{b}_{-}\right)\Vert }^2+\frac{c_3}{2}\Vert {\mathit{W}}_{1+}{\left\{{\mathit{e}}_{+}-\left(\mathit{A}{w}_2^{-}+{\mathit{e}}_{+}{b}_{-}\right)\right\}\Vert }^2 \\ +c_4{\mathit{\lambda }}^{*T}\left\{{\mathit{e}}_0\left(1-\gamma \right)-\left(\mathit{C}{w}_2^{-}+{\mathit{e}}_0{b}_{-}\right)\right\} \end{gathered}$$

(23)

It can be seen that Equations (22) and (23) are two minimization problems. Partial derivatives of $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ are respectively determined from Equations (22) and (23), and then all partial derivatives are equal to zero:

$$\begin{array}{l}{\mathit{W}}_{1+}\mathit{A}\left({\mathit{W}}_{1+}\mathit{A}{w}_1^{+}+{\mathit{W}}_{1+}{\mathit{e}}_{+}{b}_{+}\right)+c_1{\mathit{W}}_{2-}\mathit{B}\left({\mathit{W}}_{1+}\mathit{B}{w}_1^{+}+{\mathit{W}}_{1+}{\mathit{e}}_{-}{b}_{+}+{\mathit{W}}_{1+}{\mathit{e}}_{-}\right)-c_2{\mathit{\lambda }}^{*T}\mathit{C}=0\\ {\mathit{W}}_{1+}{\mathit{e}}_{+}\left({\mathit{W}}_{1+}\mathit{A}{w}_1^{+}+{\mathit{W}}_{1+}{\mathit{e}}_{+}{b}_{+}\right)+c_1{\mathit{W}}_{2-}{\mathit{e}}_{-}\left({\mathit{W}}_{1+}\mathit{B}{w}_1^{+}-{\mathit{W}}_{1+}{\mathit{e}}_{-}{b}_{+}-{\mathit{W}}_{1+}{\mathit{e}}_{-}\right)-c_2{\mathit{\lambda }}^{*T}{\mathit{e}}_0=0\end{array}$$

(24)

$$\begin{array}{l}{\mathit{W}}_{2-}\mathit{B}\left({\mathit{W}}_{2-}\mathit{B}{w}_2^{-}+{\mathit{W}}_{2-}{\mathit{e}}_{-}{b}_{-}\right)+c_3{\mathit{W}}_{1+}\mathit{A}\left({\mathit{W}}_{1+}\mathit{A}{w}_2^{-}-{\mathit{W}}_{1+}{\mathit{e}}_{+}{b}_{-}+{\mathit{W}}_{1+}{\mathit{e}}_{+}\right)-c_4{\mathit{\lambda }}^{*T}\mathit{C}=0\\ {\mathit{W}}_{2-}{\mathit{e}}_{-}\left({\mathit{W}}_{2-}\mathit{B}{w}_1^{+}+{\mathit{W}}_{2-}{\mathit{e}}_{-}{b}_{-}\right)+c_3{\mathit{W}}_{1+}{\mathit{e}}_{+}\left({\mathit{W}}_{1+}\mathit{A}{w}_2^{-}+{\mathit{W}}_{1+}{\mathit{e}}_{+}{b}_{-}-{\mathit{W}}_{1+}{\mathit{e}}_{+}\right)-c_4{\mathit{\lambda }}^{*T}{\mathit{e}}_0=0\end{array}$$

(25)

Subsequently, Equations (24) and (25) are organized into the matrix forms (Equations (26) and (27)).

$$\{\begin{array}{l}{c}_2{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{M}+c_1\mathit{N}\right\}}^{-1}=\left[\begin{array}{l}{w}_1^{+}\\ b_{+}\end{array}\right]\\ \mathit{M}=\left[\begin{array}{ll}{\mathit{W}}_{1+}\mathit{A}{\mathit{W}}_{1+}\mathit{A}& -{\mathit{W}}_{1+}\mathit{A}{\mathit{W}}_{1+}{\mathit{e}}_{+}\\ -{\mathit{W}}_{1+}{\mathit{e}}_{+}{\mathit{W}}_{1+}\mathit{A}& {\mathit{W}}_{1+}{\mathit{e}}_{+}{\mathit{W}}_{1+}{\mathit{e}}_{+}\end{array}\right]\\ \mathit{N}=\left[\begin{array}{ll}{\mathit{W}}_{2-}\mathit{B}{\mathit{W}}_{1+}\mathit{B}& -{\mathit{W}}_{1+}\mathit{B}{\mathit{W}}_1^{-}{\mathit{e}}_{-}\\ -{\mathit{W}}_{1+}{\mathit{e}}_{-}{\mathit{W}}_{2-}\mathit{B}& {\mathit{W}}_{2-}{\mathit{e}}_{-}{\mathit{W}}_{1+}{\mathit{e}}_{-}\end{array}\right]\end{array}$$

(26)

$$\{\begin{array}{l}{c}_4{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{M}+c_3\mathit{N}\right\}}^{-1}=\left[\begin{array}{l}{w}_2^{-}\\ b_{-}\end{array}\right]\\ \mathit{M}=\left[\begin{array}{ll}{\mathit{W}}_{2-}\mathit{B}{\mathit{W}}_{2-}\mathit{B}& -{\mathit{W}}_{2-}\mathit{B}{\mathit{W}}_{2-}{\mathit{e}}_{-}\\ -{\mathit{W}}_{1+}{\mathit{e}}_{+}{\mathit{W}}_{1+}\mathit{A}& {\mathit{W}}_{1+}{\mathit{e}}_{+}{\mathit{W}}_{1+}{\mathit{e}}_{+}\end{array}\right]\\ \mathit{N}=\left[\begin{array}{ll}{\mathit{W}}_{1+}\mathit{A}{\mathit{W}}_{1+}\mathit{A}& -{\mathit{W}}_{1+}\mathit{A}{\mathit{W}}_{1+}{\mathit{e}}_{+}\\ -{\mathit{W}}_{2-}{\mathit{e}}_{+}{\mathit{W}}_1^{+}\mathit{A}& {\mathit{W}}_{1+}{\mathit{e}}_{-}{\mathit{W}}_{1+}{\mathit{e}}_{-}\end{array}\right]\end{array}$$

(27)

Therefore, the solutions for $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ can be solved by Equations (26) and (27). After $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ are solved, Equation (19) can be used to construct two linear classification hyperplanes of MLT-KSVC. The proposed linear MLT-KSVC is summarized in Algorithm 1.

Algorithm 1: Linear MLT-KSVC

(1) Initialize matrices $\mathit{A},\mathit{B},\mathrm{and}\mathit{C}\in {\mathit{R}}^{l_i\times n}\left(i=1,2,3\right)$, e0 and e1, e2.

(2) Run the program based on MaxEnt structure to obtain the weight matrices ${\mathit{W}}_{1+}$, ${\mathit{W}}_{2-}$.

(3) Select the kernel parameter as “linear”, and use the grid search method to optimize the hyperparameter C and penalty factor G.

(4) Initialize $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$, $\mathit{\alpha }$ and ${\mathit{\alpha }}^{*}$, $\mathit{\gamma }$ and ${\mathit{\gamma }}^{*}$, $\mathit{\eta }$ and ${\mathit{\eta }}^{*}$, $\mathit{\lambda }$ and ${\mathit{\lambda }}^{*}$.

(5) For iter ≥ 0:

Calculate

$c_2{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{M}+c_1\mathit{N}\right\}}^{-1}=\left[\begin{array}{l}{w}_1^{+}\\ b_{+}\end{array}\right]$ $c_4{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{M}+c_3\mathit{N}\right\}}^{-1}=\left[\begin{array}{l}{w}_2^{-}\\ b_{-}\end{array}\right]$

(6) End for convergence and obtain the optimal solutions: $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$.

3.2.3. Nonlinear MLT-KSVC

Considering that the distribution of sample points is not regularly linearly separable in real classification, extending the linear classification theory of MLT-KSVC to a nonlinear version is necessary. In nonlinear MLT-KSVC case, two classification hyperplanes are no longer linear functions, they are defined as follows:

$$\{\begin{array}{l}K{w}_1^{+}\left({\mathit{x}}^T,{\mathit{D}}^T\right)+b_{+}=0\\ K{w}_2^{-}\left({\mathit{x}}^T,{\mathit{D}}^T\right)+b_{-}=0\end{array}$$

(28)

where $K(·)$ is an arbitrary kernel function [24]. It maps the complex linear inseparable problem to a high-dimensional space, transforming the linear inseparable problem into a linear separable problem. Similar to the linear MLT-KSVC, after the classification hyperplanes are obtained, the objective functions for solving $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ should be defined. The objective functions of the nonlinear MLT-KSVC are defined as follows:

$$\underset{w_1^{+},b_{+},\mathit{\alpha },\mathit{\gamma },\mathit{\eta }}{\min }\frac{1}{2}{\mathit{\alpha }}^T\mathit{\alpha }+\frac{c_1}{2}{\mathit{\gamma }}^T\mathit{\gamma }+c_2{\mathit{\lambda }}^T\mathit{\eta }$$

(29)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}{\mathit{W}}_{1+}\left(K{w}_1^{+}\left(\mathit{A},{\mathit{D}}^T\right)+{\mathit{e}}_{+}{b}_{+}\right)=\mathit{\alpha }\\ {\mathit{W}}_{2-}\left\{-{\mathit{e}}_{-}-\left(K{w}_1^{+}\left(\mathit{B},{\mathit{D}}^T\right)+{\mathit{e}}_{-}{b}_{+}\right)\right\}=\mathit{\gamma }\\ {\mathit{e}}_0\left(\mathit{\gamma }-1\right)-\left(K{w}_1^{+}\left(\mathit{C},{\mathit{D}}^T\right)+{\mathit{e}}_0{b}_{+}\right)=\mathit{\eta }\end{array}$$

and

$$\underset{w_2^{-},b_{-},{\mathit{\alpha }}^{*},{\mathit{\gamma }}^{*},{\mathit{\eta }}^{*}}{\min }\frac{1}{2}{\mathit{\gamma }}^{*T}{\mathit{\gamma }}^{*}+\frac{c_3}{2}{\mathit{\alpha }}^{*T}{\mathit{\alpha }}^{*}+c_4{\mathit{\lambda }}^{*T}{\mathit{\eta }}^{*}$$

(30)

$$\mathrm{s}.\mathrm{t}.\{\begin{array}{l}{\mathit{W}}_{2-}\left(K{w}_2^{-}\left(\mathit{B},{\mathit{D}}^T\right)+{\mathit{e}}_{-}{b}_{-}\right)={\mathit{\gamma }}^{*}\\ {\mathit{W}}_{1+}\left\{{\mathit{e}}_{+}-\left(K{w}_2^{-}\left(\mathit{A},{\mathit{D}}^T\right)+{\mathit{e}}_{+}{b}_{-}\right)\right\}={\mathit{\alpha }}^{*}\\ {\mathit{e}}_0\left(1-\mathit{\gamma }\right)-\left(K{w}_2^{-}\left(\mathit{C},{\mathit{D}}^T\right)+{\mathit{e}}_0{b}_{-}\right)={\mathit{\eta }}^{*}\end{array}$$

In the constraints of the objective functions Equation (29) and Equation (30), classification weight matrices ${\mathit{W}}_{1+}$ and ${\mathit{W}}_{2-}$ are considered, which implies that the interference of outliers can also be reduced in the nonlinear MLT-KSVC case. The constraints are substituted into objective functions of Equation (29) and Equation (30), and the following formulas obtained:

$$\begin{gathered} \underset{w_1^{+},b_{+},\mathit{\alpha },\mathit{\gamma },\mathit{\eta }}{\min }\frac{1}{2}\Vert {\mathit{W}}_{1+}{\left(K{w}_1^{+}\left(\mathit{A},{\mathit{D}}^T\right)+{\mathit{e}}_{+}{b}_{+}\right)\Vert }^2 \\ +\frac{c_1}{2}\Vert {\mathit{W}}_{2-}{\left\{-{\mathit{e}}_{-}-\left(K{w}_1^{+}\left(\mathit{B},{\mathit{D}}^T\right)+{\mathit{e}}_{-}{b}_{+}\right)\right\}\Vert }^2 \\ +c_2{\mathit{\lambda }}^T\left\{{\mathit{e}}_0\left(\mathit{\gamma }-1\right)-\left(K{w}_1^{+}\left(\mathit{C},{\mathit{D}}^T\right)+{\mathit{e}}_0{b}_{+}\right)\right\} \end{gathered}$$

(31)

$$\begin{gathered} \underset{w_2^{-},b_{-},{\mathit{\alpha }}^{*},{\mathit{\gamma }}^{*},{\mathit{\eta }}^{*}}{\min }\frac{1}{2}{\Vert {\mathit{W}}_{2-}\left(K{w}_2^{-}\left(\mathit{B},{\mathit{D}}^T\right)+{\mathit{e}}_{-}{b}_{-}\right)\Vert }^2 \\ +\frac{c_3}{2}\Vert {\mathit{W}}_{1+}{\left\{{\mathit{e}}_{+}-\left(K{w}_2^{-}\left(\mathit{A},{\mathit{D}}^T\right)+{\mathit{e}}_{+}{b}_{-}\right)\right\}\Vert }^2 \\ +c_4{\mathit{\lambda }}^{*T}\left\{{\mathit{e}}_0\left(1-\mathit{\gamma }\right)-\left(K{w}_2^{-}\left(\mathit{C},{\mathit{D}}^T\right)+{\mathit{e}}_0{b}_{-}\right)\right\} \end{gathered}$$

(32)

Partial derivatives with respect to $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ are solved, and then partial derivative equations are transformed in matrix form:

$$\{\begin{array}{l}{c}_2{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{U}+c_1\mathit{V}\right\}}^{-1}=\left[\begin{array}{l}{w}_1^{+}\\ b_{+}\end{array}\right]\\ \mathit{U}=\left[\begin{array}{ll}{\left({\mathit{W}}_{1+}K\left(\mathit{A},{\mathit{D}}^T\right)\right)}^2& {\mathit{W}}_{1+}^{2}K\left(\mathit{A},{\mathit{D}}^T\right){\mathit{e}}_{+}\\ {\mathit{W}}_{1+}^2{\mathit{e}}_{+}K\left(\mathit{A},{\mathit{D}}^T\right)& {\left({\mathit{W}}_{1+}{\mathit{e}}_{+}\right)}^2\end{array}\right]\\ \mathit{V}=\left[\begin{array}{ll}{\mathit{W}}_{1+}^2{\mathit{W}}_{2-}{\left(K\left(\mathit{B},{\mathit{D}}^T\right)\right)}^2& {\mathit{W}}_{1+}^{2}K\left(\mathit{B},{\mathit{D}}^T\right){\mathit{e}}_{-}\\ {\mathit{W}}_{1+}{\mathit{e}}_{-}{\mathit{W}}_{2-}K\left(\mathit{B},{\mathit{D}}^T\right)& {\mathit{W}}_{2-}{\mathit{W}}_{1+}{\mathit{e}}_{-}^2\end{array}\right]\end{array}$$

(33)

$$\{\begin{array}{l}{c}_4{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{U}+c_3\mathit{V}\right\}}^{-1}=\left[\begin{array}{l}{w}_2^{-}\\ b_{-}\end{array}\right]\\ \mathit{U}=\left[\begin{array}{ll}{\left({\mathit{W}}_{2-}K\left(\mathit{B},{\mathit{D}}^T\right)\right)}^2& {\mathit{W}}_{2-}^{2}K\left(\mathit{B},{\mathit{D}}^T\right){\mathit{e}}_{-}\\ {\mathit{W}}_{1+}^2{\mathit{e}}_{+}K\left(\mathit{A},{\mathit{D}}^T\right)& {\left({\mathit{W}}_{1+}{\mathit{e}}_{+}\right)}^2\end{array}\right]\\ \mathit{V}=\left[\begin{array}{ll}{\left({\mathit{W}}_{1+}K\left(\mathit{A},{\mathit{D}}^T\right)\right)}^2& {\mathit{W}}_{1+}^2{\mathit{e}}_{+}K\left(\mathit{A},{\mathit{D}}^T\right)\\ {\mathit{W}}_{2-}{\mathit{W}}_{1+}{\mathit{e}}_{+}K\left(\mathit{A},{\mathit{D}}^T\right)& {\left({\mathit{W}}_{1+}{\mathit{e}}_{-}\right)}^2\end{array}\right]\end{array}$$

(34)

After obtaining the solutions of $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$, the nonlinear MLT-KSVC classification hyperplanes are established according to Equation (28). The proposed nonlinear MLT-KSVC is summarized in Algorithm 2.

Algorithm 2: Nonlinear MLT-KSVC

(1) Initialize matrices $\mathit{A}, \mathit{B},\mathrm{and}\mathit{C} \in {\mathit{R}}^{l_i\times n}\left(i=1,2,3\right)$, e0 and e1, e2.

(2) Run the program based on MaxEnt structure to obtain the weight matrices ${\mathit{W}}_{1+}$, ${\mathit{W}}_{2-}$.

(3) Select the kernel parameter as “RBF” or “Gaussian”, and use the grid search method to optimize the hyperparameter C and penalty factor G.

(4) Initialize $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ under the selected nonlinear kernel function. (5) Initialize $\mathit{\alpha }$ and ${\mathit{\alpha }}^{*}$, $\mathit{\gamma }$ and ${\mathit{\gamma }}^{*}$, $\mathit{\eta }$ and ${\mathit{\eta }}^{*}$, $\mathit{\lambda }$ and ${\mathit{\lambda }}^{*}$.

(6) For iter ≥ 0:

Calculate

$c_2{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{U}+c_1\mathit{V}\right\}}^{-1}=\left[\begin{array}{l}{w}_1^{+}\\ b_{+}\end{array}\right]$ $c_4{\mathit{\lambda }}^{*T}\left[\begin{array}{l}\mathit{C}\\ {\mathit{e}}_0\end{array}\right]{\left\{\mathit{U}+c_3\mathit{V}\right\}}^{-1}=\left[\begin{array}{l}{w}_2^{-}\\ b_{-}\end{array}\right]$

(7) End for convergence and obtain the optimal solutions: $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$.

3.3. Multi-Classification Rule of MLT-KSVC

MLT-KSVC is an improved algorithm of LST-KSVC. As described in Section 3.1, LST-KSVC is a classification algorithm based on the "one-versus-one-versus-rest" strategy. Therefore, MLT-KSVC is also a classification algorithm based on "one-versus-one-versus-rest" strategy. In the "one-versus-one-versus-rest" strategy, the classification algorithm outputs three labels {+1, 0, −1}. When the classification number is q (q > 2), q(q − 1)/2 MLT-KSVC sub-classifiers are required to complete the classification. This classification process is a voting process. In vote classification, MLT-KSVC labels “+1” to i-th class samples, “−1” to j-th class samples, and “0” to all remaining classes, respectively, where $i,j\in \left\{1,2,\dots ,q\right\}$. Then, the hyperplane parameters $w_1^{+}$, $b_{+}$ and $w_2^{-}$, $b_{-}$ of the (i, j)th sub-classifier are obtained from Equations (26), (27), (33) and (34). In linear MLT-KSVC case, classification labels are determined using the following function:

$$f\left(\mathit{x}\right)=\{\begin{array}{l}+1, \mathrm{if} {\mathit{x}}^T{w}_1^{+}+b_{+}>-1+\epsilon \\ -1, \mathrm{if} {\mathit{x}}^T{w}_2^{-}+b_{-}<1-\epsilon \\ 0, oth\mathit{e}rwis\mathit{e} \end{array}$$

(35)

In nonlinear MLT-KSVC case, the corresponding decision function is:

$$f\left(\mathit{x}\right)=\{\begin{array}{l}+1, \mathrm{if} k\left({\mathit{x}}^T,{\mathit{D}}^T\right)w_1^{+}+b_{+}>-1+\epsilon \\ -1, \mathrm{if} k\left({\mathit{x}}^T,{\mathit{D}}^T\right)w_2^{-}+b_{-}<1-\epsilon \\ 0, oth\mathit{e}rwis\mathit{e} \end{array}$$

(36)

Finally, after q(q − 1)/2 sub-classifiers, the test samples are classified as the label with the most votes.

4. MLT-KSVC-Based Leak Detection

4.1. Overview of the Recommended Leak Detection Procedure

In this experimental section, the proposed MLT-KSVC algorithm is used for water supply pipe leak identification. A schematic of the experiment is shown in Figure 1. The experiment includes four steps as follows.

Step 1: Piezoelectric (PZT) acoustic sensor was used to acquire the vibro-acoustic emission (VAE) data on the pipe, which VAE data was used as the data source for the next feature extraction.

Step 2: Eight methods, including standard deviation, kurtosis, variance, RMS, margin, mean, waveform factor, and peak factor, were used to extract feature values from the VAE data source. These feature values have been proven by previous studies [25,26] that they can indicate the features of different leak severity. Then, the extracted features constituted the leak sample data.

Step 3: The extracted feature values were simplified by the Delaunay triangulation (DT) algorithm [27]. The simplification process was to remove redundant points in the sample center and retain the mainframe of the samples.

Step 4: According to the distribution characteristics of the sample points, MaxEnt was used to establish classification weight matrices ${\mathit{W}}_{1+}$ and ${\mathit{W}}_{2-}$, and then the samples were classified by the proposed algorithm, and finally leak detection was completed.

4.2. Acquisition of VAE Data

To simulate the pipe leak condition, a 200-m water pipeline system was built. Figure 2a shows the pipe leak test platform. The entire platform consists of pipelines, PZT sound sensor, PZT driving module, signal attenuator, National Instruments (NI) data acquisition (DAQ) device, and a computer. The PZT sensor has a resonant frequency of 18 KHz and a frequency range of 0.35–6 KHz; its output voltage signal was amplified by a PZT driving module (preamplifier). To prevent the amplified voltage of the PZT sensor from exceeding the input range of DAQ device, we designed a signal attenuator between the preamplifier and DAQ card. The maximum sampling rate of DAQ card is 1 MHz. Figure 2b shows the PZT sensor was mounted on the pipe away from the leak source. Different opening degrees of the faucet were used to simulate leak conditions. Three leak situations were utilized: background noise (no leak), small leak, and large leak (shown in Figure 3left). We collected 300 sets of data for each leak situation. Finally, 900 sets of data for the three leak situations were obtained. The right of Figure 3 shows a group of time-domain waveforms corresponding to the above three leak situations. It is not difficult to see that the leak time-domain waveform becomes more and more intense with the increasing leak volume. During the data sampling process, the sampling rate was set at 10 KHz, and the single sampling time was 10 s.

4.3. Feature Extraction of VAE Data

Eight statistical indices were used to extract feature values from the collected leak VAE data. Subsequently, these feature values were used to construct a training data set T for classification. To facilitate the visualization of the classification process, we selected standard deviation and kurtosis feature values as an example and drew a two-dimensional (2−D) scatter diagram of the feature values (Figure 4a). The classification experiment used this 2−D scatter diagram as a study case.

4.4. DT Pre-Processing of Data

It is well known that SVM and its improved algorithms are supervised machine learning algorithms, which are mainly suitable for the training and testing of the small-scale sample. Thus, original leak sample data should be simplified. However, some redundant data within the sample points are not helpful for classification. We used the DT algorithm to simplify the original leak sample data. The main process is to retain the mainframe of the samples and remove the redundant points located at the center of the sample points. Figure 4 shows the original sample data and the simplified sample points with redundant data removed.

4.5. MLT-KSVC Classification for Leak Detection

In MLT-KSVC leak classification, the first step was used, MaxEnt frame, to establish the classification weight matrices ${\mathit{W}}_{1+}$ and ${\mathit{W}}_{2-}$ for leak samples. In the process of constructing the weight matrix, the data of sample center was given a large weight, while the outliers were given a small weight. Figure 5 shows three typical outliers generated by the interference of environmental noise. Therefore, these sample points can be classified. Here, LST-KSVC was also used to classify the same leak sample points as a comparative experiment example for MLT-KSVC. In the classification process, we used the grid search method to optimize the hyperparameters C and G of MLT-KSVC and LST-KSVC. In nonlinear MLT-KSVC and nonlinear LST-KSVC classification, we selected the ‘RBF’ as the kernel function, in which all programs were run on MATLAB 2019a.

Figure 6 shows the classification results of linear MLT-KSVC and linear LST-KSVC, in which black points represent large leaks (+1 class), green points represent small leaks (−1 class), blue points represent background noise (no leak, 0 class), and red circles represent support vector points. It can be seen from Figure 6b that a misclassification color block (within the red oval box) appears in linear LST-KSVC, but linear MLT−KSVC overcomes this shortcoming. By comparing Figure 6a,b, it can be found that the classification color block of linear MLT-KSVC are more regular than linear LST-KSVC, which means that the generalization ability of linear MLT-KSVC is stronger than that of linear LST-KSVC.

Figure 7 shows the classification results of nonlinear MLT-KSVC and nonlinear LST-KSVC algorithms. In an ideal classification state, the boundary inside the red oval box in Figure 7a,b should be similar to a straight line. But the boundary inside the red oval box in Figure 7a,b is not straight, we conjecture that the phenomenon is a negative result caused by outliers. Comparing the red oval box of Figure 7a,b, we found that the nonlinear MLT-KSVC is slightly affected by outliers, but this is much less than the nonlinear LST-KSVC. The classification boundary of nonlinear LST-KSVC is not very regular in Figure 7b. It is not difficult to see that the classification boundary in the black rectangular box is wrong in Figure 7b. We speculate that the misclassification boundary is caused by the overfitting in the nonlinear LST-KSVC. However, the classification boundary (Figure 7a) of nonlinear MLT-KSVC is still more regular compare to that of nonlinear LST-KSVC, which shows that the outliers of sample points have less influence on the linear MLT-KSVC algorithm.

To further compare the performance of the two algorithms, we used MLT-KSVC and LST-KSVC to conduct multiple nonlinear classification experiments to obtain optimal hyperparameter C, penalty factor G, and cross-validation accuracy based on these sample points. Figure 8 shows a three-dimensional distribution map that combined optimal hyperparameter C, penalty factor G, and cross-validation accuracy. Ten-fold cross-validation was used to cross-validate the algorithms, which divided the samples into 10 parts, and took 9 parts as training data and 1 part as test data, in turn, to implement experiments. It can be seen from the Figure 8 that the optimal cross-validation accuracy rate of nonlinear MLT-KSVC is as high as 96.2264%, while that of nonlinear LST-KSVC is only 87.1698%. In nonlinear MLT-KSVC classification, the mean of cross-validation accuracy is 96.15%, and the standard deviation of cross-validation accuracy is 0.0330. In nonlinear LST-KSVC classification, the mean of cross-validation accuracy is 87.60%, and the standard deviation of cross-validation accuracy is 0.0610. Then, we can obtain the overall confusion matrix in

Table 1. Confusion matrix.

Predicted +1	TP	FN	FR
Predicted −1	FP	TN	FR
Predicted 0	FP	FN	TR
	True +1	True −1	True 0

and Figure 9. The following metrics were calculated by us.

$$\mathrm{Accuracy}=\frac{TP+T\mathit{N}+TR}{TP+T\mathit{N}+TR+FP+F\mathit{N}+FR}·100\%$$

(37)

$$\mathrm{Sensitivity}=\frac{TP}{TP+F\mathit{N}+FR}·100\%$$

(38)

In Table 1, TP represents the true positive judgment, TN represents the true negative judgment, TR represents the true rest judgment, FP represents the false positive judgment, FN represents a false negative judgment, and FR represents a false rest judgment. In Figure 9, L-leak represents large leak, S-leak represents small leak, BG-noise represents background noise. In Figure 9a, the overall accuracy of nonlinear MLT-KSVC is 96.2%, and the sensitivity of nonlinear MLT-KSVC is 93.5%. In Figure 9b, the overall accuracy of nonlinear LST-KSVC is 87.2%, and the sensitivity of nonlinear LST-KSVC is 92.6%.

Then, we changed the feature number of leak samples and used the classical Multi-SVM, TwinSVC, TwinKSVC, LST-KSVC and MLT-KSVC to classify these samples. In these experiments, we still chose ‘RBF’ as the kernel function of the classification algorithm.

Table 2. Leak classification accuracy and calculation time results of Multi-SVM, TwinSVC, TwinKSVC, LST-KSVC and MLT-KSVC.

Feature Number	Outputs	Multi-SVM	TwinSVC	TwinKSVC	LST-KSVC	MLT-KSVC
1	accuracy (%)	62.2641	74.7170	75.8491	85.6604	92.4528
	calculation time (s)	3.0124	2.1105	1.9012	0.2207	0.2310
2	accuracy (%)	60.3773	74.3396	76.9811	87.1698	96.2264
	calculation time (s)	2.6820	2.0164	1.9146	0.2298	0.2304
3	accuracy (%)	60.7547	75.4717	77.7358	87.9245	96.9811
	calculation time (s)	2.7504	2.3738	1.9872	0.2860	0.2834
4	accuracy (%)	61.1321	73.2075	77.3585	89.4340	95.8491
	calculation time (s)	2.7018	2.1083	1.8509	0.3019	0.3095
5	accuracy (%)	63.0189	75.8491	78.1132	90.1887	95.0943
	calculation time (s)	2.7239	2.1834	1.8957	0.3672	0.3932
6	accuracy (%)	63.7736	75.0291	78.4906	90.9434	97.7358
	calculation time (s)	2.6713	2.5263	1.9863	0.4153	0.4035
7	accuracy (%)	63.3962	76.6038	78.8679	93.2075	98.1132
	calculation time (s)	3.1121	2.5967	2.1558	0.4507	0.4518
8	accuracy (%)	64.1509	76.2264	78.8680	93.9623	97.3585
	calculation time (s)	3.2386	2.6093	2.4853	0.4964	0.5086

shows the comparison of the classification accuracy and calculation time of the classical Multi-SVM [28], TwinSVC [29], TwinKSVC [30], LST-KSVC and MLT-KSVC. In Table 2, different feature number correspond to different feature types, and the corresponding relationship between them is as follows:

• Feature number = 1: standard deviation.
• Feature number = 2: standard deviation, kurtosis.
• Feature number = 3: standard deviation, kurtosis, variance.
• Feature number = 4: standard deviation, kurtosis, variance, RMS.
• Feature number = 5: standard deviation, kurtosis, variance, RMS, margin.
• Feature number = 6: standard deviation, kurtosis, variance, RMS, margin, mean.
• Feature number = 7: standard deviation, kurtosis, variance, RMS, margin, mean, waveform factor.
• Feature number = 8: standard deviation, kurtosis, variance, RMS, margin, mean, waveform factor, peak factor.

From Table 2, it can be seen that the calculation time of nonlinear MLT-KSVC is similar to that of nonlinear LST-KSVC, but the accuracy is higher than that of nonlinear LST-KSVC and other classifiers.

4.6. Application Discussion of MLT-KSVC Classification in City Water Pipelines

Then, we will briefly discuss the MLT-KSVC application for leak recognition of city water pipelines. Figure 10 presents a complete schematic diagram of pipeline leak recognition system. From bottom to top, the recognition system is divided into three layers.

• The first is the physical layer, which uses many data acquisition (DAQ) nodes to collect the pipeline acoustic vibration data and transmit the pipe data to the relay gateway. The DAQ node comprises a sensing module, analog to digital conversion (ADC) module and data wireless transmission module. In order to obtain the pipeline position information from the DAQ node, it is necessary to take the DAQ node number as the frame header of the pipe data when transmitting the pipeline data frame.
• The second is data transmission layer. After the relay gateway received the wireless pipe data frame, and then the relay gateway uses the public networks (3G/4G/5G) to transmit the pipe data to the cloud server.
• The third is the application layer. In this layer, the cloud server will run the MLT-KSVC classification algorithm model. The application layer performs feature extraction preprocessing on the pipeline data; then, the classification model will complete the leak recognition task based on the extracted features. The user can check the leak recognition results through the terminal device, and take corresponding maintenance and repair plans for different leak statuses.

5. Conclusions

This paper used the MaxEnt model to establish two weight matrices that can be used for classification. These weight matrices can make the MLT-KSVC classification algorithm reduce sensitivity to outliers. From the linear classification result, a misclassification color block appears in linear LST-KSVC, but the linear MLT-KSVC can aovid this shortcoming. Whether in linear or nonlinear classification results perspective, the MLT-KSVC has more regular classification color blocks for leak samples than corresponding LST-KSVC, which proves that MLT-KSVC is less sensitive to outliers, and the generalization ability of MLT-KSVC is stronger than that of LST-KSVC. The MLT-KSVC and LST-KSVC took the similar calculation time to complete the classification, but still much less than Multi-SVM, TwinSVC and TwinKSVC. Although the calculation time remains comparable to that of LST-KSVC, MLT-KSVC is more accurate in classifying leak sample points. However, MLT-KSVC has some limitations. For instance, when the data sample is very large, the MLT-KSVC algorithm may crash and even terminate. Solving this problem will be an important research direction in the future.