Analysis of Estimation of Soundness and Deterioration Factors of Sewage Pipes Using Machine Learning

Abstract

In Japan, there are a massive number of sewage pipes buried in the ground. In order to operate sustainable sewerage systems, it is necessary to estimate the soundness of sewage pipes accurately and to conduct repairs and other measures according to the soundness of the pipes. In previous studies, statistical and machine learning methods have been used to estimate the soundness of sewage pipes, but all of these studies formulated the soundness of sewage pipes as a binary classification problem (e.g., good or poor). In contrast, this study attempted to predict the soundness of sewage pipes in more detail by setting up four classes of pipe soundness. Inspection data of sewage pipes in City A were used as training data, and XGBoost was used as the machine learning model. Machine learning models have a high prediction performance, but the uncertainty of the prediction basis is an issue. In this study, SHAP (Shapley additive explanations), an Explainable AI method, was used to interpret the model to clarify the influence of sewer pipe specifications (e.g., pipe age) and topographical specifications (e.g., annual precipitation) on the prediction, and to extract deterioration factors. By interpreting the model using SHAP, it was possible to quantify whether factors such as pipe age and pipe length have a positive or negative impact on the deterioration of sewage pipes. Previous studies using machine learning methods have not clarified whether factors have a positive or negative effect on deterioration. The knowledge on deterioration factors obtained in this study may provide useful information for the sustainable operation of sewage systems.

1. Background and Purpose of This Research

1.1. Background of Research

Sewage systems are essential for the sustainability of local communities, such as the prevention of waterborne diseases, the maintenance of public health by improving living conditions, the securing of water resources by improving water quality, and the prevention of flooding within cities.

Currently, the sewage system’s penetration rate in Japan is approximately 80.1% [1], as of the financial year 2020 (FY2020), and the total length of sewage pipes is approximately 490,000 km as of FY2020, with a massive infrastructure stock present. Moreover, as of the financial year 2021 (FY2021), the total length of sewage pipes that have reached a standard service life of 50 years is approximately 30,000 km (approximately 6% of the total length), which will rapidly increase to approximately 90,000 km (18%) in 10 years and approximately 200,000 km (40%) in 20 years (Figure 1) [2]. The causes of this deterioration include, but are not limited to, time-dependent factors owing to the environments in which sewage pipes are required to function; for example, they are exposed to traffic loads and the microbial corrosion of concrete, and their aging progression is faster than that of other structures. Furthermore, in 2020, the aging of sewage pipes caused 2700 road subsidence accidents, and this aging can thus cause serious social problems [2]. Preventing such situations requires the appropriate implementation of inspection and repair. Sewage pipe inspection is currently conducted via visual and video-camera surveys; however, the total length of pipes surveyed in FY2018 via visual and video-camera inspections combined was only 6686 km (1.4% of the total length [3]), and at this pace, 72 years would be required to survey the entire sewage pipe network once, which does not allow for appropriate maintenance and management. Moreover, Japan has faced shortages in financial resources, human resources, and technology owing to the financial difficulties of local governments, its declining birthrate, and its aging population. It is therefore difficult to perform a comprehensive survey of such a large number of sewage pipes.

Therefore, there is a need to manage sustainable sewage operations by formulating a stock management plan, estimating the soundness of sewage pipes for which field surveys were not conducted based on a small number of field survey results, and prioritizing the inspection/survey and repair/reconstruction of facilities. Currently, the prioritization of sewage pipe management is often based on a risk matrix (Figure 2) [4].

The evaluation approach for the occurrence probability of failure and the scale of damage in the risk matrix differs according to the local government that manages the sewage pipes. For example, in Kasukabe City, the impact is classified into four ranks based on the importance of the pipeline and the diameter of the pipe, such as trunk pipes and pipes buried under emergency transport roads, as the scale of damage [5]. On the other hand, the occurrence probability of failure is assessed by classifying the occurrence probability into five ranks, such as Rank 1, for pipes less than 10 years old, and Rank 2, for pipes between 10 and 30 years old.

Figure 3 presents a boxplot of the soundness and age of sewage pipes in City A. Details of the soundness index are described in Section 3 of this paper, but the evaluation is conducted using four ranks, from Rank A, with the lowest soundness, to Rank D, with the highest soundness, where N_A represents the amount of data belonging to Rank A, and the amount of data belonging to Ranks B, C, and D is similarly expressed as N_B, N_C, and N_D, respectively. Figure 3 shows that Ranks C and D, which comprise relatively high soundness, tend to have a younger pipe age compared to those in the case of Ranks A and B. However, there are pipes even among Ranks A and B that have a low age, and it can be confirmed that there are many overlapping pipes in the interquartile range. These results indicate that, when using only the pipe age to evaluate the soundness of sewage pipes, an appropriate evaluation cannot be conducted for old pipes with high soundness and young pipes with low soundness, and it is difficult to screen locations needed for inspection efficiently. Therefore, when evaluating the soundness of pipes, there is a high possibility that an inaccurate result would be obtained by using only the pipe age, and such evaluations should thus be conducted while taking into consideration various factors that decrease the soundness of pipes.

1.2. Purpose of This Study

In line with the purpose of this study, this chapter focuses on the following two points and describes the novelty of this study on the basis of these two points.

• The estimation of soundness (4 classes) in sewage pipes using machine learning models.
• The interpretation of the prediction basis in machine learning models using Explainable AI and the identification of deterioration factors in sewage pipes.

We used sewage-pipe inspection results and built a model for estimating the soundness of concrete pipes among sewage pipes at the span level using data that can be obtained without conducting surveys (e.g., pipe age, overburden, and surrounding environment information). Current inspection methods make it impossible to conduct soundness assessments of all pipes due to budgetary problems in the local governments. The method proposed in this study does not require a TV camera survey and therefore makes it possible to conduct soundness assessments on all pipes that are difficult to inspect.

Figure 4 presents the analysis procedure in this study. In this study, we built and evaluated a machine-learning model using the results of sewage pipe inspections conducted in City A. We used positional information recorded in the inspection results to acquire information regarding the underground pipeline environment. We built a machine-learning model for the purposes of estimating the soundness of sewage pipes using acquired information from the buried environment and the pipeline specification information recorded in the inspection results. In this study, we built two types of machine learning model using different data, depending on the shaping method of the training data that were used for the model training. Machine learning models have a high predictive performance, but the predictive basis of the models is not clear. When using machine learning to estimate the soundness of infrastructure structures, it is important to be clear about the basis on which the predictions have been made. In recent years, there has been a lot of research on Explainable AI (XAI) to reduce the predictive uncertainty in machine learning models. In this study, the machine learning model is interpreted using SHAP (Shapley additive explanations) [6], one of the XAI methods, to examine the influence of sewage pipe specifications and the buried environment (e.g., pipe age, and pipe length) on the prediction of the machine learning model. The model also extracts the deterioration factors. As mentioned in the previous section, the current maintenance and management guidelines [4] list only pipe age as a deterioration factor to be focused on in the maintenance and management of sewage pipes. The deterioration factors extracted in a data-driven manner using the XAI method in this study are expected to provide useful knowledge for the future maintenance and management of sewage pipes.

2. Summary of Previous Research and Positioning of the Present Study

In this chapter, we summarize the research conducted for the prediction of soundness, screening with respect to prioritization, and extraction of deterioration factors in sewage pipes; we conduct the research using XAI in an attempt to interpret classification models using machine learning. We then describe the novelty of the present study based on these aspects. In this section, the following two points are focused on, along with the purpose of this study, and the novelty of this study is described based on these two points.

• Research on the prediction of deterioration and factors contributing to deterioration in sewage pipes.
• Research examining the interpretation of machine learning models using Explainable AI.

2.1. Summary of Previous Research

There are existing attempts to predict the deterioration of sewage pipes using statistical and machine-learning methods. In addition, the factors related to the deterioration of sewage pipes have been examined by interpreting the constructed models.

As statistical methods, some studies have applied regression analysis [7,8] or logistic regression analysis [9,10,11,12,13]. As an example of applying regression analysis, Gedam et al. [8] constructed a regression model using pipe age, pipe diameter, pipe material, and pipe depth as explanatory variables. Also, the regression coefficients show that pipe age affects deterioration in a statistically significant manner. On the other hand, as an example of applying logistic regression analysis, Ana et al. [10] constructed a binary logistic regression model that classifies the condition of sewer pipes into two values (good or bad condition). From the constructed model, it has also been shown that pipe age, pipe material, and pipe length are important factors influencing deterioration. However, it has been mentioned that the deterioration of sewage pipes is a non-linear process, and that it is difficult for statistical models to predict this process with high accuracy [14]. On the other hand, machine learning models can establish the linear and non-linear relationships between input factors and the condition of sewage pipes, and machine learning approaches have also been addressed [15].

As an approach involving machine learning models, Sousa et al. [15] compared artificial neural networks and support vector machines (SVMs) with the performance of logistic regression and showed that the performance of ANNs was the best.

Harvey and Mcbean [16] compared the performance of decision trees and SVM and showed that decision trees perform better. In addition, they applied Random Forest to predict the state of the pipe (good or poor) and obtained good results [17].

Laakso et al. [18] built a model to predict the state (good or poor) of a pipe using Random Forest and evaluated the importance of variables using the Boruta algorithm.

Winkler et al. [19] used a method that extends the decision tree, the boosted decision tree, to perform binary classification of pipe states. The importance of the variables was assessed by calculating the feature importance of the constructed model.

Nguyen et al. [20] built a model to predict the pipe condition (Good Condition or Bad Condition) using 17 different machine learning methods (e.g., Random Forest, SVM, KNN) and compared the classification performance. Random Forest has the best classification performance, and the feature importance in Random Forest indicates that the pipe material is the most important factor, followed by the pipe age.

In September 2011, the Ministry of Land, Infrastructure, Transport and Tourism and the National Institute for Land and Infrastructure Management (NILIM) released the NILIM database (DB), consisting of sewage-pipe deterioration data, to support the introduction of asset management for sewage projects. Matsumiya et al. [21] quantitatively estimated the soundness prediction equation and made predictions regarding the amount of renovation work required using the NILIM DB. However, the proposed soundness prediction equation uses only the number of years since installation as an explanatory variable, and it cannot take into consideration differences in soundness based on pipe type, pipe diameter, number of attached pipes, etc. Moreover, although this DB is suitable for predicting the amount of renovation work required for an entire municipality, it is impossible to make detailed predictions such as which specific sewage pipe is damaged and to what extent.

Fujiu et al. [22,23,24] used the NILIM DB and converted qualitative and quantitative data that could be obtained from the DB into differential functions to prevent information reduction and apply linear discriminant analysis to this process. They were thus able to discriminate each sewage pipe span between Group 0—which is a set comprising pipes with a large degree of deterioration consisting of urgency levels I and II—and Group 1—which is a set comprising pipes with a small degree of deterioration with urgency levels III and IV.

Meanwhile, we previously used the NILIM DB to construct an urgency classification model using a one-dimensional convolutional neural network (1D-CNN), which is a type of deep learning method, wherein we concluded that improving the classification performance requires additional variables that can reproduce the environment in which the sewage pipes are buried [25].

This study aims to identify factors influencing the deterioration of sewage pipes by using Explainable AI (XAI) for machine learning models.

In a study attempting to interpret machine learning models using XAI, Xudong et al. [26] tested five algorithms, such as LightGBM, and an artificial neural network as machine learning methods using water network maintenance data, and they concluded that LightGBM exhibited the best prediction performance. Moreover, this classification model was interpreted using Shapley additive explanations (SHAP) to clarify that the socioeconomic factors of the local community influenced the water pipe damage.

Ito et al. [27] focused on the corrosion of communication pipelines and obtained the buried state of these pipelines from National Land Numerical Information, and after combining the data with inspection results, they built a binary classification model for determining the presence of corrosion in communication pipelines using XGBoost. In addition, they applied permutation importance to the constructed model and analyzed the importance of explanatory variables.

Tsukamoto et al. [28] built an evacuation selection behavior model for residents using a neural network and clarified the factors that impacted evacuation behavior selection and evacuation site selection by applying PI analysis and PD analysis in addition to XAI methods.

Koori et al. [29] used Random Forest to build a prediction model for landslide occurrence points and used SHAP, which is a type of XAI, to explain the prediction model globally and locally, thereby clarifying the basis of judgment of the prediction result.

Tatsuta et al. [30] used bridge chart information in an attempt to estimate the cause of damage and the repair method using a gradient boosting decision tree (GBDT). They also interpreted the trained model using SHAP to analyze the effects of the specifications on the cause of damage and the repair/reinforcement method, thereby demonstrating the validity of the model.

2.2. Positioning of This Study

There are two novelties in this study. First, multi-class classification models for sewage pipe soundness are built using machine learning methods.

Machine learning methods have been used previously to predict the deterioration of sewage pipes, but all of them were formulated as binary classification problems by combining the soundness ranks into binary groups. In the pipelines used in this study, there are four soundness ranks, with different measures required at each level of soundness. In this study, the process of combining soundness into binary groups is not conducted, with the aim of building a more practical model. There are no studies, to the best of the author’s knowledge, that use machine learning methods for multi-class classification that consider the buried condition of sewage pipes, as in this study.

Second, the Explainable AI method SHAP is used to interpret the classification model and to examine the deterioration factors of sewage pipes. As mentioned in the previous section, the deterioration process in sewage pipes is non-linear, and machine learning methods can be used to examine the deterioration process more accurately than statistical methods. Therefore, it is considered that interpreting the deterioration prediction models built using machine learning methods can provide appropriate knowledge on the deterioration factors. Previous studies have investigated deterioration factors by calculating feature importance in Random Forest. Feature importance makes it possible to identify the magnitude of the influence of the explanatory variables on the prediction. However, it cannot identify whether the explanatory variables have a positive or negative influence on the prediction. There are studies that use XAI methods, including SHAP, to interpret classification models, gain new knowledge, and examine the validity of the models. However, no studies have applied SHAP to sewage pipes and extracted the factors influencing the deterioration of sewage pipes, to the knowledge of the authors.

3. Used Data and Variables

3.1. Used Data and Soundness in Sewage Pipes

In this study, we used the results of sewage pipe inspections conducted in City A as training data to be used in the machine learning model. These inspection results comprised the soundness judgment made by an engineer based on the pipe soundness conditions that were confirmed via a video-camera survey. Moreover, positional information of each sewage pipe and the specification information of the pipeline (e.g., pipe age, pipe diameter, and pipeline gradient) were recorded, and the positional information was used to obtain information regarding the buried pipeline environment. The information regarding the environment in which the pipeline is placed was obtained using the recorded location information and was spatially connected with the environmental data that exist in the digital National Land Numerical Information [16] using a geographic information system. Moreover, as shown in

Table 1. Pipe soundness and diagnostic items in this study.

Diagnostic Items	Severe	Moderate	Mild	Sound
Pipe failure	A	B	C	D
Pipe cracks
Joint misalignment
Ingress water
Protrusion of the mounting pipes
Oil adhesion
Tree-root penetration
Mortar adhesion

, the soundness index of the pipelines in the present study was determined by categorizing the extent of failure in the pipeline among the four ranks of A (severe), B (moderate), C (mild), and D (sound). Among the diagnostic items in Table 1, the protrusion of the mounting pipes, oil adhesion, tree-root penetration, and mortar adhesion are items that were considered only in cases wherein they could not be removed via cleaning at the time of discovery. The machine learning model built in the present study classified target sewage pipes into four levels of soundness.

3.2. Variables Used

Table 2. Variables used in this study.

Category	Variable Name	Description	Source	Link
Specification information	Pipe age (year)	Pipe age at the time the inspection was conducted	Public sewerage system	―
	Pipe length (m)	Distance between upstream manhole and downstream manhole (1 span)		―
	Pipe diameter (mm)	Pipe inner diameter		―
	Number of pipes	Number of pipes constituting 1 span		―
	Pipeline gradient (‰)	Gradient of pipeline		―
	Pipe function	Combined flow main pipeline, Combined flow branch pipeline, Sewage flow main pipeline or Sewage flow branch pipeline		―
	Overburden (m)	Average of overburden upstream and downstream		―
	Average pipe bottom height (m)	Average pipe bottom height upstream and downstream (based on Tokyo Bay mean sea level)		―
Topographical information	Topographical classification	The property of the buried topography	Digital national land numerical information	https://nlftp.mlit.go.jp/kokjo/inspect/landclassification/land/chikei_bunrui.html (accessed on 20 June 2023)
	Population	Population at 500 m-mesh where the pipe is located (Estimated value in 2020)		https://nlftp.mlit.go.jp/ksj/gml/datalist/KsjTmpltmesh500h30.html (accessed on 20 June 2023)
	Annual precipitation (mm)	Annual precipitation at 1 km-mesh where the pipe is located		https://nlftp.mlit.go.jp/ksj/gml/datalist/KsjTmpltG02.html (accessed on 20 June 2023)
	Annual maximum snow depth (m)	Annual maximum snow depth at 1 km-mesh where the pipe is located
	Average slope angle (degree)	Average slope angle of 1 km-mesh where the pipe is located		https://nlftp.mlit.go.jp/ksj/jpgis/datalist/KsjTmplt-G04-a.html (accessed on 20 June 2023)
	Land-use zone classification	Land-use zone classification where the pipe is located (residential, industrial or commercial)		https://nlftp.mlit.go.jp/ksj/g ml/datalist/KsjTmpltA29.html (accessed on 20 June 2023)
Target variable	Soundness index	Rank A (severe), Rank B (moderate), Rank C (mild) or Rank D (sound)	The result of inspection	―

lists the variables used in the present study. In the present study, we considered not only the specification information of the pipeline (e.g., pipe age, pipe length, and pipeline gradient), but also the meteorological conditions, topographical information, and land use as explanatory variables in order to reproduce the buried pipeline conditions. The specification information included facility information such as pipe age, relationship, and pipe function.

In the case of the meteorological conditions, it was hypothesized that deterioration would occur depending on the amount of rainwater that flows into the pipeline due to rainfall and snow accumulation; thus, annual precipitation and the annual maximum snow depth in a 1 km mesh were obtained from the National Land Numerical Information [31]. In addition, as the pipeline was buried, it was determined that the influence of the external air temperature was small, and the amount of solar radiation and temperature were not considered variables in the present study.

Next, for the topographical information, we used topographical classification as the property of the buried topography to express the buried pipeline conditions. Furthermore, the amount of rainwater inflow was thought to differ depending on the location.

Not all local governments have access to the information used as explanatory variables in this study. Therefore, it is possible to obtain useful information for many local governments by clarifying the effects of a larger number of variables on the soundness of sewage pipes. Therefore, in this study, explanatory variable selection was not conducted, and all variables were used to build the model.

3.3. Correlation Analysis of Used Data

In this chapter, we examine the factors that impact the soundness of sewage pipes by analyzing the correlations between the variables in the used data. The dataset used includes quantitative variables such as the pipe age and length, as well as qualitative variables such as pipe soundness and land use. We thus used different correlation indices based on the combination of variable types. Specifically, we used Pearson’s correlation coefficient for the correlations between two quantitative variables, and Cramer’s contingency coefficient for the correlation between two qualitative variables. Furthermore, a correlation ratio was used for the correlation between the qualitative and quantitative variables. Figure 5 presents the correlation matrix calculated using the above procedure. Herein, we focus on the correlation between soundness, which is the objective variable in this study, and the explanatory variables. The correlation with each explanatory variable was calculated using the correlation ratio or Cramer’s contingency coefficient described above, while considering value in the range of 0–1, wherein a correlation closer to 1 can be interpreted as a stronger correlation. As shown in Figure 5, the sewage pipe function has the highest value as an explanatory variable that impacts soundness. However, no high correlation was found with any of the explanatory variables, which suggests that the deterioration of the sewage pipes cannot be explained using a single factor and that this is a phenomenon that occurs owing to complex factors.

3.4. Shaping of Imbalanced Data

Figure 6 presents the breakdown of the data used in this study. A very small amount of the data used in this study belonged to Rank A, with low soundness, and there is a large bias in the amount of data. When building a machine learning model, the use of imbalanced data in the training process can increase the apparent accuracy of the classification into classes with a large amount of data, and there exists the concern that the training should be optimized and its accuracy should be improved for classes with a large amount of data instead of learning important features in the classification. Therefore, in this study, we applied down-sampling and over-sampling to the imbalanced data to create a balanced dataset. First, for down-sampling, the amount of data belonging to Rank A, which is the class comprising a smaller amount of data, was adjusted to the amount of data of the other classes. Specifically, random sampling was conducted for the amount of data belonging to Rank A (115 cases) from classes other than Rank A. At this time, there may be biases in the dataset depending on the sampling method. Therefore, in this study, the model building and evaluation processes were conducted based on a dataset generated by means of multiple random sampling. The details of the model building and evaluation processes are described below. In this study, random sampling was conducted 100 times to create 100 balanced datasets with the same amount of data in each class.

Next, for over-sampling, we applied the synthetic minority over-sampling technique (SMOTE), which was proposed by Chawla et al. [32]. SMOTE is an algorithm based on the k-nearest neighbor method. Given an arbitrary positive example, we used the similarity calculation, as shown in Equation (1), to specify other positive examples that exist in the nearest vicinity. Then, a new positive example is created along a straight line connecting the two points. This process is repeated a specified number of times to increase the number of positive examples and reduce the bias between the data.

$$sim\left(m_a,m_i\right)=\sqrt{{\sum }_{j=1}^n({v_{a,j}-v_{i,j})}^2}$$

(1)

where $sim\left(m_a,m_i\right)$ is the similarity between an arbitrary point $m_a$ and its neighboring point $m_i$, $v$ is the explanatory variable at the point and $n$ is the number of explanatory variables.

4. Construction of Classification Model

In this study, we built a soundness classification model for sewage pipes using a supervised machine-learning method. In recent years, deep learning methods have been applied with great success in various domains, such as images, audio, and text [33,34,35]. On the other hand, deep learning models suitable for tabular datasets, such as TabNet [36], have also been developed. However, it has been reported that the performance of tree-based models outperforms deep learning methods on tabular datasets [37,38,39]. Therefore, in this study, XGBoost [40], a type of gradient boosting decision tree (GBDT) based on decision trees, was selected as the machine learning model.

XGBoost is a type of ensemble learning. Ensemble learning is a method of constructing a more accurate model by combining multiple weak learners and using representative methods including bagging and boosting. The XGBoost used in this research is a method that combines a decision tree and boosting, and it is characterized by the method’s focus on the training data that were not correctly classified in the $t-1$th prediction result during the $t$th training round. As previously mentioned, XGBoost uses decision trees as weak learners, but hyperparameters such as the number (n_estimators) and depth (max_depth) of decision trees, which are weak learners, are required to be set in an exploratory manner. In this study, we considered the number of decision trees (n_estimators) and the maximum depth of decision trees (max_depth) as hyperparameters. Furthermore, we adjusted the hyperparameters by examining the n_estimators and max_depth that were set in a brute-force manner, and we adopted a grid search to determine the parameters with the model that provided the highest prediction accuracy. The hyperparameters were the 11 n_estimators patterns with n_estimators = 10, 20, 30, 50, 60, 70, 80, 100, 200, 300, and 400 and nine max_depth patterns of max_depth = 2, 3, 4, 5, 6, 7, 8, 9, and 10. For the datasets described in the previous section, we adjusted the hyperparameters using one of the 100 balanced datasets in case 1 and the balanced dataset generated using SMOTE for case 2. We also used scikit-learn [41], which is a machine-learning library, for the hyperparameter search and model construction.

The final values of the hyperparameters determined by means of the grid search are shown in

Table 3. The final values of the hyperparameters.

	n_estimator	max_depth
model 1	10	3
model 2	400	10

.

5. Evaluation of the Classification Model

In this chapter, we describe the evaluation method and the results of two classification models (models 1 and 2) constructed using the two types of balanced dataset (cases 1 and 2) described in the previous section.

5.1. Evaluation Method for Model 1

Figure 7 presents an overview of the evaluation method used for model 1. The evaluation of model 1 was conducted by applying the k-fold cross-validation method to the 100 balanced datasets described in Section 3. The balanced datasets were created by performing random sampling from the original dataset (Figure 6) so that the amount of data in each class was the same. We created 100 balanced datasets by conducting 100 epochs of random sampling. In this case, random sampling was applied, so the pipes that comprise each dataset (No. 1–100) are different. As shown in Figure 7, the k-fold cross-validation method involved dividing the created dataset into k groups; furthermore, within the divided dataset, k-1 groups were used for training, and the remaining group was used for the validation of the model. This procedure was repeated k times, and the average value of the evaluation values in each iteration was used as the evaluation value of the model. In this study, the number of divisions was set as k = 5. In this study, k-fold cross-validation was applied for each of the 100 datasets, such that the same number of evaluation values calculated using k-fold cross-validation existed as the number of datasets. In this study, we calculated the average value of these evaluation values and used it as the evaluation value of the final classification model.

5.2. Evaluation Method for Model 2

For the evaluation of model 2, prior to the data augmentation with SMOTE, we used 30 spans for each class for which prior sampling was conducted, for a total of 120 spans, as the test data. These test data were predicted using model 2, which was trained with a dataset that was over-sampled using SMOTE, and a comparison between the predicted and correct values was performed to calculate the performance index, which was then used as the evaluation value of the classification model.

5.3. Performance Index of the Classification Model

In the estimation of soundness in sewage pipes, it is not desirable to make predictions on the dangerous side, such that pipes with low soundness are incorrectly assessed as sound. Therefore, it is necessary to assess whether the built classification model has the classification characteristics. In this study, we used precision, a measure of over-detection, recall, a measure of misses, and F-measure, a measure of the balance between the two, as indices of classification performance in order to evaluate the performance of the model, including its classification characteristics. Also, macro-average recall, macro-average precision, and macro-average F-measure, which are the averages of the performance indices in each class, were used as macro-performance indices of the model. The recall, Precision, and F-measure of each class were calculated using Equations (2)–(4) using the values defined in the confusion matrix presented in

Table 4. Confusion matrix.

	Predicted Positive	Predicted Negative
Actually Positive	TP	FN
Actually Negative	FP	TN

. The recall is an index that indicates how many times the classification model actually performed a correct classification ($TP+FN$) out of the total amount of data for the target class ($TP$). The precision is the index that indicates how many times the classification model performed a correct classification ($TP+FP$) out of the total amount of data that the classification model classified into the target class ($TP$). As recall and precision generally have a trade-off relationship, the F-measure is the value obtained by taking the harmonic average of both indices and is an index that indicates the balance between precision and recall.

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(2)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(3)

$$\mathrm{F}-\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}=\frac{2·\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}·\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(4)

5.4. Classification Performance of Classification Model

Table 5. Classification performance index in model 1.

	Precision	Recall	F-Measure
RankA	0.57	0.59	0.57
RankB	0.34	0.33	0.33
RankC	0.32	0.33	0.32
RankD	0.36	0.40	0.39
Average	0.40	0.40	0.39

and

Table 6. Classification performance index in model 2.

	Precision	Recall	F-Measure
RankA	0.67	0.20	0.31
RankB	0.67	0.40	0.50
RankC	0.27	0.15	0.10
RankD	0.49	0.90	0.51
Average	0.49	0.41	0.38

list the results of the classification performance of models 1 and 2, respectively. Figure 4 presents the results of the k-fold cross-validation. A comparison of the macro-average recall, precision, and F-measure of each model showed that the overall classification performance of the model was higher in the case of model 2, in which a dataset that was augmented using SMOTE was used. However, in terms of which soundness rank each model was likely to be classified as, model 1 exhibited high precision and recall values of 0.57 and 0.59, respectively, in Rank A. This indicates that, in terms of precision, approximately 57% of the pipelines that were predicted by the classification model to belong to Rank A were actually Rank A, and in terms of the recall, approximately 59% of the pipes that actually had a soundness of Rank A were correctly classified by the classification model as Rank A. Therefore, it can be stated that this is a classification model that can make predictions on the safe side because it can classify pipelines of Rank A with a low soundness with a relatively high accuracy. Meanwhile, although model 2 had a higher overall classification performance than model 1, model 2 had a low Rank A recall of 0.2, and it could be considered as a model that classifies pipes on the dangerous side and that overlooks many pipes with low soundness. Moreover, when confirming the classification tendencies in model 2, there was a high recall of Rank D, which had a large amount of data before the application of SMOTE, and a low recall of Rank A, which had the lowest amount of data. Therefore, it is observed that the number of data augmentations using SMOTE was small and there was a tendency for easier classification into classes with abundant data.

6. Analysis of Factors Impacting Deterioration of Sewage Pipes Using XAI

In this chapter, we clarify the basis for the prediction results of the classification model that was built in the previous section to discuss what types of sewage pipes have low soundness. The classification model for interpretation was set as model 1, which was able to relatively accurately classify Rank A, which indicates low soundness. We then applied XAI to clarify the characteristics of the sewage pipes that were classified as Rank A.

6.1. Overview of XAI and SHAP

AI used for deep learning and machine learning is generally referred to as a black box, and it becomes difficult to interpret “why the classification model outputs a given prediction result” proportionally to the classification performance. However, AI has recently been used in important decision-making situations related to human life and safety in the medical field. Given this background, some research has been conducted on XAI with the aim of eliminating the opacity of AI. XAI can be broadly classified into two types: global explanations and local explanations. The former global explanation is a method of clarifying which explanatory variables strongly impact the prediction of the entire model. Meanwhile, the local explanation provides an explanation of the individual prediction results in the classification model. In this study, we adopted the XAI method SHAP [19], which can provide both global and local explanations, to analyze the factors that impact the deterioration of sewage pipes from the perspective of why the classification model classifies sewage pipes as having low soundness (Rank A).

In the SHAP method, a complex model is locally approximated into a simple model, and a SHAP value that indicates the degree of contribution of each explanatory variable to the predicted value in the method is calculated, which is similar to the calculation of the Shapley value in game theory. The fact that the importance of explanatory variables with many continuous variables and category variables tends to increase with respect to the importance of the output by GBDT models has been demonstrated in previous research [42], and there is a possibility that the importance of explanatory variables that are originally important cannot be appropriately evaluated. Meanwhile, SHAP is less susceptible to the above impacts and can evaluate the importance of explanatory variables in a more appropriate manner [43].

6.2. Interpretation of Classification Model Based on SHAP Value

In this study, we analyzed the characteristics of pipes with extremely low soundness by calculating the SHAP value for Rank A in model 1. First, to calculate the SHAP value, we used the original dataset (Figure 6) and randomly sampled 115 samples from each class in order to create a dataset. This dataset was used to calculate the SHAP values for Rank A in model 1 for all the explanatory variables. Figure 8 presents a beeswarm plot of the distribution of the calculated SHAP values. The beeswarm plot is a one-dimensional scatterplot for each variable, with data that have the same value being plotted perpendicular to the axis, such that the data do not overlap. In the beeswarm plot, the horizontal axis indicates the SHAP value, with increased contributions from those that are judged as Rank A as the value increases to a positive value, and conversely, increased contributions from those that are not judged as Rank A but as other classes as the value decreases to a negative value. The explanatory variables are arranged in descending order of the average absolute value of the SHAP values, and they express the degree of influence on the classification results of the classification model. Moreover, the plot colors indicate higher values as they become red, and smaller values as they become blue. According to the beeswarm plot, in the classification of Rank A, the important characteristics are, in descending order, the number of pipes, pipe age, average pipe bottom height, pipe length, and overburden thickness. In addition, from the explanatory variable values and SHAP values, when focusing on the number of pipes, it could be observed that plots with a small value for the number of pipes were distributed in the direction of high SHAP values, and this could be interpreted as indicating that pipe networks with a small number of pipes tended to be classified as Rank A. Similarly, it could be confirmed from the pipe age that old pipes with a high pipe age tended to be classified as Rank A.

In the next section, we analyze the relationship between each explanatory variable and the SHAP value in more detail for the quantitative variables among the explanatory variables. Herein, when we focus on qualitative variables among the explanatory variables, in residential land-use areas (considered as a dummy variable, e.g., 0: not applicable, and 1: applicable), plots with 0 values were distributed in the direction of higher SHAP values, and it can thus be confirmed that there was a tendency for the pipes belonging to residential land-use areas to not be classified as Rank A. Meanwhile, pipes belonging to industrial land-use areas tended to be classified as Rank A. Therefore, we were able to use the beeswarm plots of SHAP values with respect to Rank A to broadly determine the magnitude relationship of each explanatory variable value and their degree of contribution to the classification for Rank A.

6.3. Detailed Analysis of SHAP Values and Quantitative Explanatory Variables

In this section, we analyze the relationship between the value of each explanatory variable and the SHAP value for quantitative variables among the explanatory variables in order to discuss in more detail whether the degree of contribution of the classification to Rank A changes depending on the value of the explanatory variable. In this study, the following 11 variables were used as quantitative explanatory variables: number of pipes, pipe age, average pipe bottom height, pipe length, pipeline gradient, overburden thickness, population, annual precipitation, average slope angle, annual maximum snow depth, and pipe diameter. Figure 9 presents a scatter plot with each explanatory variable along the horizontal axis and the SHAP value for Rank A along the vertical axis.

In terms of the number of pipes (upper left of Figure 9), it can be confirmed that there exists an overall trend of networks with a lower number of pipes having higher SHAP values and pipe networks with a lower number of pipes tending to be classified as Rank A. Moreover, the SHAP value was particularly high for pipe networks with a number of pipes of approximately five or fewer, which suggests that the soundness of pipe networks with fewer than five pipes tended to decrease.

In terms of the pipe age (middle of the first row in Figure 9), it can be confirmed that there is an overall trend that pipes with an older pipe age tended to have higher SHAP values and were classified as Rank A. Structures generally deteriorate over time, and this was thus determined to be a valid interpretation.

In terms of the average pipe bottom height (upper right of Figure 9), it can be confirmed as an overall trend that pipe networks with a low average pipe bottom height tended to have higher SHAP values and were classified as Rank A.

In terms of the pipe length (left end of the second row in Figure 9), it can be confirmed that pipe networks with a larger pipe diameter tended to have lower SHAP values, and pipe networks with smaller pipe diameters tended to be classified as Rank A.

In terms of the pipeline gradient (middle of the second row in Figure 9), the pipeline gradient values tended to have a low variation, while the SHAP values tended to have a high variation, and it was thought that the relationship between the magnitude of the pipeline gradient and the likelihood of classification as Rank A was weak.

In terms of the overburden thickness (right end of the second row of Figure 9), pipes with a large overburden thickness tended to have higher SHAP values and were more easily classified as Rank A.

In terms of population (left end of the third row of Figure 9), pipes with a large surrounding population tended to have low SHAP values and were more easily classified as Rank A.

In terms of the annual precipitation (middle of the third row of Figure 9), pipes with low annual precipitation tended to have high SHAP values and were more easily classified as Rank A.

In terms of the average slope angle (right end of the third row in Figure 9), it can be confirmed as an overall trend that pipes with a smaller average slope angle tended to have higher SHAP values and were more likely to be classified as Rank A. However, despite the fact that plots of pipes with an average slope angle of 5° or less were concentrated, there was high variation in the SHAP values, and it is thought that the relationship between the magnitude of the average slope angle and the degree of contribution to the classification as Rank A is small.

In terms of the annual maximum snow depth and pipe diameter (fourth row of Figure 9), there was little variation in the SHAP values, and it is thought that there was little contribution from the annual maximum snow depth and pipe diameter to the pipe’s classification into Rank A.

7. Discussion

In this chapter, the deterioration factors in sewage pipes identified in this study are compared with previous studies. As mentioned above, it is not clear whether the factors have a positive or negative influence on degradation using machine learning methods. Therefore, a comparison is made here with studies that use statistical models.

First, pipe age has been reported to be important regarding the deterioration of sewage pipes in many previous studies [6,8,9,17,18,19,44]. In this study, it was also clarified that the higher the age of the pipe, the higher the contribution to Rank A classification.

There is a strong correlation between the number of pipes and pipe length (Figure 5), and in this study, the smaller the respective values, the higher the contribution to Rank A classification. On the other hand, Ana et al. [10] mentioned that pipelines with longer pipe lengths have a higher risk of deterioration, which deviates from the present study. This is because the soundness index used in this study was calculated on the basis of the number of defective pipes out of all the pipes in the pipeline.

Regarding overburden thickness, Khan et al. [45] showed that an increase in depth in a pipe has a negative effect on the condition level of the sewage pipes. In this study, it was found that the higher the overburden thickness, the higher the contribution to the Rank A classification. This agrees with previous studies [45]. This may be due to the fact that the greater the soil cover thickness, the greater the deadload on the pipe and the higher the probability of meeting groundwater. In this study, the groundwater level has not been directly considered. However, when focusing on the average pipe bottom height and average slope angle, the average slope angle was small, and the pipes located at lower elevations had a higher contribution to the Rank A classification. It is considered that groundwater flows into areas with small average slope angles and low elevations, and that these areas are likely to have high groundwater levels. The above results suggest that the average pipe bottom height and the average slope angle in the pipeline represent the groundwater level.

Regarding pipe gradients, Ana et al. [10] showed that the risk of deterioration in pipelines with flatter gradients is higher. On the other hand, Salman and Salem [46] and Jeong et al. [47] showed that the probability of deterioration is higher in pipelines with steeper pipeline gradients. In this study, there was no significant relationship between SHAP values and the pipeline gradient, and the effect of pipeline gradient on the deterioration of sewage pipes could not be determined.

Regarding the pipe diameter, Ariaratnam et al. [9] and Jeong et al. [47] mentioned that the risk of deterioration is higher for pipelines with larger diameters. On the other hand, Micevski et al. [48] and Tran et al. [49] mentioned that the rate of deterioration is lower for pipelines with larger diameters. In this study, there was no clear relationship between SHAP values and pipe diameter, and the effect of pipe diameter on the deterioration of sewage pipes could not be determined.

Regarding the function of the pipeline, Salman and Salem [46] maintained that combined-flow sewers are more vulnerable to deterioration. On the other hand, in this study, it was found that the contribution to Rank A classification did not vary significantly with pipe function.

No studies were found that considered the same variables for topographical classification. In this study, the contribution of topographical classification to the classification of Rank A was small. On the other hand, Laakso et al. [18] considered geological information in their prediction model; it was reported that geological information was not a significant variable in the prediction model developed by Laakso et al.

In addition, Laakso et al. [18] reported that it is effective to consider spatial coordinates in the analysis. They stated that this is due to differences in the quality of installation works, land use, and the number of consumers in the water supply area at the location of the pipeline. In this study, the land-use zone classification and the population around the pipelines were used in the analysis. The results show that with regard to the land-use zone classification, pipelines belonging to industrial use areas tend to contribute more to the Rank A classification. On the other hand, for the surrounding population, it was found that pipelines with a small surrounding population had a higher contribution to the Rank A classification.

No studies were found that considered annual precipitation and annual maximum snow depth as variables in the same way. In the present study, it was found that pipelines located in areas with high annual precipitation tended to have a higher contribution to the Rank A classification.

In conclusion, some of the deterioration factors obtained in this study are in agreement with previous studies, while others are not. This may suggest that the deterioration factors differ from region to region. This suggests that it is recommended to construct different prediction models for different regions and that the deterioration factors should also be assessed for each region.

8. Conclusions and Future Issues

8.1. Conclusions

In this study, a machine learning-based soundness classification model for sewage pipes was developed using the results of inspections as an approach for sustainable operations of sewage systems. Down-sampling and oversampling were conducted to create a balanced dataset because the datasets were imbalanced. Then, two classification models (model 1, model 2) were built using XGBoost with a balanced dataset. As a result, model 1 had relatively good classification performance in Rank A (precision: 0.57, recall: 0.59). This indicates that fewer low-soundness pipes are missed, and that safe side predictions can be made. In this study, SHAP was applied to model 1 to analyze the characteristics of sewage pipes that are classified as Rank A. By applying SHAP, it was possible to quantify the effect of deterioration factors on the prediction of the classification model and to obtain knowledge about deterioration factors in sewage pipes. The factor of deterioration of sewage pipes obtained in this study is expected to provide useful information for the extraction of priority areas for inspection, and this is important knowledge for the sustainable operations of sewage systems.

8.2. Future Issues

A future issue related to this study is the improvement of the classification performance of the classification model. In this study, we attempted to reproduce the conditions in which the sewage pipes are placed by considering the information on the specifications and surrounding environment of the sewage pipes. However, only precipitation was considered as an external force in this study, and traffic load was not taken into consideration. It is thought that the impact of traffic load could be taken into consideration using ETC2.0 probe data in future studies. Moreover, the data of the surrounding environment used in this study were obtained from the National Land Numerical Information data and were aggregated in mesh units. It is also hypothesized that the use of disaggregated data, such as building polygon data [50], could improve the reproducibility of the buried sewage pipe conditions.

Furthermore, the technique to detect ground surface displacement from SAR images acquired by satellite has been developed in recent years [51,52]. Ground surface displacement may cause damage to sewage pipes. Adding the ground surface displacement acquired by SAR images as an explanatory variable is expected to improve the classification performance of the model.

In this study, a classification model was built using inspection data obtained by one local government. Therefore, the model built in this study may not be widely applicable to other local governments. In addition, the deterioration factors of sewage pipes extracted using SHAP may also be limited to City A. In the future, it will be necessary to build a model with a more generalized performance and extract more generalized deterioration factors by verifying the data of other municipalities.

In this study, one of the objectives was to identify factors of deterioration in sewage pipes, so the model was built using all variables without any variable selection. However, the many explanatory variables required for the input of the model make it impossible to use the model in the absence of data, reducing the applicability of the model. In the future, we aim to build a model with the same or better classification performance as the model built in this study with a smaller number of variables by selecting variables.