Novel Approach to Predicting Soil Permeability Coefficient Using Gaussian Process Regression

Abstract

In the design stage of construction projects, determining the soil permeability coefficient is one of the most important steps in assessing groundwater, infiltration, runoff, and drainage. In this study, various kernel-function-based Gaussian process regression models were developed to estimate the soil permeability coefficient, based on six input parameters such as liquid limit, plastic limit, clay content, void ratio, natural water content, and specific density. In this study, a total of 84 soil samples data reported in the literature from the detailed design-stage investigations of the Da Nang–Quang Ngai national road project in Vietnam were used for developing and validating the models. The models’ performance was evaluated and compared using statistical error indicators such as root mean square error and mean absolute error, as well as the determination coefficient and correlation coefficient. The analysis of performance measures demonstrates that the Gaussian process regression model based on Pearson universal kernel achieved comparatively better and reliable results and, thus, should be encouraged in further research.

1. Introduction

One of the most essential variables governing soil’s fluid-flow characteristics is its permeability. The importance of determining the soil permeability coefficient is widely acknowledged, and is affected by a variety of parameters, including mineralogy, soil density, soil structures, water content, void ratio, and others [1]. Ganjidoost et al. [2] reported that three category factors remarkably affect the soil permeability coefficient, namely, permeable soil parameters (density, clay content, viscosity etc.), inherent soil parameters (Atterberg limits, particle size distribution, etc.), and compacted soil factors (porosity, water content, density, etc.). Most of these factors are closely related to each other. It was reported that the soil permeability coefficient was decreased by over 100 times when the percentage passing through sieve No. 100 increased by in the range of 0 to 7% [3]. Conducting several experiments with the difference in percentages of granular and low-plastic marine soils, Shakoor and Cook [4] concluded that the soil permeability coefficient noticeably increased by increasing the percentage of granular material. The coefficient of soil permeability is used to solve a variety of geotechnical issues, including slope stability, and structure collapse due to ground settlement, seepage, and leakage. As a result, several authors have attempted to establish empirical correlations between affecting factors and permeability coefficients [5,6].

Field or laboratory tests can be used to determine the soil permeability coefficient. It is shown that determining the soil permeability coefficient in the field is expensive, complicated, time-consuming, and tedious [7,8,9]. However, obtaining undisturbed samples for laboratory measurements of soil permeability coefficient is problematic. In particular, laboratory samples are frequently reconstituted to match those collected in the field. As a result of the devastation of soil fabric during sampling, laboratory test findings may not reflect the true value of soil permeability in the field [10]. Due to the specific advantages and disadvantages of each test, the soil permeability coefficient is calculated using a combination of field and laboratory data [7,10]. To assess soil permeability, several researchers proposed a regression that takes into account porosity, clay percentage, and sand particle size [11]. Several other researchers calculated soil permeability based on particle shape, grain size, and bulk density [12,13]. As previously stated, soil permeability is greatly influenced by particle size distribution; nevertheless, this is not true for all soils [9,14]. These empirical relationships include limitations and uncertainties, according to Pham et al. [1] study.

Machine-learning (ML) algorithms have recently been successful in solving real-world issues in a variety of fields, including civil and environmental engineering [15], and geotechnical engineering [16,17,18,19,20,21]. Several studies have used ML methods to predict the soil permeability coefficient, such as the adaptive neuro-fuzzy system (ANFIS), artificial neural network (ANN), and hybrid optimization model of genetic algorithm-ANFIS (GA-ANFIS) [2,9,22,23]. Sezer et al. [24] used an ANFIS to estimate granular soil permeability and found that the ANFIS algorithm is effective at estimating granular soil permeability when grain size distribution and particle shape are taken into account [22]. In comparison to single ANN, ANFIS model, and the hybrid GAANN model, the hybrid model GAANFIs outperformed in terms of prediction accuracy [2]. Soft computing-based models, in general, are excellent techniques for predicting soil parameters; for instance, random forest (RF) has been effectively used to predict soil properties including shear strength and permeability coefficient [25,26]. In geotechnical research, the permeability coefficient (k) of soil is an important component for designing civil-engineering structures on soil. Correlating other soil engineering parameters using an empirical equation to estimate “k” may not be correct [5,6,27]. Therefore, the aims of this study are (1) to develop new improved prediction models based on the Gaussian process regression (GPR) on Da Nang–Quang Ngai expressway development-project-site soil by using six soil parameters, such as liquid limit LL (%), plastic limit PL (%), clay content CC (%), void ratio e, natural water content w (%), and specific density γ (g/cm³) as inputs; (2) to divide data into training and testing datasets with due attention to statistical aspects such as the minimum, maximum, mean and standard deviation of the datasets. The splitting of the datasets is performed to find out the predictive ability and generalization performance of developed models and later helps in better evaluating them; (3) to compare the proposed models to the reference models used in the published literature; and (4) to investigate the importance and impact of each input parameter on the soil permeability coefficient.

2. Methodology

2.1. Data Catalog

The dataset comprises of 84 soil samples obtained from detailed design state investigations of the Da Nang-Quang Ngai expressway development project near Da Nang, central Vietnam (Figure 1) and is reported in the research work of Pham et al. [28] (see Appendix A for complete dataset). Further details about the collection, testing and type of soils can be found in the Pham et al. [28] reference. Previous studies show that the coefficient of soil permeability is a function of the liquid limit LL (%), plastic limit PL (%), clay content CC (%), void ratio e, natural water content w (%), and specific density γ (g/cm³) [1,28]. It has been widely accepted, among researchers, that the input factors selected by Pham et al. [1,28] constitute a complete and suitable set to estimate “k”. As a result, these input variables were used to create the GPR model in the current study. The same input parameters related to permeability were used to estimate the “k” (×10⁻⁹ cm/s) of soil. Researchers have used a different percentage of the available data as the training and testing sets for different problems. For instance, Pham et al. [29] used 60%; Liang et al. [30] used 70%; while Ahmad et al. [31] used 80% of the data for training. In this study, the data set was divided into training (70%) and testing (30%) based on statistically consistency. The statistical consistency of training and testing datasets was based on statistically consistency. The statistical consistency of training and testing datasets has a substantial impact on the results when using soft computing techniques, which improves the performance of the model and helps in evaluating them better. Figure 2 depicts the cumulative percentage and frequency distributions for all of the input and output parameters of the mentioned database utilized in the modeling of soil permeability coefficient. The data points of every input parameter are distributed over its range. The statistical analysis, i.e., minimum (Min), maximum (Max), mean, and standard deviation (Std. Dev) of the training and testing datasets is presented in

Table 1. Statistical analysis of the study’s inputs and output.

Dataset	Parameters	Clay Content, CC (%)	Water Content, w (%)	Liquid Limit, LL	Plastic Limit, PL	Specific Density, γ (g/cm3)	Void Ratio, e	Permeability Coefficient, k (×10−9 cm/s)
Training	Min	5.7	16.97	19.5	12.2	2.58	0.486	0.003
	Mean	28.056	37.82	40.219	23.882	2.6715	1.0576	0.016
	Max	64	99.9	88.93	54.8	2.74	2.634	0.071
	Std. Dev	19.761	28.62	22.228	12.347	0.0413	0.7234	0.016
Testing	Min	6.7	15.09	18.9	12.5	2.63	0.462	0.004
	Mean	18.36	25.75	30.304	18.279	2.6836	0.7553	0.012
	Max	45.8	89.51	85.86	42.7	2.73	2.372	0.051
	Std. Dev	13.337	19.13	16.272	7.3879	0.0256	0.4856	0.012

.

2.2. Gaussian Process Regression

Gaussian process regression (GPR) is a probabilistic, non-parametric supervised learning method for generalizing nonlinear and complicated function mapping hidden in data sets. The GPR model is based on Rasmussen and Williams’ [32] assumption that adjacent observations should communicate information about each other; it is a means of describing a prior directly over function space. The mean and covariance of a Gaussian distribution are vectors and matrices, respectively, whereas the Gaussian process is an over function. The GPR model can recognize a prediction distribution that is similar to the test input. A GPR is a set of random variables with a joint multivariate Gaussian distribution for any finite number. Let M × N denote the input and output domains, respectively, from which n pairings (M_i, N_i) are distributed independently and identically. For regression, let N⊆ ℜ; then, a GPR on 𝜒 is defined by a mean function μ: 𝜒→ℜ and a covariance function k:𝜒 × 𝜒→ℜ. Kuss [33] is recommended for more information on GPR and other covariance functions.

Details of Kernel Functions

The kernel function is used in the GPR design process. In the literature, several kernels have been discussed [34,35,36]. The following three kernel functions are used in this study:

• Polynomial (Poly)

$$K\left(M,N\right)={\left(1+\left(M,N\right)\right)}^d$$

(1)

2.

Radial basis function (RBF)

$$K\left(M,N\right)=e^{-\lambda \left|M-N\right|}{}^{{}^2}$$

(2)

3.

Pearson universal kernel (PUK)

$$K\left(M,N\right)=\left(1/{\left[1+{\left(2{\sqrt{\Vert M-N\Vert }}^2\sqrt{2^{\left(\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\omega $}\right.\right)}-1}/\sigma \right)}^2\right]}^{\omega }\right)$$

(3)

The kernel width ($\lambda$) in RBF kernel, and parameters, σ (controls Pearson width) and ω (tailing factor of the peak) in PUK need to be established based on the precision in prediction.

2.3. Performance Metrics and Evaluation

To examine the performance of GPR modeling, the coefficient of determination (R²), correlation coefficient (R), mean absolute error (MAE), and root mean square error (RMSE) were utilized. The following formula can be used to compute these parameters:

$$R=\frac{{\displaystyle {\sum }_{i=1}^n\left[\left(y_{i_o}-{\overline{y}}_p\right)\left(y_{i_o}-{\overline{y}}_p\right)\right]}}{\sqrt{{\displaystyle {\sum }_{i=1}^n{\left(y_{i_o}-{\overline{y}}_p\right)}^2}}\sqrt{{\displaystyle {\sum }_{i=1}^n{\left(y_{i_o}-{\overline{y}}_p\right)}^2}}}$$

(4)

$$R^2=1-\frac{{\displaystyle {\sum }_{i=1}^n{\left(y_{i_p}-y_{i_o}\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(y_{i_o}-{\overline{y}}_o\right)}^2}}$$

(5)

$$MAE=\frac{1}{N}{\displaystyle {\sum }_{i=1}^n\left|y_{i_o}-y_{i_p}\right|}$$

(6)

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle {\sum }_{i=1}^n{\left(y_{i_o}-y_{i_p}\right)}^2}}$$

(7)

where $y_{i_o}$ and $y_{i_p}$ represent the actual and estimated coefficient of soil permeability values, respectively; ${\overline{y}}_o$ is the average of the reference samples’ values; and $n$ is the defined number of data points.

The R² and R are used to express the degree of collinearity between estimated and actual data. The correlation coefficient, which ranges from 1 to −1, indicates how closely actual and estimated data are related. If R is equal to 0, there is no linear relationship. If R = 1 or −1, there is a perfect positive or negative linear relationship. R² indicates how much percentage of variance in estimated data the model can explain. R² is a number that ranges from 0 to 1, with higher values indicating less error variation and values over 0.5 considered acceptable [37,38]. The MAE indicates the mean of the estimated and actual values. The adjustment has a better effect when the MAE is close to 0, meaning that the prediction model more accurately describes the set of training data [39]. The RMSE is the average magnitudes of the errors in predictions for all observations in a single measure of predictive power. The RMSE is larger than or equal to 0, with 0 signifying that the observed data is statistically perfectly fit. As a result, the lesser the values of MAE and RMSE criteria are, the better the model. Visual representations such as scatter plots were also employed to compare the performance of the established models. The flowchart of the methodology of the present study is shown in Figure 3.

3. Results and Discussion

To estimate the soil permeability coefficient, GPR models must be analyzed once they have been developed. The outcomes of the evaluation show if the models have practical value, that is, whether they can accurately estimate the soil permeability coefficient. As previously stated, 70% and 30% of total dataset records were used as training and testing sets, respectively, for modeling using the GRP approach.

The Waikato Environment for Knowledge Analysis (WEKA) software was used to implement a number of kernel-function-based Gaussian process regressions in this paper. The WEKA is a collection of machine-learning algorithms for data-mining jobs that is available as open-source software. Hyperparameters must be adjusted in most machine-learning algorithms.

Table 2. The optimal tuning parameters for various regression models.

Model	Optimal Tuning Parameters
PUK kernel	{noise = 0.6, ω = 0.1, σ = 0.1}
Poly kernel	{noise = 0.02}
RBF kernel	$\{\mathrm{noise}=0.04,\lambda$ = 0.6}

depicts how the GPR-RBF, GPR-Poly, and GPR-PUK models’ essential hyperparameters were adjusted in this study. First, the models’ tuning parameters were set, and then the trials were repeated until the best fitness measures in Table 2 were obtained.

Table 3. Comparative performance of the GRP method and previously existing models.

Model	Training				Testing				Reference
	R	R2	MAE	RMSE	R	R2	MAE	RMSE
RF	0.972	-	0.0023	0.0035	0.851	-	0.0049	0.0084	[1]
ANN	0.948	-	0.0027	0.0047	0.845	-	0.005	0.001
SVM	0.861	-	0.0056	0.0078	0.844	-	0.0064	0.0098
M5P	-	0.792	0.004	0.0064	-	0.766	0.0045	0.0081	[28]
GPR (PUK)	0.9901	0.980	0.0023	0.0038	0.9754	0.951	0.0037	0.0062	Present study
GPR (Poly kernel)	0.964	0.929	0.0028	0.0047	0.9624	0.926	0.0223	0.0634
GPR (RBF)	0.9548	0.912	0.0031	0.0048	0.9387	0.881	0.0034	0.0047
CatBoost	0.960	0.922	0.0031	0.0052	0.958	0.9178	0.0013	0.0031

“-“ respective performance measure value is not reported in the reference.

lists the developed models’ results and their comparative performance results with other models reported in the literature. The top-ranked model was GPR-PUK, according to the results. Based on the training results, the R were 0.9901, 0.964, 0.9548; R² 0.980, 0.929, and 0.912; MAE 0.0023, 0.0028, and 0.0031; and RMSE 0.0038, 0.0047, and 0.0048 for GPR-PUK, GPR-Poly, and GPR-RBF models, respectively, the GPR-PUK outputs were verified to be the most compatible with actual coefficient of soil permeability values. Following that, GPR-Poly confirmed a high level of accuracy. Similarly, the GPR-PUK has the highest value of R (0.9754) and R² (0.951), then comes the GPR-Poly (R = 0.9624; R² = 0.926) and the GPR-RBF (R = 0.9387; R² = 0.881) in the test dataset. The GPR-Poly, on the other hand, has the lowest values of MAE (0.0034), followed by the GPR-PUK (0.0037) and the GPR-RBF (0.0223), and the GPR-RBF has the lowest value of RMSE (0.0047), followed by the GPR-PUK (0.0062) and the GPR-Poly (0.0634).

Figure 4a–c and Figure 5a–c show the graphical correlation between measured (x-axis) and estimated (y-axis) coefficients of soil permeability for the training and testing datasets, respectively. The estimated values by GPR-PUK in the training and test sets have a high consistency with the actual/experimental values but fewer error points, as illustrated in Figure 4a and Figure 5a. The trend line for GPR-PUK was drawn by comparing the observed regression in Figure 4 and Figure 5, and the GPR-PUK findings have the maximum inclination to the line (see Figure 3a and Figure 4a) in the training phase (R² = 0.980) and testing phase (R² = 0.951), respectively. As a result, the GPR-PUK model proposed in this study can be utilized to calculate the soil permeability coefficient, as the predicted value agrees well with the actual value, indicating that this approach can accurately and effectively estimate the coefficient of soil permeability.

4. Comparison of Performance with Other Methods

In this section, the proposed GPR models were compared with other prediction models, i.e., RF, ANN, SVM, and M5P or M5Prime, reported in the literature and the CatBoost regression model, which is implemented in Orange software. The established values of user-defined parameters identified from various runs are the number of trees (100), maximum depth for CatBoost (10), and learning rate (0.042). In comparison to the findings of previous research published in the literature by Pham et al. [1] and compared with the CatBoost model, in the training dataset, the GPR (PUK) has the highest value of R (0.9901), followed by the GPR (Poly kernel) (0.964), CatBoost (0.960), GPR (RBF) (0.9548), RF (0.972), artificial neural network (ANN) (0.948) and the support vector machine (SVM) (0.861). The GPR (PUK) and RF has the lowest MAE (0.0023), followed by the ANN (0.0027), GPR (Poly) (0.0028), GPR (RBF) and CatBoost (0.0031), M5P (0.004) and the SVM (0.0056), respectively. In contrast, RF has the lowest RMSE (0.0035) value in comparison to the GPR (PUK) (0.0038). Similarly, in the testing dataset, the GPR (PUK) has the highest R value (0.9754), followed by the GPR (Poly) (0.9624), CatBoost (0.958), GPR (RBF) (0.9387), RF (0.851), ANN (0.845) and SVM (0.844). The ANN, on the other hand, has the lowest RMSE (0.001), and the CatBoost has the lowest MAE (0.0013). The M5P model reported by Pham et al. [28] has the R² of 0.766, RMSE of 0.0064 and MAE of 0.004, in the case of the training dataset. Whereas, in the testing dataset, there is good agreement between actual and estimated values in the testing dataset, the M5P models’ error values are RMSE = 0.0081 and MAE = 0.0045 and the determination coefficient is high (R² = 0.766) in the testing dataset. In general, the proposed GPR-PUK (R² = 0.9754) has better prediction ability and has the highest goodness of fit with the data used in the training and testing datasets when compared to other models in this study.

5. Sensitivity Analysis

The developed models were evaluated using Yang and Zang’s [40] sensitivity analysis for measuring the influence of input factors on the coefficient of soil permeability. This approach [31,34,41,42,43,44] has been employed in a number of research investigations.

$$r_{ij}=\frac{{\displaystyle {\sum }_{m=1}^n\left(y_{im}\times y_{om}\right)}}{\sqrt{{\displaystyle {\sum }_{m=1}^n{y}_{im}{}^2{\displaystyle {\sum }_{m=1}^n{y}_{om}{}^2}}}}$$

(8)

where $n$ is the number of data values, $y_{im}$ and $y_{om}$ are the input and output parameters. For each input parameter, the $r_{ij}$ value varied from zero to one, with the highest values indicating the most efficient output parameter (which was k, in this study). The value of rij must be close to 1 in order to assess the relationship between input and output variables. Figure 6 depicts the relative importance of input factors based on experimental actual and predicted coefficient of permeability values. As can be seen, the relative importance of various parameters can be displayed as follows: w > e > LL > PL > CC > γ. In other words, the w is the most significant factor for estimating the coefficient of soil permeability, while γ is the least important parameter.

6. Conclusions

In this study, the GPR modeling method was used to estimate the coefficient of the permeability of soil with six input parameters: liquid limit LL (%), plastic limit PL (%), clay content CC (%), void ratio e, natural water content w (%), and specific density γ (g/cm³). The available data is divided into two parts: training set (70%) and testing set (30%). The following is a summary of the findings of this study:

• Comparing GPR models’ performance reveals that the GPR-PUK model gives more accurate prediction results with the coefficient of determination being 0.951, achieved from the correlation between experimental and estimated values of k.
• The GPR-PUK model’s estimation of the soil permeability coefficient was found to be more reliable than that of the ANN, SVM, RF, and M5P models reported in the literature.
• The findings of the sensitivity analysis demonstrate that different input factors have varying degrees of significance on the coefficient of soil permeability as w > e > LL > PL > CC > γ.

Development and improvement of the performance of models are a continuous process. The GPR-PUK model can accurately estimate the permeability coefficient of the soil using limited soil parameters, according to the findings of this study, but more research at different sites is needed to prove its wider application. It is evident that the proposed models are open to further modification, and that more data will result in much improved prediction capacity.