Neural Computing Improvement Using Four Metaheuristic Optimizers in Bearing Capacity Analysis of Footings Settled on Two-Layer Soils

Abstract

This study outlines the applicability of four metaheuristic algorithms, namely, whale optimization algorithm (WOA), league champion optimization (LCA), moth–flame optimization (MFO), and ant colony optimization (ACO), for performance improvement of an artificial neural network (ANN) in analyzing the bearing capacity of footings settled on two-layered soils. To this end, the models estimate the stability/failure of the system by taking into consideration soil key factors. The complexity of each network is optimized through a sensitivity analysis process. The performance of the ensembles is compared with a typical ANN to evaluate the efficiency of the applied optimizers. It was shown that the incorporation of the WOA, LCA, MFO, and ACO algorithms resulted in 14.49%, 13.41%, 18.30%, and 35.75% reductions in the prediction error of the ANN, respectively. Moreover, a ranking system is developed to compare the efficiency of the used models. The results revealed that the ACO–ANN performs most accurately, followed by the MFO–ANN, WOA–ANN, and LCA–ANN. Lastly, the outcomes demonstrated that the ACO–ANN can be a promising alternative to traditional methods used for analyzing the bearing capacity of two-layered soils.

1. Introduction

Soil bearing capacity is one of the most crucial engineering parameters which needs to be meticulously investigated before any construction action [1,2]. Thus, having an accurate approximation of the bearing capacity is a very important prerequisite of many geotechnical engineering projects as it is a function of various soil characteristics [3]. The ultimate applicable stress (F_ult) is obtained based on the maximum settlement ratio, which is 0.1 of the footing width [4,5]. In this regard, many scholars investigated or introduced relationships to give the F_ult [6,7]. Lotfizadeh and Kamalian [8] used the stress characteristic lines method for forecasting the static bearing capacity of strip footing installed on two-layered soils. Up to now, different numerical and analytical approaches were utilized to analyze the bearing capacity [9,10,11]. However, as a matter of fact, traditional methods and laboratory approaches are not applicable without spending a huge amount of time and money. On the other hand, due to the high competency of artificial intelligence techniques in different engineering applications, they can be used as inexpensive yet accurate models for estimating geotechnical parameters like bearing capacity.

The advent of soft computing approaches provided proper accurate models such as artificial neural network (ANN), adaptive neuro-fuzzy inference system (ANFIS), etc. for numerous engineering calculations with a focus on estimation tasks. These models were also successfully used for bearing capacity analysis [12,13,14]. In this sense, Padmini et al. [15] employed three models of neuro-fuzzy, ANN, and fuzzy for predicting the ultimate bearing capacity of shallow foundations (on cohesionless soil). Their results showed the superiority of intelligent models to popular bearing capacity theories. Also, Alavi and Sadrossadat [16] employed linear genetic programming to estimate the ultimate bearing capacity of shallow foundations resting on rock masses.

Metaheuristic algorithms suggest potent solutions for several optimization problems. They are also used for optimizing the performance of well-known predictive models like the support vector machine (SVM), ANN, and ANFIS [17,18,19]. As for the application of metaheuristic algorithms in bearing capacity analysis, different algorithms were applied to enhance the accuracy of the mentioned models [2,20,21]. Moayedi et al. [22] applied the biogeography-based optimization (BBO) algorithm to ANN and ANFIS for estimating the failure likelihood of shallow footings. The results showed that the used algorithm can increase the classification accuracy of the ANN (from 98.2% to 98.4%) and, more considerably, the ANFIS (from 97.6% to 98.5%). Likewise, Moayedi et al. [23] compared the optimization capability of the dragonfly algorithm (DA) and Harris hawks optimization (HHO) in adjusting the computational parameters of the ANN. Their study revealed that both these algorithms can effectively handle the mentioned task. However, referring to the calculated values of area under the curve (AUC), the DA (AUC = 0.942 and error = 0.1171) performed more accurately than the HHO (AUC = 0.915 and error = 0.1350).

According to the literature review, despite the broad application of popular metaheuristics (e.g., imperialist competition algorithm (ICA) and particle swarm optimization (PSO)) for bearing capacity analysis [2,24,25], there are still many unused techniques which might be more capable. Hence, the main focus of the present paper was to investigate the applicability of several metaheuristic algorithms, namely, whale optimization algorithm (WOA), league champion optimization (LCA), moth–flame optimization (MFO), and ant colony optimization (ACO), for optimizing the performance of the ANN to discover powerful models. The necessity of coupling these algorithms lies in some computational drawbacks [26,27] of the ANN which can be prevailed through proper adjustment of the weights and biases. In other words, the main contribution of these algorithms to the stated problem is to benefit metaheuristic advantages for the accurate evaluation of the relationship between bearing capacity and soil parameters.

Hereafter, the paper is structured in four major parts. The used algorithms are described in Section 2, data provision is explained in Section 3, results are presented and discussed in Section 4, and Section 5 gives the conclusion.

2. Methodology

2.1. Artificial Neural Network

The artificial neural network (ANN) is the basic model of this study which we aimed to optimize. ANNs showed high capability for estimating different engineering parameters [28,29,30]. Their high robustness in dealing with complex and non-linear tasks makes the ANNs universal approximators [31]. The idea of neural learning was first suggested by McCulloch and Pitts [32]. The ANN can be represented by different notions like radial basis function (RBF) and generalized regression neural network (GRNN), but the most common of these is multi-layer perceptron (MLP) [33]. An ordinary view of the MLP is depicted in Figure 1. It follows a so-called backpropagation (BP) learning method [34] with a Levenberg–Marquardt (LM) training algorithm [35] by default. This model benefits the mentioned items in mapping the relationship between two variables called input(s) and target(s). Mathematically, assuming T as the input of the j-th computational unit, the response (O) is calculated as follows:

$$O_j=F({\displaystyle \sum _{m=1}^M{T}_m{W}_{mj}+b_j)},$$

(1)

where F stands for the activation function, and the terms W_j and b_j are the corresponding weight and bias, respectively.

2.2. Hybrid Metaheuristic Algorithms

Due to the successful performance of metaheuristic algorithms in optimizing regular predictive models, in this study, four recently developed algorithms are applied to the ANN. The considered optimizers are the whale optimization algorithm (WOA), league champion optimization (LCA), moth–flame optimization (MFO), and ant colony optimization (ACO), which are used as potential search methods for finding the optimal solution to a given problem. In the case of this study, a general MLP is given as the problem, and, concerning a cost function, the algorithms aim to find the best weights and biases for the network.

The WOA was designed by Mirjalili and Lewis [36], inspired by the bubble-net hunting of humpback whales. Three major stages of this algorithm are (a) shrinking encircling hunt, (b) exploitation (i.e., bubble-net attacking), and (c) exploration (i.e., searching for the prey). More information about the WOA can be found in previous studies [37,38,39,40]. The LCA was proposed by Kashan [41], based on sporting competitions in sports leagues. Considering the league schedule programming and some relationships for determining the winner/loser team in an artificial league, the most appropriate solution is found. The LCA was detailed in previous studies [42,43,44]. As a novel nature-inspired optimization technique, the MFO was suggested by Mirjalili [45]. The pivotal idea of this algorithm is the navigation method of moths, which is known as transverse orientation. The candidate solutions in the MFO are moths, and their positions in space express the problem’s variables. The optimization process of this optimizer was well described in previous studies [46,47,48]. Lastly, the name ACO implies a population-based optimization technique which mimics the foraging behavior of ant herds. It was presented by Dorigo and Di Caro [49]. In this algorithm, artificial ants guide each other to achieve a proper (i.e., short) path leading to a promising food source. For more details, please refer to References [50,51,52].

The pseudo-code of the WOA, LCA, MFO, and ACO algorithms are shown below.

Algorithm 1. The pseudo-code of the whale optimization algorithm (WOA) [53]

Initialize the whale population Xi (i = 1, 2, …, n) Calculate the fitness of each search agent X∗ the best search agent while (t < maximum number of iterations)   for each search agent    Update a, A, C, l, and P    if1 (P < 0.5)       if2 (|A| < 1)        Update the position of the current search agent       else if2 (|A| ≥ 1)       Select a random search agent (Xrand)       Update the position of the current search agent       end if2    else if1 (P ≥ 0.5)       Update the position of the current search agent    end if1   end for   Check if any search agent goes beyond the search space and amend it   Calculate the fitness of each search agent   Update X∗ if there is a better solution   t = t + 1 end while return X∗

Algorithm 2. The pseudo-code of the league champion optimization algorithm (LCA) [54]

Initialize the league size (L) and the number of seasons (S); t = 1; Generate a league schedule; Initialize team formations (generate a population of L solutions) and determine the playing strengths (function or fitness value) along with them. Let the initialization also be the team’s current best formation; While t < = S × (L − 1)    Based on the league schedule at week t, determine the winner/loser among every pair of teams using a playing strength-based criterion;    t = t + 1    For i = 1 to L    Devise a new formation for team i for the forthcoming match, while taking into account the team’s current best formation and previous week events. Evaluate the playing strength of the resulting arrangement;       If the new formation is the fittest one (that is, the new solution is the best solution achieved so far for the i-th member of the population), hereafter consider the new formation as the team’s current best formation;    End For    If mod (t, L−1) = 0       Generate a league schedule;    End If End While

Algorithm 3. The pseudo-code of the moth–flame optimization (MFO) algorithm [55]

While iteration < max iteration Update flame number Obj = fitness function (Moths); if Iteration = 1       Sort the moths based on their objective functions; update the flames       Iteration = 0; else   Sort the moths based on their objective functions and flames from last iteration; update the flames end linearly decrease the convergence constant for j = 1: Number of moths       for k = 1: Number of variables, update r and t Calculate the distance of moth from each flame; update the values of the variables of moth from the corresponding flame       end end Iteration = iteration + 1; end

Algorithm 4. The pseudo-code of the ant colony optimization (ACO) algorithm [56]

Initialization:    Algorithm parameters;    Ant population size K;    Maximum number of iteration NMax; Generation:    Generating the pheromone matrix for the ant k;    Update the pheromone values and set x* = k;    i = 1; Repeat    for k = 1 to K    Compute the cost function for the ant k;    Compute probability move of ant individual;    if f(k) < f(x*) Then         Update the pheromone values;         Set x* = k; End if End for Until I > NMax;

3. Data Collection

By implementing a two-dimensional (2D) axisymmetric finite element method, a shallow footing settled on a two-layered soil was analyzed in different conditions. The settlement (U_y) was derived in each stage. A total of 901 analyses were carried out by 15-node triangular elements, where the effective variables were unit weight ($\frac{kN}{m^3}$), friction angle, elastic modulus ($\frac{kN}{m^2}$), dilation angle, Poisson’s ratio (v), applied stress ($\frac{kN}{m}$), and setback distance (m). Figure 2 illustrates the distribution pattern of these factors.

The descriptive statistics of the dataset are also presented in

Table 1. Descriptive statistics of the used dataset.

Features	Symbol	Descriptive Index
		Mean	Standard Error	Median	Mode	Standard Deviation	Sample Variance	Skewness	Minimum	Maximum
Friction angle	X1	36.75	0.13	36.00	36.00	3.91	15.28	−0.14	30.00	42.00
Dilation angle	X2	8.28	0.09	8.00	8.00	2.61	6.83	−0.39	3.40	11.50
Unit weight (kN/m3)	X3	20.44	0.02	20.50	20.50	0.65	0.43	−0.95	19.00	21.10
Elastic modulus (kN/m2)	X4	41,087.68	546.65	35,000.00	35,000.00	16,408.72	269,246,192.50	0.22	17,500.00	65,000.00
Poisson’s ratio (v)	X5	0.29	0.00	0.29	0.29	0.03	0.00	0.14	0.25	0.33
Setback distance	X6	4.19	0.07	5.00	5.00	2.08	4.31	−0.13	1.00	7.00
Applied stress (kN/m2)	X7	289.74	7.89	245.65	0.00	236.97	56152.92	1.26	0.00	1132.65
Settlement (m)	Y	0.04	0.00	0.03	0.00	0.03	0.00	0.46	0.00	0.10

. As can be seen, the minimum and maximum values obtained for the settlement were 0 and 0.10 m, respectively. Similar to a previous study [23], it was deemed that, if the U_y is less than 0.05 m, the system fails; otherwise, it remains stable. The failure and stability of the system are represented by values 1 and 0, respectively. The gathered dataset (without normalization) was then randomly divided into two groups, namely, training (for development of intelligent models) and testing (for evaluating the prediction capability of the models). In this regard, 721 samples (i.e., 80% of whole data) were specified to the first group, and the remaining 180 samples (i.e., 20% of whole data) served as the testing data (addressing unseen soil conditions).

4. Results and Discussion

To meet the objective of the study (i.e., investigating the optimization capability of the abovementioned metaheuristic algorithms), the algorithms should be coupled with the ANN. The aim of this work was to let this algorithm find the most appropriate matrix of the weights and biases for the ANN. To this end, firstly, an ANN with one hidden layer containing six neurons (determined by a trial-and-error process) was proposed as the base model. Thus, regarding the number of input/output parameters, the considered MLP took the form 7 × 6 × 1. Note that, in the present study, the activation functions of the hidden and output neurons were set to be “tangent-sigmoid (i.e., Tansig)” and “purelin”, respectively. Next, it was mathematically synthesized with the WOA, LCA, MFO, and ACO algorithms to create WOA–ANN, LCA–ANN, MFO–ANN, and ACO–ANN neural ensembles.

4.1. Hybridizing the ANN Using Metaheuristic Algorithms

After creating the ensembles, a population-based trial-and-error process was carried out to achieve the best-fitted complexity of the metaheuristic algorithms. To do so, all four networks were tested with nine different population sizes including 10, 25, 50, 75, 100, 200, 300, 400, and 500. Each model performed 1000 repetitions to minimize the error. In this process, root-mean-square error (RMSE) was set as the objective function (OF) to measure the training error in each iteration. This function is expressed in Equation (2). Figure 3a shows the obtained RMSEs for the tested population sizes. Also, the convergence curve of the most accurate one is illustrated in this figure.

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left(Y_{i_{observed} }-Y_{i_{predicted} }\right)}^2},$$

(2)

where N is the number of data, and Y_{i observed} and Y_{i predicted} stand for the observed and predicted stability values.

As can be seen, all four models exhibited an acceptable error in analyzing the relationship between the stability condition and its influential parameters. In detail, the smallest error was obtained for the WOA–ANN with a population size of 400 (RMSE = 0.307658318), LCA–ANN with a population size of 200 (RMSE = 0.312263011), MFO–ANN with a population size of 50 (RMSE = 0.298588793), and ACO–ANN with a population size of 10 (RMSE = 0.274504799).

4.2. Accuracy Assessment Criteria

The classification accuracy of the models was measured using a well-known criterion, namely, the area under the receiving operating characteristic curve (AUROC). Note that it was obtained by plotting the ROC diagrams, which is a good way of assessing the accuracy in diagnostic problems, such as natural hazard models [57,58,59,60,61]. Moreover, two error criteria of RMSE and mean absolute error (MAE) were used to measure the performance error of the models. Equation (3) expresses the formulation of the MAE.

$$MAE= \frac{1}{N}{\displaystyle \sum }_{i=1}^N\left(Y_{i_{observed} }-Y_{i_{predicted} }\right).$$

(3)

4.3. Accuracy Assessment of the Predictive Models

In this part, the results of the best-fitted models (i.e., with elite population sizes) are evaluated to examine their simulation capability. As is known, the results of the training phase address the learning quality of the model, and the testing results indicate the generalization capability for unseen conditions of the problem.

In the training phase, the calculated values of RMSE and MAE for the typical ANN were 0.3465 and 0.3055, respectively. Both of these indices experienced considerable decreases by applying the WOA (0.3076 and 0.2555), LCA (0.3122 and 0.2592), MFO (0.2985 and 0.2430), and ACO (0.2745 and 0.1783) optimization techniques. Also, in terms of the AUROC, the accuracy of the ANN was increased from 0.956 to 0.969, 0.964, 0.969, and 0.965, respectively. At a glance, it can be deduced that the models can improve the learning capability of the ANN. Figure 4 displays the predicted and actual stability values for the ensemble models. The output ranges were [−0.196124259, 1.163826771], [−0.285459666, 1.165811194], [−0.280854543, 1.220819059], and [−0.323683705, 1.197618633], respectively.

Similar to the first phase, all the neural-metaheuristic ensembles surpassed the ANN in the testing phase which means the algorithms have performed efficiently in adjusting the computational weights and biases of this tool. In detail, the RMSE was reduced from 0.3465 to 0.3076, 0.3122, 0.2985, and 0.2745. As for the MAE, it fell from 0.3055 to 0.2555, 0.2592, 0.2430, and 0.1783. The differences between the actual and predicted stability values (labeled as error) are illustrated in Figure 5, along with the histogram of the errors. The products of the WOA-ANN, LCA-ANN, MFO-ANN, and ACO-ANN vary in the extents [−0.19260775, 1.121547514], [−0.221848351, 1.183820947], [−0.176990489, 1.103579629], and [−0.072023941, 1.206028442], respectively.

Moreover, the ROC curves for the prediction of ensemble models are shown in Figure 6. The calculated areas under the curves indicate more than 90% accuracy for all five models. However, the AUROCs of the hybrid ensembles were significantly higher than the unreinforced ANN (AUROC = 0.930).

Until now, all used criteria confirmed that the metaheuristic algorithms can develop a more powerful ANN compared to the BP learning method. The results of the WOA–ANN, LCA–ANN, MFO–ANN, and ACO–ANN tools are evaluated in this section to compare the efficiency of the algorithms. A score-based system was developed to rank the models and determine the most accurate one. As

Table 2. The obtained values of root-mean-square error (RMSE), mean absolute error (MAE), and area under the receiver operating characteristic curve (AUROC). MLP—multi-layer perceptron; WOA—whale optimization algorithm; LCA—league champion optimization algorithm; MFO—moth–flame optimization; ACO—ant colony optimization; ANN—artificial neural network.

Models	Network Results
	Training			Testing
	RMSE	MAE	AUROC	RMSE	MAE	AUROC
MLP	0.3465	0.3055	0.956	0.3687	0.3312	0.930
WOA–ANN	0.3076	0.2555	0.969	0.3399	0.2832	0.939
LCA–ANN	0.3122	0.2592	0.964	0.3426	0.2868	0.935
MFO–ANN	0.2985	0.2430	0.969	0.3330	0.2706	0.939
ACO–ANN	0.2745	0.1783	0.965	0.3133	0.2128	0.944

;

Table 3. The developed ranking system based on the calculated accuracy criteria.

Models	Scores
	Training				Testing
	RMSE	MAE	AUROC	Score	RMSE	MAE	AUROC	Score
MLP	1	1	2	4	1	1	2	4
WOA–ANN	3	3	5	11	3	3	4	10
LCA–ANN	2	2	3	7	2	2	3	7
MFO–ANN	4	4	5	13	4	4	4	12
ACO–ANN	5	5	4	14	5	5	5	15

denote, each model received three scores based on the calculated RMSE, MAE, and AUROC. Then, the summation of these scores determined the model producing the most consistent outputs in each phase. According to Table 3, the ACO-based model grasped the highest scores in terms of all accuracy criteria except for the training AUROC (0.965) which was second to the WOA and MFO (0.969). Therefore, it grasped the highest overall scores in both training and testing phases, followed by the MFO, which closely surpassed the WOA, while the LCA featured the lowest rank.

Moreover, in comparison with the HHO and DA applied to the same data by Moayedi et al. [23], it was deduced that the methods of the current study present a more accurate analysis and approximation of bearing capacity. In detail, the RMSE and MAE obtained for the DA–MLP (superior to the HHO–MLP) were 0.3421 and 0.2904, which are larger than the results obtained for our WOA–ANN, MFO–ANN, and ACO–ANN. Also, the best AUROC of this study was higher than that for both models in the mentioned reference (0.944 vs. 0.942).

4.4. Presenting the Neural Predictive Formula

In this section, due to the largest accuracy obtained for the ACO–ANN, the neural relationship of this model was extracted and presented in the form of Equation (4) to predict the stability value using the considered effective parameters. Note that this formula was developed by the optimized parameters of the MLP output neuron. There were six middle parameters (A, B, …, F), which are expressed by Equation (5).

Stability value _ACO-ANN = 0.0684 × A + 0.1911 × B − 0.4022 × C − 0.8409 × D − 0.0458 × E − 0.9175 × F − 0.5984.

(4)

$$\left[\begin{array}{c}A\\ B\\ C\\ D\\ E\\ F\end{array}\right]= Tansig\left(\left(\left[\begin{array}{ccccccc}0.1966& -0.5483& -0.9172& -1.1127& 0.2130& -0.8492& 0.2915\\ -0.8962& 0.6059& -1.1167& -0.2273& -0.1856& 0.8686& 0.1111\\ -1.0910& -0.1197& -0.0399& 0.3251& 1.0160& -0.9038& 0.3303\\ 0.6287& 0.3309& -0.7786& -0.1076& 0.7085& -0.7997& 1.0031\\ 0.3544& 1.2248& -0.5554& -0.6322& 0.6777& 0.0116& 0.6907\\ -0.9035& 0.0708& -0.8111& 0.2542& 1.0138& 0.2361& -0.8017\end{array}\right]\left[\begin{array}{c}DOS\\ Sand\\ Loam\\ Clay\\ MC\\ LL\\ LI\end{array}\right]\right)+ \left[\begin{array}{c}-1.8084\\ 1.0850\\ 0.3617\\ 0.3617\\ 1.0850\\ -1.8084\end{array}\right]\right)$$

(5)

$$\mathrm{Tan}\mathrm{sig}\left(\mathrm{x}\right)=\frac{2}{1+{\mathrm{e}}^{-2\mathrm{x}}}-1.$$

(6)

5. Conclusions

The optimization potential of four wise metaheuristic techniques, namely, WOA, LCA, MFO, and ACO, was evaluated in this paper. The algorithms were coupled with an artificial neural network, and the developed ensembles were applied to an important geotechnical problem, bearing capacity analysis. It was revealed that ACO–ANN was the most accurate model. After that, the MFO–ANN was the second efficient ensemble. Based on the findings of this study, the combination of neural computing and metaheuristic techniques (especially the ACO–ANN) provides fast and inexpensive yet accurate models for analyzing the stability of the footings over two-layered soils. This is in contrast to the use of traditional methods (e.g., laboratory studies and finite element techniques), which are time-consuming and entail implementing costly and destructive tests. Furthermore, a comparison with conventional methods (MLP neural network) showed that using metaheuristic techniques can be an advantageous way of enhancing their performance (i.e., around 95% accuracy of prediction). In fact, this research suggests the use of powerful inspirations from real-world phenomena for optimizing the computational parameters of the ANN.