Compressive Strength Prediction of Fly Ash Concrete Using Machine Learning Techniques

Abstract

It is time-consuming and uneconomical to estimate the strength properties of fly ash concrete using conventional compression experiments. For this reason, four machine learning models—extreme learning machine, random forest, original support vector regression (SVR), and the SVR model optimized by a grid search algorithm—were proposed to predict the compressive strength of fly ash concrete on 270 group datasets. The prediction results of the proposed model were compared using five evaluation indices, and the relative importance and effect of each input variable on the output compressive strength were analyzed. The results showed that the optimized hybrid model showed the best predictive behavior compared to the other three models, and can be used to forecast the compressive strength of fly ash concrete at a specific mix design ratio before conducting laboratory compression tests, which will save costs on the specimens and laboratory tests. Among the eight input variables listed, age and water were the two relatively most important features with superplasticizer and fly ash being of weaker relative importance.

1. Introduction

Concrete is one of the most commonly used building materials in the world. However, the production of concrete and its ingredients requires large amounts of energy. It not only consumes natural resources but also releases carbon dioxide, which places huge pressure on the environment [1,2,3]. The use of supplementary cementitious materials (SCMs) in concrete can help in the reduction of CO₂ emissions [4,5,6]. Therefore, utilizing SCMs in concrete may be an effective and sustainable approach [7]. Fly ash is considered as an SCM that can partially replace cement in concrete. In addition to reducing greenhouse gas emissions, fly ash concrete has some advantages [8]: (1) It can improve the fluidity, cohesion, and water retention of concrete; (2) it can reduce the heat release of hydration in concrete and obviously reduce the temperature cracks, which is especially beneficial for large volume concrete projects; (3) due to the effect of secondary hydration, the compactness of concrete is increased and the interface structure is improved, which can improve the impermeability and sulfate corrosion resistance of concrete; (4) concrete with the same strength, with the addition of fly ash, can reduce the amount of cement to about 10%, which greatly reduces the production cost. Moreover, the durability of fly ash concrete is higher than that of cement concrete due to the prolonged volcanic ash reaction. Therefore, the use of fly ash in concrete can help reduce CO₂ emissions by reducing the cement demand and producing materials with improved mechanical properties.

Among the mechanical properties of concrete, compressive strength is one of the most basic but important mechanical properties. To achieve the expected compressive strength of fly ash concrete through a reasonable mix ratio, the traditional method requires constant adjustment of the concrete mix ratio to produce laboratory concrete specimens, which are then subjected to compression tests to obtain their compressive strength. If the strength does not reach the expected standard, the specimens need to be recreated, which is a very time-consuming and labor-intensive task. If a method could be used to roughly estimate the compressive strength of concrete from a given mix before the compression tests, it would greatly save time and the cost of testing and producing specimens.

With the advent of machine learning and its development in the field of civil engineering, various machine learning models have been applied that have achieved good results in predicting the mechanical properties of concrete [9,10,11,12,13,14,15,16,17]. Xu et al. [18] proposed an improved random forest model to predict the compressive strength of high-performance concrete. The results showed that the prediction accuracy of the model was higher than that of other algorithms, and the method demonstrated a strong generalization capability. Saha et al. [19] employed the support vector regression (SVR) method to predict the compressive strength of self-compacting concrete and reached a high-correlation coefficient R exceeding 0.97 between the predicted and actual strength. Moreover, the prediction performance of the SVR model was better than the artificial neural network (ANN) and multivariable regression analysis. Farooq et al. [20] introduced four machine learning algorithms—random forest, gene expression programming, ANN, and decision tree—to predict the compressive strength of concrete, and random forest showed the best prediction performance. Shamiri et al. [21] developed an extreme learning machine model to predict the strength of high-strength concrete, and the model achieved high accuracy and good generalization ability. Chen et al. [22] utilized two models, the long short-term memory (LSTM) network and SVR, to predict the compressive strength of high-strength concrete, and the results showed that the LSTM outperformed the original SVR model with a higher correlation coefficient and smaller error metrics. To improve the performance of the original SVM, Hao et al. [23] proposed a combination of SVM and K-fold cross-validation to predict the concrete compressive strength. The prediction accuracy of the proposed hybrid model improved when compared to the original SVM model.

In general, these intelligent models have shown good applicability and performance in predicting the mechanical properties of concrete predictions. However, it is undeniable that the predictive performance of many original models is limited, and the performance of the original models is often related to the hyper-parameters of the models when the dataset is fixed. Therefore, one branch of research into machine learning models is the application of optimization algorithms in machine learning models. Further exploration in this area is needed to better understand and apply these machine learning models. Considering the good performance of the SVR model in regression prediction, this paper proposes employing SVR as the base model and using a grid search (GS) optimization algorithm to improve the prediction performance of the model to achieve an accurate prediction of the compressive strength of fly ash concrete. Then, the model prediction results are compared with the prediction results of the three models: extreme learning machine (ELM), original SVR, and random forest.

2. Methodology

2.1. Extreme Learning Machine

The extreme learning machine is used to train single hidden layer feedforward neural networks (SLFN). Unlike the traditional SLFN training algorithm, the extreme learning machine (Figure 1) randomly selects the input layer weights and the hidden layer bias. The output layer weights are calculated by minimizing a loss function consisting of the training error term and the regular term of the output layer weight parametrization based on Moore–Penrose’s generalized inverse matrix theory. Theoretical studies show that ELM maintains the generic approximation capability of SLFN, even if the hidden layer nodes are randomly generated. In the past decade, the theory and applications of ELM have been widely studied, and from the perspective of learning efficiency, the extreme learning machine has the advantages of few training parameters, fast learning speed, and strong generalization ability.

$$f_L(x)={\displaystyle \sum _{i=1}^L{\beta }_i{h}_i(x)=H(x)\beta }$$

(1)

where x is the input; β is the output weight between the hidden layer and the output layer; h_i(x) is the output of the i-th hidden layer node; and the output of the hidden layer is H(x). The output functions of the hidden layer nodes are not unique, and different output functions can be used for different hidden layer neurons. Usually, in practical applications, h_i(x) is represented as follows.

$$h_i(x)=g(W_i,b_i,x)=g(W_{i}x+b_i)$$

(2)

g(x) is the activation function, and the Sigmoid function can usually be used.

$$g(x)=\frac{1}{1+e^{-x}}=\frac{e^x}{e^x+1}$$

(3)

2.2. Random Forest

Random forest is one of the most advanced integration algorithms with the advantages of a few model parameters and high-resistance to overfitting. Decision trees are the basic predictors of random forest. As shown in Figure 2, the random forest regression algorithm uses decision trees as the basic model and produces a series of differentiated decision tree models by constructing different training datasets and different feature spaces, usually using voting or averaging to obtain the final results [18]. The mathematical expression is shown below.

$$H(x)=\frac{1}{k}{\displaystyle \sum _{i=1}^k{h}_i(x)}$$

(4)

where H(x) denotes the predicted value of the random forest regression model; and h_i(x) denotes the output of the i-th decision tree model.

2.3. Support Vector Regression Model

The support vector machine regression principle is different from the traditional regression technique, which mainly uses the structural risk minimization principle instead of the empirical risk minimization principle [24]. As shown in Figure 3, there is a sample training set $D=\{(x_i,y_i)|i=1,2,\cdots n\}$, $x_i\in R^n$, $y_i\in R$. When the regression problem is linear, a function f(x) is probed on Rⁿ so that y_i = f(x), and f(x) can be described as follows.

$$f(x)=\omega \cdot x+b$$

(5)

where ω is the weight vector, and b is the bias. In the case of nonlinearity, the nonlinear regression problem can be transformed into an optimization problem of the solution function [25].

$$\begin{array}{l}\min (R(\omega ,{\xi }_i,{\xi }_i^{*}))=\frac{1}{2}\omega \cdot \omega +C{\displaystyle \sum _{i=1}^n({\xi }_i+{\xi }_i^{*})}\\ s.t.\{\begin{array}{l}{y}_i-[\omega \cdot \varphi (x_i)+b]\le \epsilon +{\xi }_i\\ \omega \cdot \varphi (x_i)+b-y_i\le \epsilon +{\xi }_i^{*}\\ {\xi }_i,{\xi }_i^{*}\ge 0\end{array}\end{array}$$

(6)

where C is the penalty parameter, if a larger value of c indicates a heavier penalty for errors; ε is the insensitive loss parameter; ${\xi }_i$ and ${\xi }_i^{*}$ are the two slack variables.

It is generally solved by introducing the Lagrange function and using its pairwise problem.

$$\begin{array}{l}\max (W(\alpha ,{\alpha }^{*})=-\frac{1}{2}{\displaystyle \sum _{i,j=1}^n({\alpha }_i-{\alpha }_i^{*})}({\alpha }_j-{\alpha }_j^{*})[\varphi (x_i)\cdot \varphi (x_j)]\\ -{\displaystyle \sum _{i=1}^n({\alpha }_i+{\alpha }_i^{*})}\epsilon +{\displaystyle \sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})y_i}\\ s.t.\{\begin{array}{l}{\displaystyle \sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})=0}\\ 0\le {\alpha }_i,{\alpha }_i^{*}\le C,i=1,2,\cdots n\end{array}\end{array}$$

(7)

By introducing the kernel function $K(x_i,x_j)$, it can be implemented in the support vector machine algorithm to solve the high-dimensional computational problem. Then, the regression fitting function of the support vector machine is:

$$f(x)=\omega x+b={\displaystyle \sum _{i,j=1}^n({\alpha }_i-{\alpha }_i^{*})K(x_i,x_j)+b}$$

(8)

The kernel functions are the linear kernel function, polynomial kernel function, and Gaussian kernel function, with the latter being the most widely utilized [26,27]. As a result, the Gaussian kernel function is used.

2.4. Grid Search Optimization Algorithm

It is generally known that the selection of hyper-parameters for the SVR model has a significant impact on the identification results. Because the parameters are extremely nonlinear, determining the combination of parameters such as the penalty parameter C and the kernel parameter g frequently necessitates many experiments. Although Lin et al. [28] have conducted a lot of work to make SVM easier to use such as introducing LIBSVM, the choice of parameters C and g is still based on experience. As a result, there is a pressing need to implement automatic parameter tuning to acquire the best value once the parameters have been entered. These issues can be solved using the grid search method. Grid search may be the simplest and most widely used hyperparametric search algorithm, which permutes the possible values of each parameter and lists all possible combinations to generate a “grid”. Each combination is then used for SVM training, and performance is evaluated using cross-validation. In practice, the grid search method generally uses a wide search range and a large step size to find the possible locations of the global optimum, and then gradually reduces the search range and step size to find a more precise optimum.

3. Materials and Dataset Description

3.1. Input and Output Variables

A dataset with a total of 270 groups was collected from the shared dataset of the UC Irvine Machine Learning Repository [29]. These data included eight input variables and one output variable. These variables were cement, fly ash, water, superplasticizer, coarse aggregate, fine aggregate, age, and W/C (water to binder), respectively.

Table 1. Variable statistical characteristics.

Variable	Unit	Max	Min	Average	Standard Deviation	Kurtosis	Skewness
Cement	kg/m3	540	247	361	85.33	−0.50	0.82
Fly ash	kg/m3	142	0	28	48.26	−0.44	1.2
Water	kg/m3	228	140	184	19.13	0.29	−0.38
Superplasticizer	kg/m3	28	0	4	5.94	3.52	1.77
Coarse aggregate	kg/m3	1125	801	997	77.12	−0.19	−0.26
Fine aggregate	kg/m3	900	594	776	79.77	−0.07	−0.67
Age	day	365	1	53	75.91	7.01	2.62
W/C	-	0.70	0.27	0.53	0.11	−0.04	−0.92
Strength	MPa	80	6	36	14.97	−0.13	0.45

and Figure 4 show the statistical characteristics and distribution of these variables. The distribution of these variables showed that the variables were highly discrete. The Pearson coefficients, as shown in Figure 5, appear to make it difficult to identify a clear relationship between the individual components and the output variable. This also shows from the side that there is a complex nonlinear correlation between the multiple input variables and compressive strength. The purpose of this paper was to capture this nonlinear correlation. A total of 216 sets of data were chosen at random as the training set and the remainder as the test set from all of the data samples.

3.2. Evaluation Metrics

To quantify the effectiveness of the model training and prediction, five evaluation metrics were introduced [30]. The formulae for calculating these indicators are shown in

Table 2. Equations of the evaluation indices.

Evaluation Metrics	Equation
R	$R=\frac{n{\displaystyle \sum _{i=1}^n(Y_e{Y}_p)}-{\displaystyle \sum _{i=1}^n{Y}_e{\displaystyle \sum _{i=1}^n{Y}_p}}}{\sqrt{[n({\displaystyle \sum _{i=1}^n{Y}_e^2})-{({\displaystyle \sum _{i=1}^n{Y}_e})}^2][n({\displaystyle \sum _{i=1}^n{Y}_p^2})-{({\displaystyle \sum _{i=1}^n{Y}_p})}^2]}}$
MAE	$MAE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|Y_e-Y_p\right|}$
MSE	$MSE=\frac{1}{n}{\displaystyle \sum _{i=1}^n{(Y_e-Y_p)}^2}$
RMSE	$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{(Y_e-Y_p)}^2}}$
MAPE	$MAPE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{Y_e-Y_p}{Y_e}\right|}\times 100\%$

where Y_e is the true result of compressive strength and Y_p is the predicted result of compressive strength.

[31,32]. R can reflect the correlation between the predicted compressive strength and the actual compressive strength, and the closer R is to 1 indicates a stronger correlation. The other four error indicators can reflect the deviation and dispersion degree between the predicted compressive strength and the measured compressive strength.

4. Model Performance

For a grid search, a rough grid search is generally performed first, followed by a fine grid search. Figure 6 shows the matching MSE distribution for each search node under an exponential grid of [2⁻⁵, 2⁵] with a step of 2^0.2. After the grid search, the optimal combination of hyper-parameters C and g was found.

For comparison and assessment purposes, Figure 7 shows the predicted and experimental values for the training and test sets in greater detail. Among the four models listed, the ELM model exhibited the worst prediction behavior, with correlation coefficients R less than 0.9 for both the training and testing sets. The prediction correlation coefficient R of the SVR and random forest model both exceeded 0.9, but the prediction performance of the random forest model on this dataset was better than that of the classical SVR model in terms of the correlation coefficient. However, the optimized SVR-GS model showed the best prediction capacity among these four models. With a correlation coefficient R above 0.94 for both the training and test sets, the predicted values matched the experimental values well at each sample point, confirming the SVR-GS model’s correctness and effectiveness in capturing the intricate nonlinear relationships between the eight input variables and compressive strength. It is worth mentioning that the predictive accuracy of the SVR model optimized by the grid search algorithm became greatly improved. This is mainly attributed to the fact that, unlike the original SVR model, which uses default values or other random combinations of values for the hyper-parameters C and g, the optimized parameter combination achieves a global optimum, and this combination of optimal parameters greatly improves the predictive performance of the model.

Figure 8 shows a comparison of the expected and actual values for each sample for a more intuitive presentation, and the error metrics for model training and testing are shown in Figure 9. It can be observed that for both the training and test sets, the highest values for RMSE, MSE, MAPE, and MAPE were recognized by the ELM model, while the hybrid SVR-GS model achieved the highest prediction accuracy in terms of the lowest RMSE = 4.814, MSE = 23.177, MAPE = 3.432, and MAPE = 13.818%. Compared with the average relative errors of 23.71%, 16.51, and 16.17% for the ELM, random forest, and SVR models, respectively, the optimized model achieved the smallest average relative error of 13.82%, which was less than 15%. This shows that the prediction accuracy of the SVR-GS model can basically meet the engineering requirements and could serve as an efficient alternative method to predict the compressive strength of fly ash concrete.

5. Sensitivity Analysis of Input Variables

Each input variable influences the output compressive strength in the fly ash concrete mix design, although the degree of this influence varies. As a result, it is crucial to assess the relative importance and contribution of each variable to the final output. To analyze the feature importance of each variable, the Shapley additive explanations (SHAP) approach is introduced [33,34,35].

Figure 10 shows that age was the most important of the eight input factors in the current dataset, followed by W/C, water, cement, fine aggregate, and fly ash. Compressive strength was less affected by the coarse aggregate and superplasticizer. Figure 11 shows whether these variables have a positive or negative effect on the output results. A tangible sample is represented by each point in Figure 11, and the variables listed from the top to the bottom have the highest to the lowest relative importance. It can be seen that the age, cement, fly ash, and superplasticizer were all beneficial to compressive strength, and the increase in these variables caused an increase in the compressive strength. In contrast, water and W/C had a negative effect on the compressive strength.

6. Conclusions

In this work, an optimized SVR-GS model was introduced to predict the compressive strength of the fly ash concrete. The main findings are as follows.

(1)

The proposed hybrid model could effectively capture the complicated nonlinear correlations between the eight input variables and the output compressive strength of the fly ash concrete.

(2)

The prediction performance of the SVR-GS model was better than that of the other three machine learning models with a higher prediction accuracy and smaller error, and is recommended for the pre-estimation of the compressive strength of fly ash concrete before laboratory compression experiments.

(3)

Concerning the eight input variables, age was the most important, followed by W/C, water, cement, fine aggregate, and fly ash. Coarse aggregate and superplasticizer were less important for compressive strength. Moreover, age, cement, fly ash, and superplasticizer all played a positive role in the compressive strength and their increase led to an increase in the compressive strength, while water and W/C were negative for the compressive strength of fly ash concrete.

Due to the lack of sufficient datasets, the influence of other factors such as the concrete chemical composition, aggregate type, and particle size, etc. on the predictive performance of the model was analyzed. As the corresponding dataset is collected, these influencing factors will be considered in the model as input features. Moreover, the effectiveness and validity of the model in practical applications will be further analyzed and discussed in future research.