Preliminary Research on Outdoor Thermal Comfort Evaluation in Severe Cold Regions by Machine Learning

Abstract

The thermal comfort evaluation of the urban environment arouses widespread concern among scholars, and research in this field is mostly based on thermal comfort evaluation indexes such as PMV, PET, SET, UTCI, etc. These thermal comfort index evaluation models are complex in the calculation process and poor in operability, which makes it difficult for people who lack a relevant knowledge background to understand, calculate, and apply them. The purpose of this study is to provide a simple, efficient, and easy-to-operate outdoor thermal comfort evaluation model for severe cold areas in China using a machine learning method. In this study, the physical environment parameters are obtained by field measurement, and individual information is obtained by a field questionnaire survey. The applicability of four machine learning models in outdoor thermal comfort evaluation is studied. A total of 320 questionnaires are collected. The results show that the correlation coefficients between predicted values and voting values of the extreme gradient lifting model, gradient lifting model, random forest model, and neural network model are 0.9313, 0.7148, 0.9115, and 0.5325, respectively. Further analysis of the extreme gradient model with the highest correlation coefficient shows that individual factors (such as residence time, distance between hometown and residence, clothing, age, height, and weight) and environmental factors (such as air humidity (RH), wind speed (v), air temperature (Ta), and black bulb temperature (Tg)) have different influences on thermal comfort evaluation. In summary, using a machine learning method to evaluate outdoor thermal comfort is simpler, more direct, and more efficient, and it can make up for the lack of consideration of complex individual factors in the evaluation method of thermal comfort index. The results have reference value and application value for the research of outdoor thermal comfort evaluation in severe cold areas of China.

1. Introduction

The evaluation of urban environmental thermal comfort attracts wide attention from scholars [1,2,3].

In 1970, Professor Fanger developed the PMV model based on human thermal balance [4], which is mainly used to study the comprehensive influence of physical and physiological factors on human thermal comfort in the indoor thermal environment. The model, which considers three environmental parameters (air temperature, wind speed, and relative humidity) and three individual parameters (skin temperature, clothing thermal resistance, and metabolic rate), is adopted by ASHRAE 55 [5] and ISO 7730 [6], and remains the official model for evaluating the thermal comfort of buildings. In 1999, Höppe et al. proposed the physiological equivalent temperature [7]. The evaluation of outdoor thermal comfort takes into account solar radiation intensity, human metabolic intensity, and other relevant parameters, as well as the influence of the outdoor physical environment and more human physiological factors. In 2012, Bröde et al. proposed the Universal Thermal Climate Index (UTCI), which considers the dynamic physiological response of human body temperature regulation regulation and the clothing of subjects [8,9]. Scholars increasingly study and compare the relationship between thermal comfort and the use of urban open spaces [3,10,11,12,13,14]. Research in this field is mostly based on thermal comfort evaluation indexes such as PMV, PET, SET, UTCI, etc. [7,8,9,15].

A large number of studies show that human body heat relaxation is affected by physical factors [1,16,17,18] and individual factors [19,20,21]. Physical factors mainly include temperature (T), relative humidity (RH), wind speed (V), and solar radiation (G) [22,23,24,25]. Individual factors such as age, height, weight, gender, clothing thermal resistance (Clo), residence time (LoR), and hometown also have an impact. For example, studies showed that people’s sensitivity to heat decrease with age [22,26,27,28]. Thermal comfort is affected by clothing and metabolic rate [29,30], and LoR has a great influence on thermal comfort evaluation [21,31,32,33]. The main purpose of the thermal comfort index is to quantify the relationship between the outdoor thermal environment and human comfort, which has been studied by a large number of scholars. The thermal comfort index can be regarded as a comprehensive parameter of meteorological and human influence. However, the establishment process of the entire thermal comfort evaluation model is extremely complex due to the many factors that influence thermal comfort evaluation. To predict and evaluate outdoor thermal comfort in a specific area, it is often necessary to establish different thermal comfort rating scales and corresponding prediction models to match them [34,35,36,37]. Ruiz and Correa compared the applicability of the evaluation results of thermal comfort evaluation models and the original scale of actual thermal sensation to rate urban thermal comfort. The results showed a high correlation coefficient between the thermal comfort index and actual thermal sensation. However, the correct prediction percentage for all indexes was lower than 25%. To describe the neutral thermal comfort of the human body, scholars typically establish a relationship between selected indicators and the actual thermal sensation vote through regression analysis. Research results often vary in different climates [38,39,40,41], and there are still few studies on thermal comfort evaluation in severe cold areas, which are obviously different from tropical, arid, and temperate climates [42,43].

In recent years, there has been rapid development in the field of Machine Learning (ML) and Artificial Intelligence (AI). These technologies provide new possibilities for thermal comfort modeling that can adapt to regional and individual differences [41,43,44,45,46,47]. Outdoor thermal comfort has a complex relationship between the outdoor environment, individual emotion, and other factors. Some scholars have applied machine learning to study the prediction of personal thermal comfort. Peng Sha et al. investigated the satisfaction of fitness space in a park in Lhasa using a gradient lifting decision tree. The results show that there are differences between the factors affecting the satisfaction of fitness space in Lhasa and other cities in China [48].

The major factors affecting satisfaction are the greening environment, religious activities, fitness equipment, and facilities. Tang Hao developed statistical and machine learning model to explain the correlation between people’s satisfaction with the overall environment and individual environments. The models were evaluated based on their accuracy in predicting satisfaction reduction, global stability, and interpretability. The contribution of various independent variables to explanatory variables was quantified using the SHAP analysis method [49].

In summary, outdoor thermal comfort in severe cold area has not been widely studied, and existing thermal comfort evaluation models are complex in calculation, poor in operability, and inaccurate in prediction results, which cannot integrate all influencing factors. In order to provide an intuitive, fast, and easy-to-understand prediction method and make up for the lack of consideration for complex individual factors in thermal comfort index evaluation methods, this paper studies the physical environment parameters obtained by field measurements in typical cities in the severe cold area of China (Shenyang), and individual factor information obtained by field questionnaire investigation. The applicability of four machine learning models in outdoor thermal comfort evaluation is discussed.

The neural network model (NN) has been widely utilized in recent years as a machine learning model capable of capturing complex relationships between response and explanatory variables. Random Forest (RF), Gradient Boosting Decision Trees (GBDT), and Extreme Gradient Boosting (XGBoot) all consist of multiple decision trees, with the final results determined collectively by these trees. In terms of their inception, RF predates GBDT, which, in turn, predates XGBoot. They progressively adjust the weights of decision trees in their algorithms. All three models are suitable for regression model computations and data classification. Researchers have employed NN, RF, GBDT, and XGBoot for predictive modeling. Given the versatility and effectiveness of these models, our research team chose these four machine learning models for our study.

This article discusses the factors that affect thermal comfort evaluation in severe cold areas and their degree of influence. The information provided can be used as a reference for research and application of outdoor thermal comfort evaluation in severe cold areas in China.

2. Method

The research focuses on the outdoor space of a university campus in the severe cold area of Shenyang. The research investigates the influencing factors of outdoor thermal comfort and the evaluation of outdoor thermal comfort of the interviewees during spring in Shenyang. This is achieved through field measurement, questionnaires, and machine learning techniques. On this basis, four thermal comfort prediction models using machine learning algorithms were established. The models are compared by the best fitting state verification and effectiveness verification. Finally, the model with the highest determinant R² is selected as the main analysis object, and its characteristic importance and partial dependence are analyzed.

2.1. Data Acquisition Methodology

This study uses field measurements and questionnaire surveys to obtain research data. The investigation site consists of an open grassland and a shaded rest area on a university campus in Shenyang (Figure 1). Shenyang (41.8° N, 123.4° E) is located in Northeast China, and belongs to severe cold zone B according to GB 50176-2016 [50]. Shenyang experiences a short spring with abundant sunshine and a changeable climate. It is a windy season of the year, with an average daily temperature of −4 °C to 7 °C.

To determine the thermal comfort of the subjects under possible weather conditions in the severe cold area during spring, we conducted a continuous outdoor thermal comfort follow-up survey. To improve the survey response rate, the research team recruited ten female and ten male students who were willing to participate in the study prior to the survey, and organized a description of the survey to help the volunteers understand what they needed to do. To eliminate incomplete and negative surveys, the research team helped the volunteers complete the questionnaire on the spot (without guidance) and provided small souvenirs after completing the questionnaire to increase their enthusiasm. The research dates were from 8:00 to 18:00 (three days) on 4 April, 16 April, and 24 April 2023, in typical spring months.

The weather information for the monitoring days in Shenyang is shown in

Table 1. Weather conditions on monitoring days in Shenyang.

	Date	Minimum Temperature	Maximum Temperature	Weather	Wind Speed	Wind Direction
1	4 April 2023	−1 °C	14 °C	Cloudy to light rain	3.4–7.9	Northwest
2	16 April 2023	6 °C	18 °C	Sunny to cloudy	3.4–5.4	South
3	24 April 2023	8 °C	20 °C	Cloudy and sunny	3.4–7.9	North

.

Students were required to arrive at two survey sites in the morning, midday, and evening and to complete the current environmental thermal comfort evaluation questionnaire after experiencing the thermal environment of the survey sites for 5 min. The questionnaire included questions about gender, age, height, weight, time of arrival in Shenyang, hometown, dress situation, and TCV (Thermal Comfort Vote). The TCV evaluation system is widely used to assess thermal comfort, employing a 7-scale method. While outdoor thermal comfort evaluation is more complex and requires a more detailed evaluation, a percentage-based questionnaire is a more commonly encountered method in people’s daily lives. In our on-site survey questionnaire, participants completed the online questionnaire by sliding a slider. The research employed a percentile bipolar refinement thermal comfort score system (1–100). The system stipulates that 1 point means very uncomfortable, 50 points mean moderate, and 100 points mean very comfortable.

Physical environment parameters were automatically recorded by relevant measuring instruments, and the average value was taken every 5 min from 8:00 to 18:00. All instruments were selected according to ISO 7726 [51] standards. Refer to

Table 2. Technical parameters of the instrument.

Meteorological Parameters	Instrument Model	Measuring Range	Precision	Sampling Rate
Wind speed (V) m/s	testo 405 i anemometer	0~30 m/s	±0.1 m/s	2 s~12 h
Temperature (Ta) °C	AZ87786 thermodynamic index meter	0~50 °C	±0.6 °C	10 s~24 h
Humidity (RH) %	AZ87786 thermodynamic index meter	0~99%	±3%	10 s~24 h
Black Bulb Temperature (Tg) °C	AZ87786 thermodynamic index meter	0~80 °C	±1.5 °C	10 s~24 h

testo 405 i anemometer, testo, Titisee-Neustadt, Germany. AZ87786 thermodynamic index meter, AZ instrument corp, Taichung City, Taiwan.

and Figure 1 for technical parameters such as model, range, and accuracy of the instrument.

2.2. Research Methodology

Four machine learning methods are used in this study. After adjusting the corresponding learning parameters, the model is trained. The training is divided into two stages. The measured data are divided into two datasets: 320 data in the training set and 20 data in the verification set. In the training model stage, 320 pieces of data are used to establish the thermal comfort prediction model. In the prediction model verification stage, 20 pieces of data are used to evaluate the prediction performance of the prediction model. After these two stages, the simulation results of the four machine learning models are compared and analyzed. The training model that determines the best prediction performance is selected as the main model of this study. Then, the study conducts characteristic importance analysis and partial dependence analysis of its influencing factors.

2.2.1. Variable Parameters

(1) Label

Thermal Comfort Evaluation Vote (TCV) is chosen as the label, as referred to in Section 2.1, ‘Data acquisition methodology’.

(2) Feature

The dependent variables in this study include two types: environmental factors and individual factors. Environmental factor variables refer to four physical parameters in the PET evaluation system [8], including: air temperature (Ta), radiation temperature (Tg), wind speed (v), and air humidity (RH). There are seven individual factors, including: gender (male = 0, female = 1), height (Ht, in cm, only calculated in the model), weight (Wt, in kg, only calculated in the model), time to Shenyang (time, in months, only calculated in the model), hometown (DIST, divided by climate division, severe cold area = 1, cold area = 2, hot summer and cold winter area = 3, mild area = 4, hot summer and warm winter area = 5), clothing thermal resistance (clo., distributed in 0–2 according to the clothes of the subjects), and age (age, in years).

2.2.2. Machine Learning Model

Four machine learning models suitable for thermal comfort evaluation and prediction are selected.

(1) Neural Network Model

Neural Network (NN) is a new computing model inspired by biological neural networks, which are mainly used for artificial intelligence and machine learning tasks. It consists of a large number of neuron nodes, which transmit information through connection weights. Neural Network can carry out various tasks such as pattern recognition, data classification, and prediction [52].

(2) Stochastic Forest Model

Random Forest (RF) is an ensemble learning method, which is used to solve classification and regression problems, and improves the stability and accuracy of the model by combining multiple decision trees. The core idea of the stochastic forest is to combine several weak learners (decision trees) into a strong learner, and obtain the final prediction result through classification or regression. The basic unit of the random forest is the decision tree [52].

(3) Traditional Gradient Lifting Decision Tree

Traditional Gradient Boosting Decision Trees (Gradient Boosting Trees, abbreviated as GBT), is an ensemble learning method, which is used to solve regression and classification problems. It builds multiple decision trees iteratively step by step, and each decision tree is trained on the basis of the residual error of the previous tree, thus gradually reducing the error of the model [52].

(4) Extreme Gradient Lifting Model

Extreme Gradient Boosting (XGBoot) is a machine learning model based on gradient lifting tree and an ensemble learning algorithm. It is optimized and improved by regularization, feature enhancement, and custom loss function to provide higher performance, faster training speeds, and better robustness. XGBoot is one of the most powerful algorithms, widely used in data science and machine learning at present for various complex regression and classification problems and is suitable for thermal comfort evaluation problems [52].

2.3. Analytical Methods

The objective of this section is to address the regression between the measured value and the predicted value of thermal comfort evaluation. Firstly, this paper calls and debugs four models in turn. Secondly, after comparing and analyzing the simulation results of the four machine learning models, the machine learning model with the best determination coefficient (R²) is selected as the key model in this study. Subsequently, effectiveness analysis, important feature analysis, and partial dependence analysis are carried out. In addition, the best fit state is determined (Section 2.3.1).

2.3.1. Determination of the Best Fit State

In order to determine the best-fitting state of the model and prevent overfitting, the best-fitting parameters are determined by the minimum value of the Root Mean Squared Error (RMSE). RMSE measures the average absolute error between the predicted value of the model and the actual observed value, both of which range from 1 to 100. The smaller the absolute error is, the smaller the error is. The calculation of RMSE is as follows:

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\left(Y_i^{exp}-Y_i^{pred}\right)}^2}$$

(1)

In the formula, The Root Mean Square Error (RMSE) is a metric used to measure the average distance between the predicted and actual values of a regression model, N is the number of samples, and $Y_i^{exp}$ and $Y_i^{pred}$ are the average values of actual, predicted, and experimental values of $Y_i^{exp}$, respectively.

2.3.2. Validity Analysis

To evaluate the prediction accuracy of the model, the performance index determination coefficient (R²) is used in the validity analysis. The determination coefficient (R²) is an index used to measure the fitting degree of the regression model, which indicates how much the change in the dependent variable can be explained by the independent variable. The value range of R² is between 0 to 1, with a higher value indicating a better fit of the model to the dependent variable. The calculation formula for the determination coefficient R² is as follows:

$${\mathrm{R}}^2=1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(Y_i^{exp}-Y_i^{pred}\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(Y_i^{exp}-{\overline{Y}}_{ave}^{exp}\right)}^2}$$

(2)

In the formula, the coefficient of determination (R²) is a metric used to measure the goodness of fit of a regression model, n is the number of samples, and $Y_i^{exp}$ and $Y_i^{pred}$ are the average values of actual, predicted, and experimental values of ${\overline{Y}}_{ave}^{exp}$, respectively.

2.3.3. Important Feature Analysis

For models with many features, analyzing the importance of features can help us understand the decision-making process of the model and determine which features play a key role in the model’s performance. The study analyzes the influence degree of eleven independent variables. After the training model is completed, the importance score of each feature is obtained by the feature importance attribute provided by the model. Matplotlib, the basic drawing library in Python ecosystem, is called to visualize it, and these scores reflect the influence degree of influencing factors’ characteristics on the prediction model.

2.3.4. Characteristic Partial Dependence Analysis

Partial dependency is a tool used to analyze the relationship between machine learning model features and predicted values. With the help of the partial dependence analysis tool, this paper analyzes the influence of different characteristics on prediction. The SHAP (Shapley Additive exPlanations) function library is called to visualize the analysis results. It takes the eigenvalue as the horizontal axis and the vertical axis as the expected prediction value, keeping other features unchanged when generating data points, that is, fixing the values of other features in the training data. We ensure that we analyze the relationship between specific characteristics and predictions, and that other characteristics will not interfere with the results. By examining the trend in the partial dependence graph, we can determine how the change of feature value affects the prediction result.

3. Results

3.1. Best-Fitting State

After 50% cross-validation, the parameters with the lowest RMSE were selected as the best fit. The results are shown in Figure 2. The four kinds of machine learning are as follows: the best hidden layer size of the neural network (NN) is 69 neurons, the best number of fitting trees for random forest (RF) is 41, the best number of fitting trees for the traditional gradient lifting model (GBDT) is 16, and the best number of fitting trees for the limit gradient lifting model (XGBoot) is 10, and there are no overfitting problems.

3.2. Forecast Accuracy Analysis

Based on the prediction results of the four models, it is evident that the XGBoost model has the highest R² value for both the test set (0.8332) and the training set (0.9313), outperforming the NN, RF, and GBDT models. Therefore, the XGBoost model is chosen as the main analysis model for this study as it achieved the best performance in both stages (R² training = 0.9313, R² test = 0.8332).

Table 3. Correlation results table.

	Training Set	Test Set
NN	0.5325	0.4684
RF	0.9115	0.8041
GBDT	0.7148	0.8103
XGBoot	0.9313	0.8332

shows the specific training results.

The prediction results are shown in Figure 3:

3.3. Feature Importance Analysis

The proportion of influencing factors calculated by the XGBoot model is shown in Figure 4. The model indicates that relative humidity (RH) has the greatest influence on the prediction model, with a value of 0.161. This suggests that changes in RH have a significant impact on the thermal comfort of the environment. RH is influenced by air temperature and wind speed, which are related to the large temperature difference and relative humidity changes in spring in Shenyang. The evaluation of thermal comfort is influenced by various factors. Among these factors, wind speed, air temperature, and radiation temperature have a significant impact on the thermal comfort evaluation model (more than 0.8). This indicates that outdoor environmental factors play a crucial role in thermal comfort evaluation. On the other hand, gender has a negligible influence on the thermal comfort model (less than 0.4). It is important to note that subjective evaluations should be clearly marked as such to maintain objectivity.

3.4. Partial Dependency Analysis

The partial dependence of independent variables calculated by the XGBoot model is shown in Figure 5. The independent variables RH and clo. have a negative correlation trend with the thermal comfort evaluation of strain quantity, while wind speed, Ta, and Wt have a positive correlation trend with strain quantity. When the control independent variable is only RH, the predicted thermal comfort evaluation trend indicates that the fluctuation increases before reaching the extreme value, and then it shows a negative correlation trend after reaching the extreme value. The evaluation decreases from a high level (61) to a middle-high level of 52, then increases to a high level of 64, and finally decreases slowly with the increase of humidity, reaching a minimum value of 42, with a change range of 20. The evaluation of thermal comfort is significantly influenced by changes in humidity. The highest thermal comfort evaluation occurs at a humidity level of 21%. As humidity decreases or increases, the thermal comfort evaluation of the subjects gradually decreases, as shown in Figure 5a. When considering only the climate zone of the hometown as the independent variable, the thermal comfort evaluation value rapidly decreases from the high level of 74 to a low level of 58 with a change in climate zone. It then gradually decreases to its lowest value of 55.

4. Conclusions

This study obtained experimental data through field tests on university campuses in severe cold areas, and four machine learning models were used to predict the thermal comfort evaluation index. The study provided a construction method for an outdoor thermal comfort evaluation model in severe cold areas based on a machine learning model. The results are as follows:

(1)

The outdoor thermal comfort evaluation model based on the XGBoot model is effective. The correlation coefficients between predictive values and voting values of the extreme gradient lifting model, gradient lifting model, stochastic forest model, and neural network model are 0.9313, 0.7693, 0.7291, and 0.5311, respectively. The extreme gradient boosting model is the most effective.

(2)

The importance of the independent variables of the XGBoot model indicates that the evaluation of outdoor thermal comfort is influenced by numerous factors. The evaluation of outdoor thermal comfort is influenced by various individual and environmental factors. These include living time, distance between hometown and residence, clothing, age, height, weight, air humidity (RH), wind speed (v), air temperature (Ta), and black bulb temperature (Tg). It is important to consider all of these factors objectively when evaluating outdoor thermal comfort.

(3)

Partial dependence of the XGBoot model shows that various influencing factors have different effects on the evaluation of outdoor thermal comfort. Respondents in the climate zone of severe cold areas in their hometowns believe that the thermal comfort evaluation in a slightly cold thermal environment is higher, while those with a larger clothing coefficient often give a lower thermal comfort evaluation value.

There are still some limitations in this study. The data mainly consist of students from college campuses in severe cold areas during spring. Although the students come from different climatic zones, the measured climatic zones are limited. Additionally, the sample size is insufficient, and the student group only represents the 20–30 age group, which cannot be widely representative of social groups. Therefore, further research is necessary.