A Hybrid Model for Spatiotemporal Air Quality Prediction Based on Interpretable Neural Networks and a Graph Neural Network

Abstract

To effectively address air pollution and enhance air quality, governments must be able to predict the air quality index with high accuracy and reliability. However, air quality prediction is subject to ambiguity and instability because of the atmosphere’s fluidity, making it challenging to identify the temporal and spatial correlations using a single model. Therefore, a new hybrid model is proposed based on an interpretable neural network and a graph neural network (INNGNN), which simulates the temporal and spatial dependence of air quality and achieves accurate multi-step air quality prediction. A time series is first interpreted using interpretable neural networks (INN) to extract the potentially important aspects that are easily overlooked in the data; second, a self-attention mechanism catches the local and global dependencies and associations in the time series. Lastly, a city map is created using a graph neural network (GNN) to determine the relationships between cities in order to extract the spatially dependent features. In the experimental evaluation, the results show that the INNGNN model performs better than comparable algorithms. Therefore, it is confirmed that the INNGNN model can effectively capture the temporal and spatial relationships and better predict air quality.

1. Introduction

Human health and ecosystems depend heavily on the air quality, and air pollution is becoming a major global issue [1]. The term “air quality” refers to the quantity and make-up of several pollutants found in the atmosphere, including carbon monoxide, sulfur dioxide, nitrogen dioxide, PM2.5, PM10, and others. It is calculated based on the level of pollutant concentration and the degree of health impact, and is used to measure the air quality index (AQI). In addition to causing respiratory illnesses, air pollution has a negative impact on human health [2]; moreover, it also damages the environment and leads to an imbalance in the ecosystem [3]. The higher the AQI value, the worse the air quality. The air quality index (AQI) is a crucial metric that can help people understand that day’s air quality and take the appropriate precautions to safeguard their health. Additionally, pollutants can lower the AQI value and preserve the ecosystem’s equilibrium.

Air quality forecasting is a technique for predicting future air quality. This technology uses professional monitoring equipment and technology to collect and analyze atmospheric parameters (such as temperature, humidity, wind speed, air pressure, precipitation, etc.), geographical location, the air quality index, and other data, to predict and warn about air quality in the future [4]. The accurate prediction of AQI is crucial for safeguarding public health and maintaining ecosystem balance. However, air quality prediction (AQP) remains a challenging task due to complex temporal and spatial dependencies:

• Temporal dependence. Periodicity and trend are the primary ways in which dynamic changes in air quality over time are expressed. Periodicity is the occurrence of similar patterns or regular changes over a certain span of time. As shown in Figure 1a, the air quality index changes periodically over a week (① indicates a period of change). Trend is when a certain data pattern shows a continuous directional development over a certain period of time. Figure 1a shows a downward trend on certain days and for a certain period of time with cyclical changes. As shown in Figure 1b, the air quality index over a day changes with time (② indicates a shift in the trend over a certain amount of time); for instance, air pollutants from a prior or longer period of time have an impact on the current AQI.
• Complex spatiotemporal correlations. In addition to changing dynamically over time, the spatial location also has an impact on air quality. As illustrated in Figure 2, city B’s air quality will be impacted by the atmospheric conditions in city A. Even when taking time into account, the spatial relationship is still challenging.

There are many existing AQP methods, some of which consider the correlation with time, including the autoregressive integrated moving average model [5], the recurrent neural networks model [6], and the long short-term memory model (LSTM) [7]. It is challenging for us to estimate the level of air quality with accuracy, since the aforementioned approaches take into account the dynamic changes in air quality, but do not consider the spatial relationship between air quality and geographic location. Some researchers have used convolution neural networks (CNNs) [8], which are often used in Euclidean space. So, in order to make up for these shortcomings, researchers could use graph convolutional neural networks (GCNs) [9], which can model non-Euclidean spaces well, but are unable to accurately represent the dependency of air quality between cities. Some researchers have used CNNs to analyze and model air quality in order to express its spatial characteristics. In order to improve the defects of GCNs, the group aware graph neural network (GAGNN) [10] constructs the dependencies between cities. Although these models can represent the spatial relationship of air quality between cities, they ignore the temporal relationship. The graph convolutional neural network–long short-term memory model (GCN-LSTM) [11] incorporates time and space relationships to fully consider time and space dependencies; nonetheless, it still has flaws in handling non-stationary data problems. The ability of researchers to identify patterns and trends across various time ranges is still lacking.

Time series forecasting challenges have been the focus of research when using interpretable neural network (INN) models. A number of strategies and techniques have been applied to enhance the model’s interpretability. The proposed NA-BEMD [12] model interprets the model by analyzing the weights of each stage of the time series through an attention mechanism. The N-BEATS [13] model makes the model interpretable through the use of specific functions in order to solve the problem of time series point prediction. In order to improve the estimation accuracy of time series forecasting, the MLP-M [14] model makes the middle layer of the neural network model adjustable and, thus, improves the interpretability of the model. The above model improves the accuracy of time series prediction by enhancing the interpretability of the neural network model, and allowing for a better understanding of the process of feature learning and prediction in the model.

The key to AQP is to effectively capture the patterns and trends in various time frames. A hybrid prediction model for multistage, spatiotemporal air quality prediction is provided based on interpretable neural networks and graph neural networks (INNGNN) in order to address the issues raised by this problem, and to enhance prediction stability and accuracy. The contributions of this paper are summarized as follows:

• An advanced AQP module is constructed, which introduces the extraction of interpretable trends and periodic time series features. In order to thoroughly extract the properties from the data, the interpretation module uses residual connections in conjunction with the trend and periodicity of the time series, to extract the features that are easily missed and are challenging to extract at random moments.
• The INNGNN hybrid model is proposed to perform a spatiotemporal prediction of AQP from time and space dimensions. Graph neural networks (GNN) are used to extract the spatial dependency between different cities, and interpretable neural networks (INN) are used to capture the temporal dependence between the observations made on multiple time scales, and allows for self-attention to acquire the local and global dependence of time. The prediction accuracy of the INNGNN model is enhanced, as shown by its more accurate prediction outcomes in the evaluation we conducted on a Chinese urban air quality dataset.

2. Related Work

Deep learning models and non-deep learning models are the two broad groups into which existing AQP techniques can be separated. Deterministic models and statistical models are the two types of non-deep learning models. The most representative and latest deterministic models for predicting air quality include the community multi-scale air quality (CMAQ) model [15,16,17] and the comprehensive air-quality model with extension (CAMx) [18,19]. These models make it simple to see how pollutants that have an impact on air quality are produced, transformed, and transported, but they have significant limitations when it comes to the incomplete previous knowledge of complex chemicals and their diffusion processes. The statistical models highlight the non-linear correlations among all the factors that may affect air quality, thereby overcoming the shortcomings of the deterministic models. Classic statistical methods, such as the auto regression integrated moving average (ARIMA) [5,20,21,22] and geographically weighted regression (GWR) [23], have been used with statistical models for predicting small datasets and univariate time series of air quality. In addition, traditional machine learning methods along with statistical models have also been applied to AQP, such as random forest (RF) [24], SVM [22,25,26,27], improved SVM (LSSVM) [28], LR [29,30], ANNs [31,32], and the improved neural network models BPNN [33], GRNN [32], RBFNN [31], and other models for processing. Based on restricted datasets, the typical machine learning approach can only capture limited nonlinear temporal and spatial correlation aspects affecting air quality. It is more appropriate for shallow hidden features, as the results demonstrate. Very complex dynamic nonlinear spatiotemporal connection features are still not amenable for the efficient acquisition of huge datasets.

In recent years, deep learning has been applied extensively to AQP, and this has significantly increased prediction accuracy. Among them, a recurrent neural network (RNN) [34] has been used to capture temporal dependence. RNN’s enhanced models—long short-term memory (LSTM) [35,36,37] and gated recurrent units (GRUs) [38,39]—provide an improved long-term dependency model for AQP. Some researchers have proposed improved LSTM-related models, including read-first LSTM (RLSTM) [40] and vanilla LSTM with multichannel input and multi-route output (IVLSTM-MCMR) [41], which have enhanced the function of the multidimensional feature extraction of air quality. An auto-encoder with Bi-LSTM neural networks (AE-Bi-LSTM) [42] has been used to analyze the correlation between air quality and climate variables. Some hybrid models have been used to improve the temporal dependence; for example, the use of LSTM-LSTM [43] improves the nonlinear capability of air quality data. For AQP, the hybrid model attention-LSTM [44] is used to automatically train and focus on significant time steps in the time series data. While the temporal correlation of AQP can be extracted efficiently using the deep learning RNN correlation model, the patterns and trends for various time scales in the time series remain ineffectively captured. In addition, a number of variables that impact air quality must be taken into account, and that information is intricate. Temporal and spatial linkages must be taken into account while modeling, yet these models still have issues with handling these relationships.

Some academics have suggested using different deep learning algorithms for time and space modeling in order to make up for the drawbacks of the RNN-related models. Strong feature extraction is a capability of convolutional neural networks (CNNs) [45,46,47,48], which may also be used to extract the spatial correlations between several sites and regions. When used in AQP, they produce better prediction outcomes than the RNN correlation models. The primary use of CNN is in Euclidean space. AQP’s multiregional space is typically non-Euclidean in nature. Graph convolutional neural networks (GCNs) [9,11] are capable of performing undirected graph data creation. The model is a good representation of the non-Euclidean space of the network of air quality monitoring points. In order to better represent the city dependence of AQP, the group-aware graph neural network (GAGNN) [10] has been proposed, and its results are better than those of the GCNs. These models have demonstrated clear advantages for the extraction of spatial features. Several hybrid deep learning models have been developed to address the spatial–temporal dependence of AQP. These models, which include CNN-LSTM [47,49,50], GC-LSTM [11], and CNN-biLSTM [51], are able to extract spatiotemporal correlations from historical air quality data.

Combined with the hybrid model, which combines a residual neural network (ResNet), bidirectional long short-term memory (BiLSTM), and GCN, the adaptive monitoring network topology can adapt to the characteristics of pollutants and improve the prediction accuracy [52]. The GCN and CNN hybrid model, incorporating a spatiotemporal attention mechanism module, enhances the features of critical information collected from across the different dimensions [53]. Even though these models have performed well in AQP, they still struggle to handle non-stationary data issues. The fusion of multiple CNNs and backpropagation neural networks (BPNN) has good denoising capabilities [54]. The fusion of CNN with a spatiotemporal attention mechanism and residual learning enhances the temporal and spatial dimension feature extraction [55]. The fusion of LSTM and wavelet transform can handle unstable time series signals very well [56], but these models still cannot handle the correlation between multiple sites well. The fusion of a time convolutional network and a graph convolutional network can deal with multi-site dependency and time dependency well, but it ignores the characteristics of repeated time series changes within a certain time interval [57]. The AAGCN model integrates the transformer and GCN, which can capture multivariable dynamic dependencies, but cannot fully capture the time dependencies of time series [58]. Therefore, more research is necessary to identify the patterns and trends across a range of time scales.

3. Methodology

3.1. Problem Definition

In this study, the goal of air quality forecasting is to predict the air quality for a certain period of time based on the AQI, weather data, and geographic location. We will first define some concepts to describe the AQP problem.

Definition 1:

cities and city groups. We use a set $\mathrm{S}={\left\{s_i\right\}}_{i=1}^{N_s}$ to represent the collection of cities, where $N_s$ represents the number of cities, and the location of a city is represented by its longitude and latitude, and the location matrix of the city is expressed as $L\in {\mathbb{R}}^{N_s\times 2}$. In addition, a set $\mathrm{U}={\left\{u_i\right\}}_{i=1}^{N_u}$ is used to represent the set of city groups, where $N_u$ represents the number of city groups.

Definition 2:

city group graph. We will consider city groups as cities that have strong dependencies, and construct the city group graph as a fully connected graph. We use an undirected graph $g(v,\epsilon )$ to represent the topological structure of a city group graph, as shown in Figure 3, where $v=\left\{v_1,v_2,\dots \dots ,v_n\right\}$ represents the set of all city group nodes, and for any node $v_i$, we use $R_i$ to represent its attribute value, and the attribute values of all nodes in $v$ can be expressed as the matrix $R$. $\epsilon =\left\{{\epsilon }_1,{\epsilon }_2,\dots \dots ,{\epsilon }_n\right\}$ represents the set of edges of all city groups. For any edge ${\epsilon }_i$, we use $M_i$ to represent its attribute value, and the attribute values of all the edges in $\epsilon$ can be expressed as the matrix $M$.

Definition 3:

city graph. We use an undirected graph $G(V,E)$ to represent the topology of a city graph, as shown in Figure 3, where $V=\left\{{\mathrm{v}}_1,{\mathrm{v}}_2,\dots \dots ,{\mathrm{v}}_n\right\}$ represents the set of all city nodes, and for any node $V_i$, we use $X_i$ to represent its attribute value, and the attribute values of all nodes in $V$ can be expressed as the matrix $X$. $E=\left\{e_1,e_2,\dots \dots ,e_n\right\}$ represents the set of all city edges. For any edge $E_i$, we use ${\mathrm{Y}}_{\mathrm{i}}$ to represent its attribute value, and the attribute values of all the edges in $E$ can be expressed as the matrix $\mathrm{Y}$. The city graph is constructed as follows:

$${\mathrm{Y}}_{\mathrm{i},\mathrm{j}}=\frac{1}{\sqrt{{(a_i-a_j)}^2+{(b_i-b_j)}^2}},$$

(1)

$$d_{i,j}=\sqrt{{(a_i-a_j)}^2+{(b_i-b_j)}^2},0<d_{i,j}<r_n,$$

(2)

where ${\mathrm{Y}}_{\mathrm{i},\mathrm{j}}$ represents the edge attribute of the edge between city $S_i$ and city $S_j$, $[a_i,b_i]$ represents the location of city $S_i$, and $[a_j,b_j]$ represents the location of city $S_j$. $d_{i,j}$ represents the Euclidean distance between city $S_i$ and city $S_j$, and $r_n$ represents the distance threshold. When the distance between the two cities is very large, the cities are hardly affected by each other. The strategy we chose is that when the distance is less than $r_n$, city $S_i$ and city $S_j$ can be connected on the city map.

Definition 4:

urban AQP. This paper mainly predicts the future AQI of a city by giving the historical observation sequence of the city’s location, time, AQI, and weather data. That is, for city  $S_i$, given the city’s location  $L$, the time vector  $T^t$, and the historical observation sequence  $H=(h_i^{t_1},h_i^{t_2},\dots \dots ,h_i^{t_m})$  of m time steps at time t, we can predict the AQI observation sequence  $\widehat{H}=({\widehat{h}}_i^{t_{m+1}},{\widehat{h}}_i^{t_{m+2}},\dots \dots ,{\widehat{h}}_i^{t_{m+n}})$  of the city’s n time steps.

3.2. Framework

Figure 4 displays the INNGNN framework structure. The temporal dependency module and the spatial dependency module make up the INNGNN. The temporal dependency module extracts the local and global time series features using self-attention and the interpretable trend, and the periodic time series features using INN. The geographical dependence of the geographic location is extracted with the spatial dependency module using GNN. First, the historical observation sequence feature matrix of the 209 cities is input, and the time series features of the different cities are extracted through multiple INNs. Then, the features extracted from the different cities are spliced together. Self-attention is added to the time series in order to capture the local and global dependencies of the various time steps. The input for self-attention is the feature matrix output obtained from splicing several city characteristics that were retrieved using multiple INNs. Natural language processing frequently uses encode–decode to manage mapping relationships and to map one sequence onto another more effectively [59,60]. The GNN module receives its input from the self-attention output. The GNN module uses the encode–decode structure and, to implement the message transmission mechanism, two layers of GNN are layered in the encode–decode, respectively. The decoder uses the output from the encoder to update the city and implement the mapping relationship between the individual cities and city groups. Ultimately, the fully connected layer outputs the prediction findings.

3.3. Temporal Dependency Modeling

3.3.1. Interpretable Neural Networks

Given its periodic and trend properties, we suggest using interpretable neural networks (INN) to explain the periodicity and trend of the AQI time series. The two stacks comprising the interpretable neural networks, as depicted in Figure 5, are utilized to elucidate the trend and periodicity of the time series. Each stack consists of three blocks, and each block in a stack comprises two parts: the first portion comprises four layers of fully connected layers, and the second component is used to generate the expansion coefficients for the trend and periodic functions. A residual connection is introduced, meaning that each stack completes its own information aggregation procedure, and the input of the current block, less the output of the current block, is used as the input of the next block in order to capture the features of different levels in the input data. Input the historical observation sequence $H_i$ of the i-th city, and the first part is described by Formula (3) as follows:

$$h_{l,1}=F{C}_{l,1}(x_l),h_{l,2}=F{C}_{l,2}(h_{l,1}),h_{l,3}=F{C}_{l,3}(h_{l,2}),h_{l,4}=F{C}_{l,4}(h_{l,3}),$$

(3)

where $h_{l,n}$ represents the $n$th layer of the $l$th block in each stack, $x_l$ represents the residual of the previous block as the input of the current block, and $FC$ represents the standard fully connected layer of the RELU nonlinear function, that is, $h_{l,1}=RELU(W_{l,1}{H}_l+b_{l,1})$. The second part is determined using Formula (4), and is described as follows:

$$\begin{array}{l}{\varphi }_l^a=LINEA{R}_l^a(h_{l,4}),{\varphi }_l^b=LINEA{R}_l^b(h_{l,4})\\ {\widehat{y}}_l=f_l^b({\varphi }_l^b),{\widehat{x}}_l=f_l^a({\varphi }_l^a)\end{array},$$

(4)

where $LINEAR$ is a simple linear projection layer, such as ${\varphi }_l^b=W_l^b{h}_{l,4}$. ${\widehat{y}}_l$ and ${\widehat{x}}_l$ represent the output function of the current block of each stack and the residual calculation function of the next block, and ${\varphi }_l^a$ and ${\varphi }_l^b$ are the expansion coefficients of ${\widehat{y}}_l$ and ${\widehat{x}}_l$, respectively. The periodicity and trend of the time series’ interpretability are specifically described as follows.

• Trend: The time series of air quality has a certain trend, as shown in Figure 1b. One of the trend characteristics is that it exhibits an upward or downward trend over time. Either a slowly changing function or a monotone function can simulate the trend. A slowly changing function is used here. The formula is described as follows:

$${\widehat{y}}_l^{trend}=T{\varphi }_l^b,{\widehat{x}}_l^{trend}=T{\varphi }_l^a,$$

(5)

where $T=\left[1,t,\dots ,t^p\right]$ is a power exponent matrix of $t$. When the $p$ value is small, such as 3 or 4, it will force ${\widehat{y}}_l^{trend},{\widehat{x}}_l^{trend}$ to simulate the trend. $t$ is a discrete grid from 0 to $\left(N-1\right)/N$, and $t$ is expressed as ${\left[0,1,2,\dots ,N-2,N-1\right]}^T/N$, $N$ is the number of steps, the first stack is used as the trend model, the historical sequence $H_i$ is used as the input, and the process formulae for the input and output are described as follows:

$$\begin{array}{l}{\widehat{x}}_1=H_i-{\widehat{x}}_1^{trend},{\widehat{x}}_2={\widehat{x}}_1-{\widehat{x}}_2^{trend},{\widehat{x}}_3={\widehat{x}}_2-{\widehat{x}}_3^{trend}\\ y={\displaystyle \sum _l{\widehat{y}}_l^{trend}+}{\widehat{x}}_3\end{array},$$

(6)

where ${\widehat{x}}_1,{\widehat{x}}_2,{\widehat{x}}_3$ represent the residuals of each block in the first stack and $y$ represents the final output of the first stack, which is used as the input for the periodicity of the next stack.

• Periodicity: As seen in Figure 1, there is a certain periodicity to the air quality time series (a). Periodic functions can be chosen to mimic the periodicity. Periodicity is defined as recurring or cyclical patterns within a specific time span. The Fourier series is chosen here, and the formula is described as follows:

$${\widehat{y}}_l^{per}=P{\varphi }_l^b,{\widehat{x}}_l^{per}=P{\varphi }_l^a,$$

(7)

where $P=[1,\cos (2\pi t),\dots ,\cos (2\pi ⌊N/2-1⌋t),\sin (2\pi t),\dots ,\sin (2\pi ⌊N/2-1⌋t)]$ is the matrix of the sine and cosine waveforms, and the meanings of $t$ and $N$ are the same as those for the trend. The second stack has periodicity characteristics, and the process formulae for the input and output are described as follows:

$$\begin{array}{l}{\tilde{x}}_1=y-{\widehat{x}}_1^{per},{\tilde{x}}_2={\tilde{x}}_1-{\widehat{x}}_2^{per},{\tilde{x}}_3={\tilde{x}}_2-{\widehat{x}}_3^{per}\hfill \\ {\mathrm{Y}}_i^{\prime }={\displaystyle \sum _l{\widehat{y}}_l^{per}+}{\tilde{x}}_3\hfill \end{array},$$

(8)

where ${\tilde{x}}_1,{\tilde{x}}_2,{\tilde{x}}_3$ represent the residuals of each block in the second stack, respectively, and ${\mathrm{Y}}_i^{\prime }$ represents the final output of the second stack, which is the feature extraction of the i-th city. Then, the features of each city are merged and completed using the stacking approach. The formula is described as follows:

$${\mathrm{Y}}_i=\left[{\mathrm{Y}}_1^{\prime };{\mathrm{Y}}_2^{\prime };{\mathrm{Y}}_3^{\prime };\dots ;{\mathrm{Y}}_4^{\prime }\right],$$

(9)

where ${\mathrm{Y}}_i$ represents the feature merging of all the cities as the input for the next part of the model.

3.3.2. Time Step Local and Global Dependency Capturing

A self-attention method is introduced in order to be able to capture the correlation between the distinct time steps of the time series, as well as the global dependent features and local dependent features. As shown in Figure 6, the self-attention mechanism is designed by referring to transformer [61] architecture, in which a multi-head attention can consider local and global dependencies at the same time. Each head can pay attention to the dependencies between the various time steps by computing the correlation weights between the items. The computation procedure is represented by the following formula:

$$Q=f({\mathrm{Y}}_i,W_Q),K=f({\mathrm{Y}}_i,W_K),V=f({\mathrm{Y}}_i,W_V),$$

(10)

$$Attention(Q,K,V)=soft\max \left(\frac{Q{K}^T}{\sqrt{d_K}}\right)V,$$

(11)

where $Q$ is the query matrix, $K$ is the key matrix, and $V$ is the value matrix. $W_Q,W_K,W_V$ are all learnable parameters, $K^T$ represents the transpose of $K$, and $d_K$ represents the dimension of the $K$ vector. Through matrix multiplication, the degree of association between each element’s query vector and key vector in the matrix is realized, a weight calculation is performed, a normalization operation is realized through $soft\max$ and, then, a weighted summation is performed. Finally, all the dimensions are flattened using skip connection. The output is obtained through a fully connected (FC) layer.

3.4. Spatial Dependency Modeling

The spatial relationship of the location data for each city can be shown as a graph. The spatial interdependence between cities must be taken into account in order to forecast the air quality of each city. A graph neural network (GNN) is used to construct the dependencies between cities. A detailed description of the composition process is as follows.

• City grouping: The cities with strong dependencies are assigned to a city group, and each city is mapped onto a city group one by one; this grouping method allows us to identify any potential spatial dependencies between the cities. We use the $\mathsf{\Omega }\in {\mathbb{R}}^{{\mathrm{N}}_{\mathrm{s}}\times {\mathrm{N}}_{\mathrm{u}}}$ matrix to represent the mapping relationship between the cities and city groups. Cities can be assigned to multiple city groups. In order to illustrate the correlation between cities and city groups, $\mathsf{\Omega }$ is randomly initialized during training and optimized at the same time. For the case given, and shown in Figure 7, there are 10 cities divided into 3 city groups, among which the probability of city $s_6$ being assigned to city group $u_1$ is 0.1, the probability it of being assigned to city group $u_2$ is 0.8, and the probability it of being assigned to city group $u_3$ is 0.1. This shows that city $s_6$ has a stronger correlation with city group $u_2$. In order to capture the spatial dependence between the cities, the geographic location $L$ of the city is added, and the process definition for city grouping is as follows:

$${\widehat{X}}_i=FC(g_v(X_i,L_i)),$$

(12)

$$R_j={\displaystyle \sum _{i=1}^{N_s}{\Omega }_{j,i}^T{\widehat{X}}_i},$$

(13)

where $X_i$ is the output of the temporal dependent module, $L_i$ represents the geographic coordinates of city $s_i$, ${\widehat{X}}_i$ represents city $s_i$ and contains location information, $g_v(\cdot )$ represents the function implemented by using a FC layer, and ${\Omega }_{j, i}^{}$ represents city $s_i$ assigned to the city probabilities for groups $u_j$. We then have ${\displaystyle {\sum }_{j=1}^{N_u}{\Omega }_{i,j}=1}$, where $R_j$ represents the city group representation of the cities assigned to the city group, and ${\Omega }_{}^T$ represents the transpose of the matrix $\Omega$.

• Dependencies between city groups: In a city group, the nodes of each city group are fully connected to generate a fully connected undirected graph $\mathrm{g}$, and the dependency relationship between the city groups is modeled through the mechanism of message passing. The modeling process is as follows:

$$M_{i,j}=f_g(R_i,R_j,T^t),$$

(14)

$$\begin{array}{c}{M}_i={\{(R_i,R_j,M_{i,j})\}}_{i\ne j}\\ {\hat{M}}_i\leftarrow {\mu }_g(M_i),{\hat{R}}_i\leftarrow {\phi }_g({\hat{M}}_i,R_i)\end{array},$$

(15)

where $M_{i,j}$ represents the city group $u_i$, the city group $u_j$ contains the edge attributes of the time vector $T^t$, and the time vector $T^t$ is composed of months, weeks, and days. $R_i$ and $R_j$ are the attribute values of the city groups $u_i$ and $u_j$, respectively, $M_i$ is all the information collected and is passed on to city group $u_i$ through message passing, and is converted into vector ${\widehat{M}}_i$ after the message passing is completed. ${\widehat{R}}_i$ is the city group representation after the city group $u_i$ is updated. $f_g$, ${\mu }_g$, and ${\phi }_g$ are all implemented using a FC layer.

• Dependencies between cities: During city representation, the cities are updated through the process of assigning cities to city groups, and the dependency relationship between the cities is completed through the transmission of messages in city graph $G$, and the message transmission mechanism is similar to that of the city groups. The difference is that the cities incorporate time series features from the temporal dependency module output, geographic location information, and city group information assigned to the city. The specific process is as follows:

$${\tilde{X}}_i={\displaystyle \sum _{j=1}^{N_u}{\Omega }_{i,j}{\widehat{R}}_j},$$

(16)

$$X_i^{\prime }=concat(X_i,{\tilde{X}}_i),$$

(17)

$$\begin{array}{c}{\mathrm{Y}}_i={\{({\tilde{X}}_i,{\tilde{X}}_k,{\mathrm{Y}}_{k,i})\}}_{k\in k(i)}\\ {\hat{\mathrm{Y}}}_i\leftarrow \mu ({\mathrm{Y}}_i),X_i^{″}\leftarrow \phi (X_i^{\prime },{\hat{\mathrm{Y}}}_i)\end{array},$$

(18)

where ${\widehat{R}}_j$ represents the updated city group $u_j$; ${\tilde{X}}_i$ represents the city group containing city $s_i$; $X_i^{\prime }$ represents city $s_i$ and integrates information from the time series features and city group information, which contains geographic location information; and ${\mathrm{Y}}_i$ represents all the information collected through message transmissions that were passed to city $s_i$. The neighbor information is aggregated, and is then converted into vector ${\hat{\mathrm{Y}}}_i$, $X_i^{″}$, which is the updated city representation of city $s_i$. The connection functions $concat$,$\mu$, and $\phi$ are also implemented using a FC layer.

4. Experiments

4.1. Data Description

The INNGNN model was evaluated using the AQI dataset of some cities in China. With an hourly sampling frequency from 1 January 2017 to 30 April 2019, 209 cities were gathered as the dataset. The dataset includes geographic location, weather, and AQI data. The details of the data are as follows:

• AQI dataset: The data come from the national urban air quality real-time release platform, downloaded from the public platform https://drive.google.com/file/d/1I_vpbLJhOJpNh-TpLdSWsaG3xCpzMVSQ/ (accessed on 15 June 2023.).
• Weather data: Weather data include the humidity, wind direction, rainfall, wind speed, air pressure, temperature, and visibility. The data come from the open platform for environmental big data http://www.envicloud.cn/ (accessed on 15 June 2023).
• Geographic location data: The geographic location of each city is shown in Figure 2. The geographic locations of all the cities are marked on the map with red dots.

The time step in this study is 1 h, the time step for a day is 24, and samples with a step of one were generated using a sliding window. We sorted all of the generated samples into training, test, and validation sets, and arranged them chronologically.

4.2. Experimental Settings

Experiments on the GPU were conducted, and all model constructions were completed within the open-source pytorch framework. The training epoch for all the models was set to 300, the batch size was set to 64, and the learning rate was set to 0.001. In addition, the power index $p$ of matrix $T$ was set to 4, the size of the city group was 15, the hidden unit of the GNN was 32, the distance threshold $r_n$ was 250 km, and the GNN was set to 2 layers. We used the Adam [62] optimizer to optimize the training of the model.

In addition, AQP is a multiple linear regression problem. In order to effectively evaluate the model, the deviation between the predicted value and the observed value was calculated, using the mean absolute error (MAE) and the root mean square error (RMSE) as the indicators for the model’s evaluation. The calculation process is as follows:

$$MAE=\frac{1}{N_S\times \tau }{\displaystyle \sum _{i=1}^{N_S}{\displaystyle \sum _{t=1}^{\tau }\left|{\mathrm{O}}_i^t-{\tilde{O}}_i^t\right|}},$$

(19)

$$RMSE=\sqrt{\frac{1}{N_S\times \tau }{\displaystyle \sum _{i=1}^{N_S}{\displaystyle \sum _{t=1}^{\tau }{\left|{\mathrm{O}}_i^t-{\tilde{O}}_i^t\right|}^2}}},$$

(20)

where $\tau$ represents the number of time samples, and $O_i^j$ and ${\tilde{O}}_i^j$ represent the observed value and predicted value for city $i$ at time $t$, respectively. The same experimental parameters were used for five verifications for each of our tests, and the average result was used to determine the final outcome.

4.3. Experimental Results

4.3.1. Comparative Prediction Results

In this section, the INNGNN model is compared to the following baseline models:

DeeperGCN [63]: An extended form of GCN, DeeperGCN uses a deeper network structure that enhances its capacity to model complicated graph data. Eight layers were used in the experimental comparison.

LSTM [35]: LSTM (long short-term memory) is a variant of a recurrent neural network (RNN), which solves the problems of traditional RNNs regarding gradient disappearance and gradient explosion. The experiment’s hidden unit was set to 32, the predicted value of the AQI was the output, and the features from the historical observation sequence were the input.

GC-LSTM [11]: The GC-LSTM model combines a graph convolutional neural network (GCN) with a long short-term memory network (LSTM). In the experimental settings, the optimal setting of LSTM hidden units was 64.

GAGNN [10]: GAGNN is a neural network model designed to analyze graph data with intra- and inter-group links. It is specifically designed for graph-structured data. The hidden layer unit was set to 32 in the experiment.

SHARE [64]: With numerous hierarchical recursive layers and graph attention layers, SHARE is a semi-supervised hierarchical recursive graph neural network model for graph-structured data. The shared semi-supervised portion of the experimental settings was removed, while the other parameters stayed the same.

ST-UNet [65]: ST-UNet is a neural network model for processing graph-structured time series data. To improve the time series modeling ability of the graph-structured time series data, the model presented an the extended GRU. The hidden unit of the expanded GRU wad 32 in the experiment.

XGBoost [66]: XGBoost is based on the decision tree model and introduces feature parallel computing. For parameter optimization of the experiments, network search methods were used.

HighAir [67]: A neural network model, HighAir uses graph structural information to optimize the graphs. It creates connections between the graph data by simulating the dynamic elements and creating the graphs from a hierarchical viewpoint.

The prediction task results for the baseline model and the INNGNN model, which were run on the dataset for one to six hours per day, are displayed in

Table 1. Prediction results of the INNGNN model and baseline model.

Model	Metric	1 h	2 h	3 h	4 h	5 h	6 h
INNGNN	MAE	5.48	8.49	10.67	12.34	13.72	14.91
	RMSE	10.70	16.03	19.66	22.29	24.37	26.11
DeeperGCN	MAE	6.54	9.74	11.77	13.40	15.29	16.41
	RMSE	13.67	18.93	21.14	23.83	26.25	28.02
LSTM	MAE	6.50	10.26	13.18	15.52	17.40	18.91
	RMSE	13.85	19.26	23.52	26.83	29.46	31.55
GC-LSTM	MAE	5.95	9.16	11.58	13.46	15.00	16.31
	RMSE	11.91	16.98	20.82	23.69	25.97	27.82
GAGNN	MAE	5.56	8.59	10.80	12.52	13.91	15.10
	RMSE	10.81	16.17	19.84	22.51	24.63	26.37
SHARE	MAE	5.84	9.07	11.49	13.35	14.74	15.79
	RMSE	11.27	16.84	20.77	23.60	25.80	27.38
ST-UNet	MAE	5.95	9.30	11.58	13.38	14.82	16.02
	RMSE	11.74	18.01	21.34	23.90	25.94	27.64
XGBoost	MAE	6.85	10.89	13.99	16.27	18.14	19.56
	RMSE	14.25	19.80	24.72	28.14	30.63	33.44
HighAir	MAE	5.50	8.52	10.81	12.50	14.00	15.09
	RMSE	10.80	16.10	19.85	22.70	24.91	26.40

. The INNGNN model’s assessment metrics, MAE and RMSE, showed the best prediction results across all the prediction ranges, demonstrating the model’s efficacy in the task of spatiotemporal air quality prediction.

In terms of performance, the hybrid INNGNN model is still superior to the graph neural network. As shown in Table 1, for the 1 h and 6 h prediction tasks, the MAE error of the INNGNN model was reduced by 19% and 10%, respectively, and the RMSE error was reduced by 27% and 7%, compared to the DeeperGCN model. In the 6 h prediction task, the RMSE error of the INNGNN model was reduced by 4% and 5.8%, compared to the SHARE model and the ST-UNet model, respectively. When compared to the enhanced graph neural network models, GAGNN and HighAir, the INNGNN model improved the MAE and RMSE errors for short-term, medium-term, and long-term predictions during the 1 h to 6 h prediction tasks. For the 1 h prediction task, the MAE error was reduced by 6.6% and 8.6%, respectively. Our model’s MAE and RMSE errors are still significantly better than those of the HighAir model, which has a better prediction effect. From this thorough comparison, the INNGNN model improves upon the graph neural network foundation by incorporating the extraction of temporal data. This allows for a more accurate capture of the temporal and spatial connections, particularly for long-term predictions, demonstrating its superior benefits.

Regarding the time series feature extraction process, the INNGNN model adds trend and periodic feature extraction, as compared to the GC-LSTM hybrid model. This greatly enhances the feature capture of burst points. In order to better extract the spatiotemporal aspects of air quality prediction, the feature extraction for the correlation between cities is integrated into the spatial feature extraction process concurrently. As demonstrated in Table 1, the INNGNN model’s prediction accuracy outperformed that of the GC-LSTM model over the 1–6 h prediction test.

4.3.2. Comparative Analysis: Individual Module vs. Hybrid Model

A few of the model’s modules were investigated to confirm that the suggested hybrid INNGNN model is accurate in its forecasting performance. Figure 8 illustrates how we compared the RMSE and MAE values of the prediction outcomes of the GNN module, INNGNN model, and INN module from one hour to six hours, without including self-attention. The results displayed in the figure demonstrate the greater predictive performance of the hybrid model, with the INNGNN model outperforming each separate module.

4.3.3. Ablation Studies

Ablation investigations were carried out to confirm the prediction performance of our proposed INNGNN model. The model for prediction used in the experiment was renamed INNGNN-INN after the INN part was removed, and INNGNN-GNN after the GNN part was removed for making predictions. This model compares the linear correlation between the INNGNN model and its corresponding predicted values. Among these three models, it can be observed from Figure 9 that the INNGNN model has the best fitting effect on the predicted value and the observed value, and the correlation coefficient $R^2$ is 0.91, which is higher than that of the INNGNN-INN model (the correlation coefficient $R^2$ is 0.88), indicating that the temporal correlation results in a significant improvement of the prediction results. At the same time, it is higher than the INNGNN-GNN model (the correlation coefficient $R^2$ is 0.88), indicating that the spatial correlation significantly improved the prediction results.

4.3.4. Display and Analysis

This section shows the AQP results for two cities from 1 April to 8 April using the INNGNN model. In Figure 10, the blue line represents the real value, and the red line represents the predicted value. Figure 10a represents the prediction effect for cities with low-scale fluctuations in air quality, and Figure 10b represents the prediction effect for cities with high-scale fluctuations in air quality. Our INNGNN model reacted promptly to time series fluctuations in both scenarios. The AQI increased significantly between 2 April and 3 April, as shown in Figure 10a, but the INNGNN model was still able to react promptly to the time series changes, demonstrating the model’s strong accuracy and efficient performance.

5. Conclusions

This study proposes a hybrid model, INNGNN, to forecast air quality by combining GNN and self-attention. The INN module is used to fully extract the time series characteristics in order to obtain the temporal dependence, and to give the periodicity and trend the interpretability needed for forecasting. These issues with periodicity and trend are often ignored in time series forecasting. Furthermore, the self-attention mechanism is incorporated in order to capture the time series’ local and global dependencies, as well as the varying relevance of each input feature at different times. In order to achieve the expected spatial dependencies, the combination of GNN models can determine the connection relationship between cities in order to capture the dependencies between them. When the INNGNN model was compared to previous models and assessed using real air quality datasets, it performed better over a range of prediction ranges. To further validate its performance, ablation experiments were conducted by comparing different modules of the model, and the INNGNN consistently exhibited superior performance.

In conclusion, the spatiotemporal aspects of urban air quality data are effectively captured by the INNGNN model, and this model can be expanded and used for other multivariate timeseries spatiotemporal applications. Environmental factors or specific dataset characteristics that were not considered in this study may influence the model’s performance. Additionally, the possibility of further investigating and improving the comprehensibility of the model’s forecasts exists. In subsequent studies, we want to investigate further how graph neural networks could be integrated with other time series models in order to increase the predictability and versatility of these models. We will also concentrate on enhancing the interpretability of the model and resolving any biases or limitations present in the dataset.