Next Article in Journal
Cryptanalysis and Improvement of a Chaotic Map-Based Image Encryption System Using Both Plaintext Related Permutation and Diffusion
Next Article in Special Issue
The Evolution Characteristics of Systemic Risk in China’s Stock Market Based on a Dynamic Complex Network
Previous Article in Journal
Entropic Dynamics in Neural Networks, the Renormalization Group and the Hamilton-Jacobi-Bellman Equation
Previous Article in Special Issue
Pricing Constraint and the Complexity of IPO Timing in the Stock Market: A Dynamic Game Analysis
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Systemic Importance of China’s Financial Institutions: A Jump Volatility Spillover Network Review

1
School of Mathematics and Statistics, Changsha University of Science and Technology, Changsha 410114, China
2
Hunan Provincial Key Laboratory of Mathematical Modeling and Analysis in Engineering, Changsha University of Science and Technology, Changsha 410114, China
3
School of Management, China Institute for Studies in Energy Policy, Xiamen University, Xiamen 361005, China
4
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China
*
Author to whom correspondence should be addressed.
Submission received: 18 April 2020 / Revised: 18 May 2020 / Accepted: 19 May 2020 / Published: 24 May 2020
(This article belongs to the Special Issue Complexity in Economic and Social Systems)

Abstract

:
The investigation of the systemic importance of financial institutions (SIFIs) has become a hot topic in the field of financial risk management. By making full use of 5-min high-frequency data, and with the help of the method of entropy weight technique for order preference by similarities to ideal solution (TOPSIS), this paper builds jump volatility spillover network of China’s financial institutions to measure the SIFIs. We find that: (i) state-owned depositories and large insurers display SIFIs according to the score of entropy weight TOPSIS; (ii) total connectedness of financial institution networks reveal that Industrial Bank, Ping An Bank and Pacific Securities play an important role when financial market is under pressure, especially during the subprime crisis, the European sovereign debt crisis and China’s stock market disaster; (iii) an interesting finding shows that some small financial institutions are also SIFIs during the financial crisis and cannot be ignored.

1. Introduction

With the development of economic globalization, the financial system has become more and more closely interconnected by investment networks, debtor–creditor and trade contacts [1,2,3,4]. Financial institutions such as depositories, broker-dealers and insurance companies permeate each other by related business and display significant complex network properties [5,6,7]. The failure of several financial institutions may lead to a severe economic crisis [8,9,10]. One of the typical examples is the global financial crisis triggered by the collapse of Lehman Brothers in 2008 [11,12,13]. Therefore, how to accurately evaluate the systemic importance of financial institutions (SIFIs) so as to provide early warning and deal with the crisis effectively has become an emergent work [14,15,16].
Usually, there are three ways to measure the SIFIs. The first way is to employ Pearson correlation coefficient to calculate the financial institutions’ default probabilities [17,18,19]. Pearson correlation coefficient ignores the heterogeneity of financial data at different times [20]. Adopting a tail-dependence method to measure the systemic risk contributions between financial institutions is the second method. Girardi and Ergün (2013) [21] used the conditional value-at-risk (CoVaR) method to estimate systemic risk of each financial institution. Acharya et al. (2017) [22] employed the systemic expected shortfall (SES) model to calculate financial institution’s losses by considering its leverage. Wang et al. (2019) [23] proposed CSRISK model to investigate financial institutions’ capital shortfall under the market crash. The above two methods are based on the local correlation and disregard the interlinked among the financial institutions, which may underrate systemic risk contribution [24]. The latest financial crisis manifests that intricate connections among financial markets can spread risk [25,26,27,28,29]. Using the complex network theory to research SIFIs comes up to the third method. Billio et al. (2012) and Gong et al. (2019) [30,31] applied Granger causality model to build financial institution network and utilized the out degree to compute the total connectedness. Hautsch et al. (2014) [24] proposed VaR model to set up financial institution network and adopted systemic risk betas to investigate the systemic risk contribution. Härdle et al. (2016) and Wang et al. (2018) [32,33] adopted CoVaR model to construct financial institution network and used the index of out degree to measure the system risk contribution.
Most of the literature evaluates the SIFIs by out degree [31,32,33], which can reveal the range of risk contagion but restrict to the local information of the network [34,35]. Recently, considering financial institutions have the characteristics of deeper risk contagion extent, higher risk contagion efficiency and greater risk contagion degree after the outbreak of financial risk, some other indicators such as, clustering coefficient [36], closeness centrality [37] and Leaderrank value [38] are also applied to measure the SIFIs. Although most of the above indicators have been investigated extensively and many findings on SIFIs have been reported recently, which only reflect one characteristic of the network [36,37,38]. A comprehensive evaluation with respect to the entire network appears to be very few. Such studies are however essential to accurately evaluate SIFIs in practice. To deal with this issue, combining four indicators (out degree [32], clustering coefficient [36], closeness centrality [37] and Leaderrank value [38]) and assigning different weights to each indicator may give a better evaluation. However, the selection of weight is often based on the subjective experience of researchers, rather than sufficient scientific support, which may lead to inaccurate evaluation results. As we all known, entropy weight technique for ioder preference by similarities to ideal solution (TOPSIS) is a multiple criteria decision making method, and it bases on the conception that the selected alternative should have the shortest distance from the positive ideal solution and the farthest distance from the negative ideal solution. Entropy weight TOPSIS has been proved to be a good method in strategic decision making and successfully applied in some fields, such as coal mine safety [39], multinational consumer electronics company [40] and transport [41]. Therefore, it seems that adopting entropy weight TOPSIS to comprehensively assess the SIFIs might be a better choice.
It should be pointed out that the all above mentioned literatures on measuring SIFIs have been greatly limited to low-frequency data. The low-frequency data with daily, weekly, monthly, quarterly or annual sampling frequency can not accurately measure the whole-day volatility information [42]. Nowadays, more and more scholars have realized that the high-frequency data with the frequency of hours, minutes or even shorter includes the rich information of asset price, and it has been intensively studied in applied finance risk management [43,44,45,46]. On the other hand, with the unexpected changes of macroeconomic conditions, international events and economic policy in recent years, financial markets are increasingly volatile [47]. Some researches detected jump volatility in the volatile process of financial assets based on high-frequency data [48]. For example, Wright and Zhou (2007) [49] found that jump volatility can explain much of the countercyclical movements in bond risk premium. Zhang et al. (2016) [50] found that jump volatility is an important component of Dow Jones Industrial Average stocks’ volatility. Audrino and Hu (2016) [51] found that jump volatility can improve the forecast of S&P 500’s volatility.
The jump volatility depicts an infrequent but a sharp change of asset price, and it can better describe violent volatility of financial market than continuous volatility [52]. Measuring SIFIs associated with jump volatility spillover network and high-frequency data has not been reported yet and it still remains a challenging problem. Motivated by the above discussions, in this paper, we aim to employ high-frequency data of China’s financial institutions to construct jump volatility spillover network, and then utilize entropy weight TOPSIS to comprehensively assess the SIFIs. The innovations of this paper are as follows:
(1)
Many scholars investigated the jump volatility of a single financial asset on its price fluctuation from the perspective of prediction. We first propose Granger-causality test to identify the jump volatility spillover among financial institutions.
(2)
Financial markets are extremely volatile, and the low-frequency data might lose a lot of important information. By employing 5-min high-frequency data, we establish the jump volatility spillover network, which can capture the jump volatility spillover among financial institutions.
(3)
We use entropy weight TOPSIS rather than a single indicator to comprehensively assess the SIFIs.
The reminder of this paper is arranged as follows. In Section 2, we introduce the methodology. In Section 3, we present the data. In Section 4, we give an empirical analysis. Finally, we make conclusions and discuss our findings in Section 5.

2. Methodology

In this section, we introduce the method of network construction and the indicator for assessing the SIFIs. Specifically, in Section 2.1, we use Granger causality test to build the network, which reflects statistically significant relations between jump volatility spillover of financial institutions. In Section 2.2, out degree, clustering coefficient, closeness centrality and leaderrank algorithm are employed to evaluate the SIFIs, respectively. In Section 2.3, by the method of entropy weight TOPSIS, we integrate the above four indicators into a comprehensive indicator to measure the SIFIs.

2.1. Network Construction

We establish jump volatility spillover network of financial institution according to the following three steps, where each financial institution represents a network node, and each pair of the financial institution is connected with an edge calculated by Granger-causality test.
In the first step, we employ Andersen et al. (2007, 2012) tests to extract jump volatility of financial institutions. We suppose that the logarithmic price of a financial institution ( p t = ln P t ) within the trading day obeys a standard jump diffusion process:
d p t = μ t d t + σ t d W t + κ t d q t , 0 t T ,
where μ t denotes the drift term, which includes a continuous volatility sample path; σ t represents a strictly positive stochastic volatility process; W t stands for a standard Brownian motion; κ t d q t is the pure jump component.
Meanwhile, the logarithmic return volatility can be expressed as quadratic volatility, which contains jump volatility rather than unbiased estimator of integrated volatility:
Q V t = t 1 t σ s 2 d s + t 1 < s t k s 2
where t 1 t σ s 2 represents the continuous volatility, and t 1 < s t k s 2 stands for intra-day jump volatility.
Since the quadratic volatility can not be gained directly, this paper employs the estimated realized volatility R V t to replace it based on Andersen et al. (2012) [53]:
R V t = i = 1 M r t , i 2 ,
where r t , i = ( ln P t , i ln P t 1 , i ) × 100 , P t , i denotes the closing price of financial institution i at time t, and M = 48 represents the daily trading frequency.
In addition, when M , t 1 t σ s 2 d s can be calculated by the realized bipower volatility M e d R V t based on Barndorff-Nielse et al. (2004) [54,55].
M e d R V t = π 6 4 3 π M M 2 i = 2 M 1 Med r t , i 1 | r t , i | | r r , i + 1 | 2 ,
If there is no jump in the price of financial institutions, the difference between realized volatility and bipower volatility is 0. Otherwise, Z-statistic is adopted to identify jump volatility [54]:
Z t = R V t M e d R V t R V t 1 μ 1 4 + 2 μ 1 2 5 1 M max 1 , m e d R T Q t m e d R V t 2 N ( 0 , 1 ) ,
where μ 1 = 2 / π , and med R T Q t = 3 π M 9 π + 72 52 3 M M 2 i = 2 M 1 Med r t , i 1 | r t , i | | r r , i + 1 | 4 , which stands for realized tri-power quarticity.
Based on Z statistics, we can obtain realized jump volatility:
J t d = I Z t > Φ t R V t M e d R V t ,
where I ( · ) is an indicator function, and α chooses as 0.95 (see Andersen et al., 2007 [56]).
In the second step, after extracting the jump volatility of a single financial institution, we investigate whether there is jump volatility spillover between financial institutions according to Granger-causality test [57]. If the p values of Granger-causality test are smaller than the critical values under the 5% significance level [58], there exists causality relationships between financial institutions.
In the last step, we construct a Granger-causality jump volatility spillover network of financial institutions. And the network can be represented by an adjacency matrix AD:
A D = ( V , E ) = 0 A D 1 j A D 1 n A D i 1 A D i j A D i n A D n 1 A D n j 0
where V is nodes set and E is the edge set. n is the number of financial institutions. A D i j is defined as follow:
A D i j = 1 i G r a n g e r c a u s e s j s i g n i f i c a n t l y 0 i d o e s n t G r a n g e r c a u s e s j

2.2. Indicator for Assessing the Systemic Importance of Financial Institutions

There are a growing number of indicators to evaluate SIFIs. Taking into account financial institutions have the characteristics of wider risk contagion range, deeper risk contagion extent, higher risk contagion efficiency and greater risk contagion degree, we choose out degree, clustering coefficient, closeness centrality and leaderrank value to assess the SIFIs, respectively. And more and more scholars use these four indicators to study the SIFIs [33,36,37,38].

2.2.1. Out Degree

Out degree ( O D ) calculates the number of edges that node i point to other nodes. It is used to measure the risk contagion range [36]. When the risk occurs, it will directly transfer the risk to the connected nodes. The higher the out degree of nodes, the wider the range of risk transmission. The expression of out degree is as follows:
O D o u t ( i ) = j = 1 n A D i j ,
where A D i j stands for the adjacency matrix of financial institution network.

2.2.2. Clustering Coefficient

The clustering coefficient (C) measures the degree of interconnection between the neighbors of a node in the graph. If one node owes high clustering coefficient, the risk may spread to their neighbor nodes when one financial institution fluctuates. Furthermore, the interconnectedness of neighbor nodes will cause risk contagion again and aggravate the risk contagion extent of the whole financial institutions [36]. Therefore, we employ clustering coefficient as the risk contagion extent of each financial institution, and it is computed as follows:
C i = m i / α i ( α i 1 ) ,
where α i ( α i 1 ) represents the maximum number of possible edges of financial institution i, and m i stands for the actual number of existing edges.

2.2.3. Closeness Centrality

Closeness centrality ( C C ) quantifies how close a node is to all other nodes in the financial institution network. The closeness centrality of a node is inversely proportional to the average shortest path distance from one node to any other nodes in the network. The larger value of the closeness centrality of a node, the faster the risk will be transferred from one node to any other nodes. Hence, the closeness centrality can depict how efficiently each node transmits risk to all other nodes [37], and it is expressed as follows:
C C o u t ( i ) = j = 1 , j i N 2 d i j ,
where d i j is the shortest distance i to j.

2.2.4. Leaderrank Algorithm

LeaderRank ( L R ) algorithm is a method to identify key nodes in a complex network. The basic idea of the algorithm is as follows. We add a new node (called ground node) and connect it to all others by bidirectional edges for a directed network with M nodes and N edges. The new network is strongly linked, which owes M + 1 nodes and N + 2 M edges. Matrix A = ( a i j ) depicts the connectivity of the network. If A D i j = 1 , which means that node i can pass financial risk to node j. The L R gives a score to each node, where score denotes the SIFIs. Scores are assign by L R g ( 0 ) = 0 for ground node and L R i ( 0 ) = 1 for other nodes. Thus, scores are updated by
L R i ( t ) = j = 1 M + 1 a j i O D o u t ( j ) L R i ( t 1 ) ,
where O D o u t ( j ) is out degree.
After t iterations, the L R values of all nodes are stable. At this time, the ground node score is averagely distributed to each network node. Consequently, the final score of the network node reflects its cumulative risk ability. The higher the score, the stronger the cumulative risk degree of the node [38].

2.3. Entropy Weight TOPSIS

Measuring risk contagion of financial institutions from different indicators may lead to inconsistent results. Therefore, the construction of risk contagion composite index of financial institutions is an essential step in this paper. We adopt entropy weight TOPSIS to evaluate SIFIs. It can avoid the subjectivity of weight selection and make full use of the sample data [39,40,41].
The entropy of each indicator is calculated as below:
e j = 1 ln N i = 1 N p i j ln p i j , p i j = x i j i = 1 N x i j 2 ,
where j = 1 , N ; i = 1 , n ; N = 24; n = 4, x i j denotes the jth indicator value of the ith financial institution of the initial matrix X; p i j stands for the jth normalized indicator value of the ith normalized matrix P.
The weight ( w j ) can be calculated as follows:
w j = 1 e j N i = 1 n e j , j = 1 , 2 , , N .
Then, the TOPSIS method ranks financial institutions based on their relative proximities, and the positive ideal solution and the negative ideal solution. The distance of each indicator from D i + and D i can be calculated as the following:
D i + = j = 1 N S i j S i + 2 , j = 1 , 2 , , N ,
D i = j = 1 N S i j S j 2 , j = 1 , 2 , , N ,
where S i j = w j p i j , S i + = max 1 j n S i j , and S i = min 1 j n S i j .
The relative proximity K i , which is regarded as score of each financial institution, can be computed as follows:
K i = D i / D i + D i + , j = 1 , 2 , , N .
Finally, in order to measure ability of risk contagion, we use K i to rank the SIFIs.

3. Data

We select 24 listed financial institutions from 2008 to 2018 in China, similar sample selection can be found in Wang et al. (2018) [33]. We choose 2008 as the starting date due to several important financial institutions do not go public until 2007, such as the China Construction Bank. We divide listed financial institutions into three sectors: (1) depositories, (2) insurance companies and (3) broker dealers. The data are available from Wind Financial dataset, and the descriptive statistics of 5-min high-frequency closing price data of financial institutions are shown in Table 1.

4. Empirical Analysis

4.1. Jump Volatility Spillover Network Construction of Financial Institution

The sampling frequency selection of intraday high-frequency data is very important for jump volatility measurement. The low sampling frequency may not fully express the jump volatility information, while high sampling frequency can cause micro structural noise. According to Haugom et al. (2014) [59], Gong and Lin (2018) [46] and Wen et al. (2019) [55], this paper first adopts 5-min high-frequency closing price data of financial institutions to compute jump volatility based on Equations (1)–(9). Then, using Granger causality test to investigate jump volatility spillover relations among financial institutions, we can obtain the p value of a 24 × 24 matrix. This paper chooses a threshold of 0.05 according to Jiang et al. (2017) [58]. Finally, the financial institution network gets a total of 24 nodes and 137 edges, and the results are illustrated by Figure 1.

4.2. Assessing the Systemic Importance of Financial Institutions

The research on the systemic importance of financial institutions (SIFIs) has become a hot topic in financial risk management. In this section, we choose out degree, clustering coefficient, closeness centrality and leaderrank value to measure the SIFIs. The results are shown in Table 2.
Table 2 displays four dimensions of risk contagion measurement. (1) In terms of risk contagion range, the larger the out degree value, the wider risk contagion range of financial institution. We can find that CMB, BOC and CNCB are all from depository sector, with the highest out degree value of 10. One possible reason is that China’s financial system is a depository-led system. As claimed by the annual reports of CBRC, CSRC, and CIRC in 2019, the total assets of depository, broke-dealer and insurance sectors were 261.4 trillion Yuan, 6.2 trillion Yuan and 18.3 trillion Yuan, respectively. This indicates that the depository sector size is 42 times larger than the broke-dealer sector, or 14 times larger than the insurance sector. (2) In terms of risk contagion extent, the larger the clustering coefficient value, the greater risk contagion extent of financial institution. We can see that SLSC and HTSEC, which are all from broke-dealer sector, have the highest clustering coefficient value. The results show that broke-dealer sector’s risk contagion ability can not be neglected in China’s financial system. (3) In terms of risk contagion efficiency, the larger the closeness centrality value, the faster the risk will be transferred from one financial institution to any other financial institutions. We can discover that CJSC and PAI have the highest closeness centrality value, indicating that some broke-dealers and insurances have gradually become important departments in China’s banking-led financial system. (4) In terms of risk contagion degree, the larger the leaderrank value, the higher the risk contagion degree. The risk contagion degree of financial institutions manifests a hierarchical feature, i.e., the greatest degree of risk contagion is insurance sector, followed by broke-dealer sector, and depository sector have the lowest risk contagion degree.
The results of the above four dimensions are inconsistent in measuring the SIFIs. In order to comprehensively assess the SIFIs, this paper proposes entropy weight TOPSIS (EWTOPSIS) to obtain the weight of each indicator. As a result, we gain the weight of out degree, clustering coefficient, closeness centrality and leaderrank algorithm by 0.2807, 0.2499, 0.0401 and 0.4293, respectively.
Table 3 shows the score of SIFIs computed by EWTOPSIS in the whole period. We can find that: (1) CLI is the most SIFIs. This may be related to the deregulation reform of the China’s insurance sector in 2014. China’s state council issue that the insurance depth will reach 5% and the insurance density will reach 3500 yuan/person by 2020. (2) CCB, BOB, ICBC and BOCOM are deem as more systemically important financial institutions. Because most of them come from state-owned depositories, which dominate China’s depository sector about 45% of the lending business in 2018. (3) CJSC and NJBK have a relatively low score, which imply that the impact of some small financial institutions on the financial system can be neglected.
Compared with the traditional evaluation SIFIs methods such as Equal weight [38], principal component analysis (PCA) and TOPSIS method [60], we will show our proposed EWTOPSIS method has obvious advantages. Just as reported by Sandoval (2014) [61], Wang et al. (2018) [33] and Wang et al. (2019) [62], market capitalization as a financial indicator could reflect the market influence of financial institutions. We calculate the correlation between each index (depending on different methods) and market capitalization, and the results can be presented as Table 4. It is easy to see that the EWTOPSIS is most effective method to measure the SIFIs because of the correlation between EWTOPSIS and market capitalization is the largest at a significant level of 1%.

4.3. Assessing the Dynamic Systemic Importance of Financial Institutions

As we all know, the financial markets are complex dynamic systems. The jump volatility spillover between financial institutions is time-varying. Thus, we employ time-varying Granger causality test to build dynamic jump volatility spillover of financial institution networks [63]. 2677 financial institution networks are obtained.
Figure 2 exhibits the evolution of the number of total linkages as a percentage of all possible linkages (TP). We can discover that it has three prominent cycles based on high TP values. The first cycle started at January 2008 and ended until April 2008, which was in the period of the subprime crisis. During this time, the CSI 300 Index decreased by 661 points (almost 17%) from 4620 to 3959. Then, TP value followed by a quickly descending trend until June 2008. The second cycle started at June 2008 and ended at October 2008, which was in the later period of the subprime crisis, and in the initial period of the European sovereign debt crisis. During this period, the CSI 300 Index dropped to 1948 points (almost 54%) from 3611 to 1663. Thereafter, the TP value entered a stationary period from October 2009 to June 2015. The Third cycle began at May 2015 and ended at August 2015, which was in the period of China’s stock market disaster. At this stage, the CSI 300 Index declined by 1474 points (nearly 31%) from 4840 to 3366. Henceforth, the TP value entered another stationary period.
Furthermore, we employ the index of out degree, clustering coefficient, closeness centrality and leaderrank algorithm to measure the risk contagion range, risk contagion extent, risk contagion efficiency and risk contagion degree in each period, respectively. Then, entropy weight TOPSIS method is adopted to compute the score of SIFIs. We list the top 10 of financial institutions with systematically important score and market capitalization (MC) corresponding to the highest TP in three cycles, and the results are shown in Table 5, Table 6 and Table 7.
Table 5, Table 6 and Table 7 display the top 10 financial institutions ranked by the systematically important score on 25 April 2008, 23 September 2008 and 10 July 2015. We can find that CIB, PAB and PSC are included in the three periods, indicating that large commercial banks and insurances play an important role during the financial crisis. Moreover, the top five largest financial institutions ranked by market capitalization are not all included in the table, and some small financial institutions are also systemically important. It means that the SIFIs in network may be “too big to fail” or “too interconnected to fail”.

5. Conclusions and Discussion

This paper adopts 5-min high-frequency data of China’s financial institutions to extract realized jump volatility. Then, we employ Granger-casuality test to construct the jump volatility spillover network. Furthermore, out degree, clustering coefficient, closeness centrality and leaderrank value are used to evaluate the SIFIs, respectively. In addition, we utilize entropy weight TOPSIS to comprehensively evaluate the SIFIs.
Some basic results of our research can be summed up as follows: (1) The highest frequency of jump volatility is 44.30% in 2008. This may be related to the outbreak of the subprime mortgage crisis in 2008. (2) We measure the SIFIs from four dimensions. In terms of risk contagion range, we can find that CMB, BOC and CNCB, which are all from depository sector, possess the highest out degree value of 10. One possible explanation is that China’s financial system is a depository-led system. In terms of risk contagion extent, one can see that SLSC and HTSEC, which are all from broke-dealer sector, have the highest clustering coefficient value. This indicates that broke-dealer sector’s risk contagion ability can not be neglected in China’s financial system. In terms of risk contagion efficiency, we discover that CJSC and PAI have the highest closeness centrality value, which means that some insurances have gradually become important departments. In terms of risk contagion degree, the results reveal that the greatest degree of risk contagion is insurance sector, followed by broke-dealer sector, and depository sector have the worst risk contagion degree. (3) Based on the comprehensive evaluation of the SIFIs, by the method of entropy weight TOPSIS, the obtained results show that CLI, CCB, BOB, ICBC and BOCOM are identified as the influential nodes. (4) According to highest values of total linkages in each period, we can find three prominent cycles. The first cycle started at January 2008 and ended at April 2008, which was in the period of the subprime crisis. The second cycle started at June 2008 and ended at October 2008, which was in the later period of the subprime crisis, and in the initial period of the European sovereign debt crisis. The third cycle began at May 2015 and ended at August 2015, which was in the period of China’s stock market disaster. (5) Total connectedness of financial institution networks reveal that large commercial banks and insurances play an important role when financial market is under pressure, especially during the subprime crisis, the European sovereign debt crisis and China’s stock market disaster. Meanwhile, some small financial institutions are also systemic importance, which may be related to their too much interconnection with other financial institutions.
By the way, the work presented in this article does not consider the following points: (1) The data do not contain all publicly listed financial institutions in China because we have deleted those for which we have experienced long suspension periods. Therefore, developing new tools that have limited sample for investigating SIFIs is a worthy target. (2) We just select 24 top financial institutions. Non-financial institutions may also play an important role as a result of their interactions with these financial ones. It would be interesting to research some financial institutions and non-financial institutions at the same time. We will leave this challenging yet interesting problem as future research. (3) It is important to measure SIFIs based on the network. This paper employs linear Granger-casuality test to construct the jump volatility spillover network. There is a nonlinear relationship between financial markets. The method of network construction could be extended by employing the Granger-casuality test. We have to leave these challenging yet interesting problems as future research.

Author Contributions

Conceptualization, C.H.; Data curation, C.H.; Formal analysis, X.Z., X.G. and X.Y. (Xiaoguang Yang); Funding acquisition, X.G.; Investigation, X.Y. (Xin Yang), X.Y. (Xiaoguang Yang) and C.H.; Methodology, X.Y. (Xin Yang) and X.Z.; Project administration, X.Y. (Xin Yang) and C.H.; Resources, X.G.; Software, X.Y. (Xin Yang) and X.G.; Supervision, X.Y. (Xiaoguang Yang); Validation, X.Z.; Visualization, X.Z. and X.G.; Writing—original draft, X.Y. (Xin Yang); Writing—review & editing, X.Y. (Xin Yang) and C.H. All authors have read and agreed to the published version of the manuscript.

Funding

Partially supported by the National Natural Science Foundation of P. R. China (Nos. 11971076, 51839002, 71701176), Hunan Provincial Natural Science Foundation (No. 2019JJ50650), Scientific Research Fund of Hunan Provincial Education Department (No. 18C0221),Research Promotion Program of Changsha University of Science and Technology (No. 2019QJCZ050).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lentz, H.H.K.; Konschake, M.; Teske, K.; Kasper, M.; Rother, B.; Carmanns, R.; Petersen, B.; Conraths, F.J.; Selhorst, T. Trade communities and their spatial patterns in the German pork production network. Prev. Vet. Med. 2011, 98, 176–181. [Google Scholar] [CrossRef]
  2. Bowden, R.J. Directional entropy and tail uncertainty, with applications to financial hazard. Quant. Financ. 2011, 11, 437–446. [Google Scholar] [CrossRef]
  3. Dimpfl, T.; Peter, F.J. Using transfer entropy to measure information flows between financial markets. Stud. Nonlinear Dyn. Econom. 2013, 17, 85–102. [Google Scholar] [CrossRef] [Green Version]
  4. Stutzer, M.J. The Role of Entropy in Estimating Financial Network Default Impact. Entropy 2018, 20, 369. [Google Scholar] [CrossRef] [Green Version]
  5. Gencay, R.; Signori, D.; Xue, Y.; Yu, X.; Zhang, K.Y. Economic links and credit spreads. J. Bank. Financ. 2015, 55, 157–169. [Google Scholar] [CrossRef]
  6. Li, C.; Wang, L.; Sun, S.W.; Xia, C.Y. Identification of influential spreaders based on classified neighbors in real-world complex networks. Appl. Math. Comput. 2018, 320, 512–523. [Google Scholar] [CrossRef] [Green Version]
  7. Wen, S.; Tan, Y.; Li, M.; Deng, Y.; Huang, C. Analysis of global remittance based on complex networks. Front. Phys. 2020, 8, 85. [Google Scholar] [CrossRef]
  8. Acharya, V.V.; Richardson, M. Causes of the financial crisis. Crit. Rev. 2009, 21, 195–210. [Google Scholar] [CrossRef]
  9. Braouezec, Y.; Wagalath, L. Risk-Based Capital Requirements and Optimal Liquidation in a Stress Scenario. Rev. Financ. 2018, 22, 747–782. [Google Scholar] [CrossRef]
  10. Braouezec, Y.; Wagalath, L. Strategic fire-sales and price-mediated contagion in the banking system. Eur. J. Oper. Res. 2019, 274, 1180–1197. [Google Scholar] [CrossRef]
  11. De Haas, R.; Van Horen, N. International shock transmission after the Lehman Brothers collapse: Evidence from syndicated lending. Am. Econ. Rev. 2012, 102, 231–237. [Google Scholar] [CrossRef] [Green Version]
  12. Gradojevic, N.; Caric, M. Predicting Systemic Risk with Entropic Indicators. J. Forecast. 2017, 36, 16–25. [Google Scholar] [CrossRef] [Green Version]
  13. Erdemlioglu, D.; Gradojevic, N. Heterogeneous investment horizons, risk regimes, and realized jumps. Int. J. Financ. Econ. 2020, in press. [Google Scholar] [CrossRef]
  14. Yang, X.; Wen, S.G.; Liu, Z.F.; Li, C.; Huang, C.X. Dynamic Properties of Foreign Exchange Complex Network. Mathematics 2019, 7, 832. [Google Scholar] [CrossRef] [Green Version]
  15. Yang, X.; Wen, S.G.; Zhao, X.; Huang, C.X. Systemic importance of financial institutions: A complex network perspective. Phys. A Stat. Mech. Appl. 2020, 545, 123448. [Google Scholar] [CrossRef]
  16. Huang, C.X.; Wen, S.G.; Li, M.G.; Wen, F.H.; Yang, X. An empirical evaluation of the influential nodes for stock market network: Chinese A-shares case. Financ. Res. Lett. 2020, 101517. [Google Scholar] [CrossRef]
  17. Huang, X.; Zhou, H.; Zhu, H.B. A framework for assessing the systemic risk of major financial institutions. J. Bank. Financ. 2009, 33, 2036–2049. [Google Scholar] [CrossRef] [Green Version]
  18. Valahzaghard, M.; Bahrami, M. Prediction of default probability in banking industry using CAMELS index: A case study of Iranian banks. Manag. Sci. Lett. 2013, 3, 1113–1118. [Google Scholar] [CrossRef]
  19. Derbali, A.; Jamel, L. Dependence of Default Probability and Recovery Rate in Structural Credit Risk Models: Case of Greek Banks. J. Knowl. Econ. 2018, 10, 711–733. [Google Scholar] [CrossRef] [Green Version]
  20. Wen, F.H.; Yang, X.; Zhou, W.X. Tail dependence networks of global stock markets. Int. J. Financ. Econ. 2019, 24, 558–567. [Google Scholar] [CrossRef] [Green Version]
  21. Girardi, G.; Ergün, A.T. Systemic risk measurement: Multivariate GARCH estimation of CoVaR. J. Bank. Financ. 2013, 37, 3169–3180. [Google Scholar] [CrossRef]
  22. Acharya, V.V.; Pedersen, L.H.; Philippon, T.; Richardson, M. Measuring systemic risk. Rev. Financ. Stud. 2017, 30, 2–47. [Google Scholar] [CrossRef]
  23. Wang, J.N.; Hsu, Y.T.; Lee, J.M.; Chen, C.C. Measuring Systemic Risk: Capital Shortfall and CSRISK. Int. Rev. Financ. 2019, 1–11. [Google Scholar] [CrossRef]
  24. Hautsch, N.; Schaumburg, J.; Schienle, M. Financial network systemic risk contributions. Rev. Financ. 2014, 19, 685–738. [Google Scholar] [CrossRef] [Green Version]
  25. Diebold, F.X.; Yilmaz, K. On the network topology of variance decompositions: Measuring the connectedness of financial firms. J. Econom. 2014, 182, 119–134. [Google Scholar] [CrossRef] [Green Version]
  26. Levy-Carciente, S.; Kenett, D.Y.; Avakian, A.; Stanley, H.E.; Havlin, S. Dynamical macroprudential stress testing using network theory. J. Bank. Financ. 2015, 59, 164–181. [Google Scholar] [CrossRef]
  27. Dai, Z.F.; Zhou, H.T.; Wen, F.H.; He, S.Y. Efficient predictability of stock return volatility: The role of stock market implied volatility. N. Am. J. Econ. Financ. 2020, 52, 101174. [Google Scholar] [CrossRef]
  28. Dai, Z.F.; Zhu, H. Stock return predictability from a mixed model perspective. Pac. Basin Financ. J. 2020, 60, 101267. [Google Scholar] [CrossRef]
  29. Dai, Z.F.; Dong, X.D.; Kang, J.; Hong, L.Y. Forecasting stock market returns: New technical indicators and two-step economic constraint method. N. Am. J. Econ. Financ. 2020, 53, 101216. [Google Scholar] [CrossRef]
  30. Billio, M.; Getmansky, M.; Lo, A.W.; Pelizzon, L. Econometric measures of connectedness and systemic risk in the finance and insurance sectors. J. Financ. Econ. 2012, 104, 535–559. [Google Scholar] [CrossRef]
  31. Gong, X.L.; Liu, X.H.; Xiong, X.; Zhang, W. Financial systemic risk measurement based on causal network connectedness analysis. Int. Rev. Econ. Financ. 2019, 64, 290–307. [Google Scholar] [CrossRef]
  32. Härdle, W.K.; Wang, W.N.; Yu, L.N. TENET: Tail-Event driven NETwork risk. J. Econom. 2016, 192, 499–513. [Google Scholar] [CrossRef]
  33. Wang, G.J.; Jiang, Z.Q.; Lin, M.; Xie, C.; Stanley, H.E. Interconnectedness and systemic risk of China’s financial institutions. Emerg. Mark. Rev. 2018, 35, 1–18. [Google Scholar] [CrossRef]
  34. Li, P.; Wang, B.H. Extracting hidden fluctuation patterns of Hang Seng stock index from network topologies. Phys. A Stat. Mech. Appl. 2007, 378, 519–526. [Google Scholar] [CrossRef]
  35. Zhang, Q.; Li, M.Z.; Deng, Y. Measure the structure similarity of nodes in complex networks based on relative entropy. Phys. A Stat. Mech. Appl. 2018, 491, 749–763. [Google Scholar] [CrossRef]
  36. Sun, Q.R.; Gao, X.Y.; Wen, S.B.; Chen, Z.H.; Hao, X.Q. The transmission of fluctuation among price indices based on Granger causality network. Phys. A Stat. Mech. Appl. 2018, 506, 36–49. [Google Scholar] [CrossRef]
  37. Ji, Q.; Bouri, E.; Roubaud, D. Dynamic network of implied volatility transmission among US equities, strategic commodities, and BRICS equities. Int. Rev. Financ. Anal. 2018, 57, 1–12. [Google Scholar] [CrossRef]
  38. Deng, X.R.; Cao, H. Systematic Risk, network contagion and sssessment of SIFIs. J. Cent. Univ. Financ. Econ. 2016, 3, 52–60. (In Chinese) [Google Scholar]
  39. Li, X.X.; Wang, K.S.; Liu, L.W.; Xin, J.; Yang, H.R.; Gao, C.Y. Application of the entropy weight and TOPSIS method in safety evaluation of coal mines. Procedia Eng. 2011, 26, 2085–2091. [Google Scholar] [CrossRef] [Green Version]
  40. Onar, S.C.; Oztaysi, B.; Kahraman, C. Strategic Decision selection using hesitant fuzzy TOPSIS and interval type-2 fuzzy AHP: A case study. Int. J. Comput. Intell. Syst. 2014, 7, 1002–1021. [Google Scholar] [CrossRef] [Green Version]
  41. Kim, A.R. A Study on competitiveness analysis of ports in Korea and China by entropy weight TOPSIS. Asian J. Shipp. Logist. 2016, 32, 187–194. [Google Scholar] [CrossRef]
  42. Yang, R.J.; Yu, L.; Zhao, Y.J.; Yu, H.X.; Xu, G.P.; Wu, Y.T.; Liu, Z.K. Big data analytics for financial Market volatility forecast based on support vector machine. Int. J. Inf. Manag. 2020, 50, 452–462. [Google Scholar] [CrossRef]
  43. Kitamura, Y.; Stutzer, M.J. Connections Between Entropic and Linear Projections in Asset Pricing Estimation. J. Econom. 2002, 107, 159–174. [Google Scholar] [CrossRef]
  44. Liu, A.Q.; Paddrik, M.E.; Yang, S.Y.; Zhang, X.J. Interbank Contagion: An Agent-Based Model Approach to Endogenously Formed Networks. J. Bank. Financ. 2017, 112, 105191. [Google Scholar] [CrossRef] [Green Version]
  45. Gong, X.; Wen, F.H.; Xia, X.H.; Huang, J.B.; Pan, B. Investigating the risk-return trade-off for crude oil futures using high-frequency data. Appl. Energy 2017, 196, 152–161. [Google Scholar] [CrossRef]
  46. Gong, X.; Lin, B.Q. The incremental information content of investor fear gauge for volatility forecasting in the crude oil futures market. Energy Econ. 2018, 74, 370–386. [Google Scholar] [CrossRef]
  47. Zhao, S. Spatial restructuring of financial centers in mainland China and Hong Kong: A geography of finance perspective. Urban Aff. Rev. 2003, 38, 535–571. [Google Scholar] [CrossRef]
  48. Bajgrowicz, P.; Scaillet, O.; Treccani, A. Jumps in high-frequency data: Spurious detections, dynamics, and news. Manag. Sci. 2016, 62, 2198–2217. [Google Scholar] [CrossRef] [Green Version]
  49. Wright, J.H.; Zhou, H. Bond Risk Premia and Realized Jump Volatility. Soc. Sci. Res. Netw. 2007, 2007, 1–37. [Google Scholar] [CrossRef]
  50. Zhang, X.; Kim, D.; Wang, Y.Z. Jump Variation Estimation with Noisy High Frequency Financial Data via Wavelets. Econometrics 2016, 4, 34. [Google Scholar] [CrossRef] [Green Version]
  51. Audrino, F.; Hu, Y.J. Volatility Forecasting: Downside Risk, Jumps and Leverage Effect. Econometrics 2016, 4, 8. [Google Scholar] [CrossRef] [Green Version]
  52. Barndorff-Nielsen, O.E.; Shephard, N. Realized power variation and stochastic volatility models. Bernoulli 2003, 9, 243–265. [Google Scholar] [CrossRef]
  53. Andersen, T.G.; Dobrev, D.; Schaumburg, E. Jump-robust volatility estimation using nearest neighbor truncation. J. Econom. 2012, 169, 75–93. [Google Scholar] [CrossRef] [Green Version]
  54. Barndorff-Nielsen, O.E.; Shephard, N. Power and bipower variation with stochastic volatility and jumps. J. Financ. Econom. 2004, 2, 1–37. [Google Scholar] [CrossRef] [Green Version]
  55. Wen, F.H.; Zhao, Y.P.; Zhang, M.Z.; Hu, C.Y. Forecasting realized volatility of crude oil futures with equity market uncertainty. Appl. Econ. 2019, 51, 6411–6427. [Google Scholar] [CrossRef]
  56. Andersen, T.G.; Bollerslev, T.; Diebold, F.X. Roughing it up: Including jump components in the measurement, modeling, and forecasting of return volatility. Rev. Econ. Stat. 2007, 89, 701–720. [Google Scholar] [CrossRef]
  57. Zheng, Q.H.; Song, L.R. Dynamic Contagion of Systemic Risks on Global Main Equity Markets Based on Granger Causality Networks. Discret. Dyn. Nat. Soc. 2018, 2018, 1–13. [Google Scholar] [CrossRef] [Green Version]
  58. Jiang, M.; Gao, X.; An, H.; Li, H.; Sun, B. Reconstructing complex network for characterizing the time-varying causality evolution behavior of multivariate time series. Sci. Rep. 2017, 7, 10486. [Google Scholar] [CrossRef] [Green Version]
  59. Haugom, E.; Langeland, H.; Molnár, P.; Westgaard, S. Forecasting volatility of the US oil market. J. Bank. Financ. 2014, 47, 1–14. [Google Scholar] [CrossRef] [Green Version]
  60. Du, Y.X.; Gao, C.; Hu, Y.; Mahadevan, S.; Deng, Y. A new method of identifying influential nodes in complex networks based on TOPSIS. Phys. A Stat. Mech. Appl. 2014, 399, 57–69. [Google Scholar] [CrossRef]
  61. Sandoval, L. Structure of a global network of financial companies based on transfer entropy. Entropy 2014, 16, 4443–4482. [Google Scholar] [CrossRef]
  62. Wang, Z.; Gao, X.Y.; An, H.Z.; Tang, R.W.; Sun, Q.R. Identifying influential energy stocks based on spillover network. Int. Rev. Financ. Anal. 2019. [Google Scholar] [CrossRef]
  63. Lu, F.B.; Hong, Y.M.; Wang, S.Y.; Lai, K.K.; Liu, J. Time-varying Granger causality tests for applications in global crude oil markets. Energy Econ. 2014, 42, 289–298. [Google Scholar] [CrossRef]
Figure 1. Jump volatility spillover network of financial institution. Note: Nodes (financial institutions) from the same sector are signed as the same shape and color. Depositories, broker dealer, and insurances are labelled as red circle, blue square, and green triangle, respectively.
Figure 1. Jump volatility spillover network of financial institution. Note: Nodes (financial institutions) from the same sector are signed as the same shape and color. Depositories, broker dealer, and insurances are labelled as red circle, blue square, and green triangle, respectively.
Entropy 22 00588 g001
Figure 2. The number of total linkages of the jump volatility spillover network as a percentage of all possible linkages over time.
Figure 2. The number of total linkages of the jump volatility spillover network as a percentage of all possible linkages over time.
Entropy 22 00588 g002
Table 1. The descriptive statistics of 5-min high-frequency closing price data of 24 financial institutions.
Table 1. The descriptive statistics of 5-min high-frequency closing price data of 24 financial institutions.
CodeFinancial InstitutionAbbreviationMeanSDMAXMIN
Panel A: Depositories
S000001.SZPing An BankPAB14.775.3844.458.03
S002142.SZBank of NingboNBCB13.003.5824.275.87
S600000.SHShanghai Pudong Development BankSPDB14.736.9161.807.11
S600015.SHHuaxia BankHXB10.372.1423.526.14
S600016.SHChina Minsheng Banking Corp., Ltd.CMBC7.561.8516.383.89
S600036.SHChina Merchants BankCMB17.216.5743.589.42
S601009.SHBank of NanjingNJBK11.243.5523.496.41
S601166.SHIndustrial BankCIB18.948.4761.798.62
S601169.SHBank of BeijingBOB10.243.1922.575.53
S601328.SHBank of CommunicationsBOCOM6.011.7616.153.60
S601398.SHIndustrial and Commercial Bank of China Ltd.ICBC4.610.838.353.15
S601939.SHChina Construction BankCCB5.391.1710.183.50
S601988.SHBank of ChinaBOC3.530.696.932.44
S601998.SHChina CITIC BankCNCB5.601.2910.933.39
Panel B: Broker-dealers
S000686.SZNortheast SecuritiesNESC17.828.7856.335.10
S000728.SZGuoyuan SecuritiesGYSC15.367.0045.705.55
S000783.SZChangjiang SecuritiesCJSC11.504.7840.154.07
S600030.SHCITIC SecuritiesCITICS19.0510.5798.079.10
S600109.SHSinolink SecuritiesSLSC17.839.2871.665.77
S600837.SHHaitong SecuritiesHTSEC14.177.2561.587.06
S601099.SHPacific SecuritiesPSC9.666.3047.451.94
Panel C: Insurances
S601318.SHPing An Insurance (Group) Co. of China, Ltd.PAI47.4714.37112.5519.90
S601601.SHChina Pacific Insurance (Group) Co., Ltd.CPIC24.727.0950.2510.36
S601628.SHChina Life Insurance (Group) Co., Ltd.CLI23.246.6560.6012.89
Note: SZ denotes that financial institution is transacted by the Shenzhen stock exchange market, and SH means that financial institution is transacted by the Shanghai stock exchange market.
Table 2. Indicators score of 24 financial institutions.
Table 2. Indicators score of 24 financial institutions.
SymbolODCCCLR
PAB21.00000.26020.0200
NBCB80.83930.16670.0120
SPDB80.56000.16840.0389
HXB90.68890.16330.0151
CMBC80.76320.17390.0645
CMB100.65740.15090.0157
NJBK10.00000.28830.0120
CIB11.00000.29090.0129
BOB90.56880.17390.0806
BOCOM80.60810.19280.0747
ICBC90.58540.16330.0786
CCB80.51120.18820.0930
BOC100.61360.15380.0403
CNCB100.66670.14950.0120
NESC11.50000.29910.0157
GYSC60.84290.19750.0302
CJSC10.00000.31070.0271
CITICS30.83330.22220.0157
SLSC22.00000.25200.0120
HTSEC22.00000.25200.0120
PSC41.08330.21330.0120
PAI10.52780.31070.0764
CPIC70.66850.19750.0720
CLI90.33820.16840.1563
Table 3. The score of systematic importance of financial institutions.
Table 3. The score of systematic importance of financial institutions.
RankSymbolScoreRankSymbolScore
1CLI0.748913GYSC0.2846
2CCB0.516014CMB0.2829
3BOB0.496215CNCB0.2703
4ICBC0.489816HXB0.2658
5BOCOM0.460517NBCB0.2539
6CPIC0.438518NESC0.2239
7CMBC0.440919PSC0.2088
8BOC0.369720PAB0.1914
9PAI0.324421CITICS0.1701
10SPDB0.317922CIB0.1472
11SLSC0.292023CJSC0.0705
11HTSEC0.292024NJBK0.0123
Table 4. The correlation between the market capitalization and four indicators.
Table 4. The correlation between the market capitalization and four indicators.
IndexEqual WeightPCATOPSISEWTOPSIS
Correlation coefficients0.33550.5193 ***0.4871 ***0.5647 ***
Note: *** denotes significant at 1%.
Table 5. Top 10 financial institutions ranked by the score of SIFIs on 25 April 2008.
Table 5. Top 10 financial institutions ranked by the score of SIFIs on 25 April 2008.
RankSymbolScoreRank of MC
1PSC0.908718 (46,722,978,887)
2CIB0.868511 (199,800,000,000)
3NBCB0.860320 (36,175,000,000)
4NESC0.855324 (18,917,836,544)
5HTSEC0.838112 (176,610,181,629)
6ICBC0.83611 (2,167,782,336,669)
7BOCOM0.83365 (522,280,130,274)
8PAB0.821916 (64,100,729,703)
9GYSC0.797719 (42,005,029,000)
10SLSC0.778723 (22,425,428,420)
Note: This table lists the market capitalization (MC) and its corresponding rank of top 10 financial institutions.
Table 6. Top 10 financial institutions ranked by the score of SIFIs on 23 September 2008.
Table 6. Top 10 financial institutions ranked by the score of SIFIs on 23 September 2008.
RankSymbolScoreRank of MC
1CIB0.957914 (79,900,000,000)
2CMB0.92307 (251,933,891,562)
3PSC0.922918 (27,435,468,619)
4NBCB0.910021 (18,750,000,000)
5PAB0.871816 (36,046,919,598)
6ICBC0.81231 (1,452,981,997,613)
7CCB0.81222 (1,088,991,131,440)
8CITICS0.801410 (133,603,922,140)
9PAI0.79966 (261,557,349,223)
10BOCOM0.79045 (292,496,470,707)
Table 7. Top 10 financial institutions ranked by the score of SIFIs on 10 July 2015.
Table 7. Top 10 financial institutions ranked by the score of SIFIs on 10 July 2015.
RankSymbolScoreRank of MC
1HTSEC0.908114 (256,372,893,000)
2PAB0.874115 (212,626,927,426)
3CJSC0.873119 (62,553,148,673)
4CIB0.845110 (336,845,313,758)
5PSC0.815023 (40,635,675,469)
6HXB0.796116 (143,827,801,960)
7CMB0.78867 (478,924,867,963)
8SPDB0.787812 (322,331,986,051)
9CITICS0.784511 (335,880,700,848)
10CLI0.75224 (1,019,790,556,400)

Share and Cite

MDPI and ACS Style

Yang, X.; Zhao, X.; Gong, X.; Yang, X.; Huang, C. Systemic Importance of China’s Financial Institutions: A Jump Volatility Spillover Network Review. Entropy 2020, 22, 588. https://0-doi-org.brum.beds.ac.uk/10.3390/e22050588

AMA Style

Yang X, Zhao X, Gong X, Yang X, Huang C. Systemic Importance of China’s Financial Institutions: A Jump Volatility Spillover Network Review. Entropy. 2020; 22(5):588. https://0-doi-org.brum.beds.ac.uk/10.3390/e22050588

Chicago/Turabian Style

Yang, Xin, Xian Zhao, Xu Gong, Xiaoguang Yang, and Chuangxia Huang. 2020. "Systemic Importance of China’s Financial Institutions: A Jump Volatility Spillover Network Review" Entropy 22, no. 5: 588. https://0-doi-org.brum.beds.ac.uk/10.3390/e22050588

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop