Next Article in Journal
A Conditional Approach to Panel Data Models with Common Shocks
Next Article in Special Issue
A Method for Measuring Treatment Effects on the Treated without Randomization
Previous Article in Journal
How Credible Are Shrinking Wage Elasticities of Married Women Labour Supply?
Previous Article in Special Issue
Forecasting Interest Rates Using Geostatistical Techniques
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Forecasting Value-at-Risk under Different Distributional Assumptions

1
Center for Operations Research and Econometrics (CORE), Université catholique de Louvain, 34 Voie du Romans Pays, B-1348 Louvain-la-Neuve, Belgium
2
Center for Research in Finance and Management (CeReFiM), Université de Namur, 62 Rue de Bruxelles, B-5000 Namur, Belgium
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Submission received: 15 July 2015 / Revised: 16 December 2015 / Accepted: 21 December 2015 / Published: 11 January 2016
(This article belongs to the Special Issue Recent Developments of Financial Econometrics)

Abstract

:
Financial asset returns are known to be conditionally heteroskedastic and generally non-normally distributed, fat-tailed and often skewed. These features must be taken into account to produce accurate forecasts of Value-at-Risk (VaR). We provide a comprehensive look at the problem by considering the impact that different distributional assumptions have on the accuracy of both univariate and multivariate GARCH models in out-of-sample VaR prediction. The set of analyzed distributions comprises the normal, Student, Multivariate Exponential Power and their corresponding skewed counterparts. The accuracy of the VaR forecasts is assessed by implementing standard statistical backtesting procedures used to rank the different specifications. The results show the importance of allowing for heavy-tails and skewness in the distributional assumption with the skew-Student outperforming the others across all tests and confidence levels.
JEL classification:
C01; C22; C52; C58

1. Introduction

Value-at-Risk (VaR) is a quantitative tool used to measure the maximum potential loss in value of a portfolio of assets over a defined period for a given probability. Specifically, VaR construction requires a quantile estimate of the far-left tail of the unconditional returns distribution. Though widely-used as a risk measure in the past, standard methods of VaR construction assuming iid-ness and normality have come under criticism due to their failure to incorporate three stylized facts of financial returns, namely ( i ) the presence of volatility clustering, indicated by high autocorrelation of absolute and squared returns, ( i i ) excess kurtosis (fat tails) and ( i i i ) skewness in the density of the unconditional returns distribution.
The ability to account for volatility clustering is one of the key strengths of the ARCH modelling approach developed in Engle [1] and extended in Bollerslev [2]. By combining this approach with a non-normal conditional distribution assumption for the returns, several papers have shown that univariate GARCH models can produce reliable out-of-sample volatility forecasts. For example, Angelidis et al. [3] combine three GARCH specifications with the univariate skew-Student and skew-GED (Generalized Error) distributions to show that these are able to produce superior VaR forecasts compared to the normal. Specifically, they apply the exponential GARCH (EGARCH) model of Nelson [4] and the threshold ARCH (TARCH) model to five univariate returns series and find that while the choice of a skewed, heavy-tailed distribution significantly improves the forecasting performance, the choice of the volatility model appears to be irrelevant. These findings are echoed in Mittnik and Paolella [5] who combine the Asymmetric Power ARCH (APARCH) model of Ding et al. [6] with an asymmetric generalised Student distribution. Within the univariate framework the most complete study of VaR prediction methods is provided by Kuester et al. [7] who compare fully parametric models with VaR constructed using historical simulation, extreme-value theory and quantile regression. Their results show that considerable improvement over normality is achieved when using innovation distributions that allow for skewness and fat tails.
Another salient feature of financial returns series is the fact that comovements between markets increase during periods of high volatility, as shown for example by Longin and Solnik [8] and Brooks et al. [9]. In light of this, a number of studies predict the VaR of a portfolio using multivariate models for the system of individual asset returns in order to achieve forecast improvements due to the use of more information. However, as mentioned in Bauwens et al. [10], in high dimensional frameworks these models can suffer from the “curse of dimensionality” problem thus being more computationally intensive.
In order to shed some light on this issue, several papers focus on direct comparison of the predictive performance of univariate and multivariate GARCH (MGARCH) models under various distributional assumptions with the aim of providing evidence in favour of one of the two approaches. Key studies in this literature include Giot and Laurent [11] and Santos et al. [12]. In the first, the authors compare the univariate APARCH model with a multivariate TVC-APARCH combined with the normal, Student and skew-Student distributions, showing that in both the univariate and multivariate settings the latter produced superior VaR forecasts. In the second, the authors study a number of univariate and multivariate volatility models with normal and Student distributions, finding that the multivariate models combined with the Student offer superior out-of-sample performances.
This paper builds on their approach by widening the set of distributions used to model the error term in both the univariate and multivariate frameworks while maintaining a generic specification for the conditional volatility. Specifically, we consider three symmetric distributions, i.e., normal, Student and Multivariate Exponential Power (MEP), and their corresponding skewed counterparts obtained by applying the transformation of Bauwens and Laurent [13] 1. By incorporating skewness in the corresponding symmetric densities by means of new parameters, we can explicitly analyse its marginal contribution as well as its joint effect with heavy-tails in the model forecasting performance.
As for the choice of the volatility models, within the MGARCH literature we employ the Rotated BEKK (RBEKK) model of Noureldin et al. [15] as it is easy to estimate using covariance targeting even for moderately large cross sections, while in the univariate setting GARCH(1,1) specifications are used to modeling the conditional portfolio variance. Our choice of a relatively simple volatility model allows us to limit our focus entirely on whether the chosen distribution contributes to adequately capture features of the return data. Within the univariate scenario we also consider the NCT-APARCH model proposed by Krause and Paolella [16], as the authors developed an extremely fast method for parameter estimation that is found to outperform highly competitive models and can be easily compared to ours 2.
Both univariate and multivariate models are estimated employing the aforementioned set of distributional assumptions and their accuracy in producing out-of-sample VaR forecasts is assessed by means of statistical backtesting procedures. The selected tests include the Unconditional Coverage (UC), Independence (IND) and Conditional Coverage (CC) tests of Christoffersen [17], the Duration-Based Test of Independence (DBI) of Dumitrescu et al. [18], the Time Until First Failure (TUFF) test of Kupiec [19] and the Dynamic Quantile (DQ) test of Engle and Manganelli [20]. The results of the tests are summarized using a grading scheme based on the number of acceptances of the null hypothesis which determines the distributional assumption providing the most accurate VaR forecasts.
Results from VaR backtesting show that in the multivariate setup the skew-Student clearly outperforms all other distributions. Moreover, its univariate version produces more accurate VaR forecasts than the NCT-GARCH and is able to compete with the NCT-APARCH which incorporates asymmetry into the conditional volatility specification. Overall, our results show that allowing for heavy-tails and skewness produces the most accurate VaR forecasts in both the univariate and multivariate setups. By comparison, specifications including only heavy tails underperform relative to their skewed counterparts with the difference being more pronounced at 5% VaR. Interestingly, the univariate skew-normal distribution produces Var forecasts comparable to the more heavily-parametrized skew-Student and skew-MEP. As regards comparing the performance of univariate and multivariate models in general, our results do not allow us to clearly advocate the use of one methodology over the other. However, given that the hierarchy of distributions according to VaR forecast accuracy is preserved under both frameworks, the univariate approach may be preferred due to its lower computational burden.
The paper is organized as follows: Section 2 reviews the GARCH modelling framework, the theoretical methodology used for constructing the skewed distributions and the Maximum Likelihood (ML) estimation of the models with the selected distributional assumptions. Section 3 introduces the empirical methodology, comprising the portfolio construction, VaR estimation and backtesting procedures. Section 4 provides estimation results and outcomes of the VaR tests and Section 5 concludes with some final remarks.

2. Theoretical Framework

This section illustrates the key points of our theoretical framework. Namely, we outline the alternative approaches to obtain portfolio VaR forecasts using univariate and multivariate models, we describe the procedure used to construct skewed distributions from the corresponding symmetric counterparts and finally we provide an overview of the set of employed distributions, comprising their likelihood derivation.
Two things are worth mentioning. First, we focus on the portfolio VaR for a long position, implying that the predictive power of the models is linked to their ability in modelling large negative returns. Second, we define the asset allocation scheme via an N-dimensional vector of equal weights w t , where N denotes the number of assets in the portfolio and w t = ( w 1 , t , w N , t ) , with w i = 1 / N , i = 1 N w i = 1 . As shown in DeMiguel et al. [21], Tu and Zhou [22], Brown et al. [23] and Fugazza et al. [24], this “naive” diversification rule is able to consistently outperform more sophisticated methods. Moreover, it has the advantage of not being affected by the specified target return as in the Markowitz framework, being only driven by the number of assets.

2.1. VaR Estimation

Let y t = ( y 1 , t , . . . , y N , t ) denote the N-dimensional discrete time vector of de-meaned daily returns at time t and w t the vector of equal weights known at time t 1 . The portfolio return is obtained as r p , t = w t 1 y t , and the portfolio VaR at time t is equal to
V a R t , α = μ p , t + σ p , t q α ,
where μ p , t and σ p , t are respectively the portfolio mean and standard deviation and q α is the left quantile of the assumed conditional distribution at α%. As usually done in practical applications, the considered VaR confidence levels are α = 5 % and 1 % . Note that in the remainder of the paper we will set μ p , t = 0 , thus considering a simplified analytical formula for the computation of the VaR which only accounts for the portfolio conditional variance. Alternative approaches, as done for example in Bauwens et al. [25], fit an ARMA-type structure to the portfolio conditional mean or just assume it to be constant over time (see Santos et al. [12]). Ultimately, when present, the dynamic dependence in the conditional means of portfolio returns is known to be weak and quite difficult to predict, thus assuming a zero mean will have a negligible effects on the VaR forecasts.
The specification of the portfolio standard deviation σ p , t depends on whether we consider a univariate or multivariate approach, with the difference between the two occurring in the conditioning set used.
In the univariate case, the portfolio standard deviation is obtained as the standard deviation of the portfolio returns conditional on past portfolio returns, i.e., as the square root of
σ p , t 2 = E r p , t 2 | r p , 1 , r p , t 1 ,
where the conditional variance σ p , t 2 is estimated using a model chosen from the univariate GARCH class. More precisely, we use a simple GARCH(1,1) specification with variance targeting, which is written as
σ p , t 2 = σ ¯ + a 1 r p , t 1 2 + b 1 σ t 1 2 ,
where σ ¯ = ( 1 a 1 b 1 ) ω ¯ and ω ¯ equals the unconditional variance of returns. Covariance stationarity requires that a 1 + b 1 < 1 , with { a 1 , b 1 } 0 scalar parameters to be estimated.
In the multivariate setup, the conditioning set is made up of the entire vector of past returns:
σ p , t 2 = E r p , t 2 | Y 1 , Y t 1 = w t 1 H t | t 1 w t 1
such that H t | t 1 = E ( H t | t 1 ) is the conditional covariance matrix of returns given the information set available at time t 1 . In this case, a multivariate model for H t needs to be specified. Given its computational ease in practical application, we implement the rotated BEKK (RBEKK) model of Noureldin et al. [15], as it can be estimated for relatively large cross sections even with reach dynamics. The basic idea underlying the model is to transform the original data by performing a rotation and then to fit to the rotated returns the popular BEKK specification of Engle and Kroner [26]. Consequently, the model can be easily estimated with covariance targeting, where the long-run covariance is given by the identity matrix. More precisely, let H ¯ define the unconditional covariance of y t and let the latter be rewritten as y t = H ¯ 1 / 2 ϵ t . Using the spectral decomposition of H ¯ , i.e., H ¯ = P Λ P , where P is a squared matrix of eigenvectors and the eigenvalue matrix Λ is diagonal with nonnegative elements, we can get the symmetric square root of H ¯ as H ¯ 1 / 2 = P Λ 1 / 2 P . The series of rotated returns is thus defined as
ϵ t = H ¯ 1 / 2 y t = P Λ 1 / 2 P y t ,
with V a r ( ϵ t ) = I N , the N-dimensional identity matrix. Note that the conditional covariance of the rotated returns is V a r ( ϵ t | t 1 ) = G t , given that V a r ( y t | t 1 ) = H t = H ¯ 1 / 2 G t H ¯ 1 / 2 . Hence, any dynamic covariance model for H t can also be applied to model the dynamics of G t . Since the rotated returns are orthogonal, a suitable parameterization is represented by the scalar BEKK, which ensures the positive definiteness of the matrix G t being a function of only two scalar parameters, under the assumption of covariance stationarity. Its dynamic equation is expressed as
G t = ( 1 a 2 b 2 ) I N + a 2 ϵ t 1 ϵ t 1 + b 2 G t 1 ,
where, as in the univariate setup, { a 2 , b 2 } 0 and a 2 + b 2 < 1 .

2.2. Constructing Skew Densities

As far as financial applications are concerned, modelling and inference based on the normal distribution have often been proven to be of limited usefulness, as it is possible to gain statistical efficiency by allowing for more involved distributions featuring heavy tails and skewness. As a way to capture higher moments, the literature offers several alternatives. For example, the multivariate noncentral t distribution has fat tails and is skewed; however, the skewness is linked directly to the location parameter, making it somewhat inflexible. The lognormal distribution has also been used to model asset returns, but its skewness is a function of its mean and variance, not a separate parameter. Others, such as the generalized hyperbolic (GH) or mixtures of distributions have also been employed in financial applications, despite being more computationally demanding (see Barndorff-Nielsen [27] for an introduction and Paolella and Polak [28] for a recent application).
In this respect, Fernández and Steel [29] in the univariate case and Bauwens and Laurent [13] in the multivariate one developed a practical procedure for constructing skewed densities from their symmetric unimodal counterparts. These densities can be defined by introducing skewness in the corresponding symmetric densities by means of new parameters, such that the symmetric density results as a particular case. We build on their findings in order to enlarge the set of distributions to be used for VaR forecasting. In the following, we briefly recall the main steps of the procedure, being the discussion restricted to the more general multivariate framework for sake of space.
We begin by defining the notion of symmetry of a standardized density used hereafter. In the univariate case, symmetry corresponds to g ( x ) = g ( x ) , where g ( x ) is a unimodal probability density function with zero mean. In the multivariate case, we rely on the general notion of M-symmetry stated in Definition 1 of Bauwens and Laurent [13], which encompasses the class of spherically symmetric densities. These can be obtained as a special case of the general family of multivariate elliptical distributions, denoted as
g ( x ; μ , Σ , η ) h ( ( x μ ) Σ 1 ( x μ ) , η ) ,
where x is a random vector with an integrable, positive function h ( · ) : R + R + , and η captures the shape parameter of the distribution, when present. The spherically symmetric set of distributions, comprising the standard normal, Student and MEP, are obtained by setting μ and Σ equal to zero and I N , respectively.
The idea of introducing skewness into an M-symmetric standardized distribution revolves around scaling it differently for negative (positive) values by multiplying (dividing) by a positive constant. The value of this scaling parameter (hereafter referred to as ξ) determines whether the resulting distribution is skewed to the left ( 0 < ξ < 1 ) or to the right ( ξ > 1 ).
Definition 1. Given a random vector z = ( z 1 , , z N ) with multivariate symmetric standardized distribution g ( z ; η ) following Equation (6), the standardized skewed density f ( z | ξ ; η ) with vector of asymmetry parameters ξ = ( ξ 1 , ξ N ) , is obtained as:
f ( z | ξ , η ) = 2 N i = 1 N ξ i 1 + ξ i 2 g ( z ; η )
with
z = ( z 1 , . . . , z N )
z i = z i ξ i I i
and
I i = 1  if  z i 0 1  if  z i < 0 .
The marginal r t h -order moment of the obtained skewed distribution can be computed directly from the standardized r t h moment of the symmetric density g ( · ) . This is accomplished by applying the following transformation function:
E ( z i r | ξ ) = M i , r ξ i r + 1 + ( 1 ) r ξ i r + 1 ξ i + 1 ξ i ,
where the r t h -order moment of the marginal g i ( · ) , truncated to the positive real values, is given by
M i , r = 0 2 u r g i ( u ) d u .
Since only the first two moments are required in the transformation process, their analytical expression for r = 1 , 2 in Equation (11) is reported below:
m i = E ( z i | ξ i ) = M i , 1 ξ i 1 ξ i
s i 2 = Var ( z i | ξ i ) = M i , 2 M i , 1 2 ξ i 2 + 1 ξ i 2 + 2 M i , 1 2 M i , 2 .
As shown in Appendix A, s i 2 can also be expressed directly as a function of m i 2 . Note that the resulting skewed distribution, f ( z | ξ , η ) from Definition 1, is not centered at 0 and the variance is a function of ξ (and, where is the case, of the shape parameter η). Given that the elements of z are uncorrelated (since those of x are uncorrelated by assumption), standardization of z is achieved by the following transformation:
z = ( z m ) . / s ,
where m = ( m 1 , . . . , m N ) and s = ( s 1 , . . . , s N ) are the vectors of unconditional means and standard deviations of z computed in Equations (13) and (14) respectively and “./” denotes element-by-element division. Consequently, the standardized form of Definition 1 requires replacing Equation (9) with the following one:
z i = ( s i z i + m i ) ξ i I i ,
where
I i = 1  if  z i m i s i 1  if  z i < m i s i

2.3. Distributions

As already mentioned, three symmetric and three asymmetric distributions are considered in the univariate and multivariate framework. Again, for sake of brevity, we only report the multivariate log-likelihood functions as the univariate can be obtained as special cases 3. The algebraic derivations of the formulas of the moments can be found in Appendix A.2. In all cases, estimation of the parameters is performed in one step by Maximum Likelihood (ML). Namely, the log-likelihood function for T observations is expressed as
T ( ψ ) = t = 1 T log f ( · | ψ , F t 1 ) ,
where ψ is the finite-dimensional vector of model parameters and f ( · | ψ , F t 1 ) denotes the assumed conditional density function of the portfolio returns, in the univariate case, or of the asset return vector in the multivariate one.
Multivariate normal distribution This is the most commonly employed distribution in the literature as it is uniquely identified by its conditional first and second moments, which renders ML estimation much simpler from a computational point of view. In addition, given that the score of the normal log-likelihood function has the martingale difference property when the first two conditional moments are correctly specified, the Quasi Maximum Likelihood (QML) estimates are still consistent and asymptotically normal even if the true DGP is not normally-distributed Bollerslev and Wooldridge [30]. The log-likelihood function for T observations is expressed as follows
T ( ψ ) = 1 2 t = 1 T N log ( 2 π ) + log | H t | + y t H t 1 y t .
As the multivariate normal does not incorporate extra parameters, here we have ψ = ( a 2 , b 2 ) 4.
Multivariate Student distribution The Student distribution is a symmetric and bell-shaped distribution, with heavier tails than the normal. Under the multivariate Student assumption, the log-likelihood function is obtained as
T ( ψ ) = 1 2 t = 1 T log | H t | + ( N + ν ) log 1 + y t H t 1 y t ν 2 + T log Γ ν + N 2 log Γ ν 2 N 2 log ( π ) N 2 log ( ν 2 ) ,
where Γ ( ν ) = 0 e z z ν 1 d z denotes the Gamma function and ν > 2 is the degree of freedom parameter representing the thickness of the distribution tails. As ν increases, the distribution converges to the multivariate normal. The vector of parameter to be estimated is ψ = ( a 2 , b 2 , ν , ξ ) .
Multivariate Exponential Power (MEP) distribution This distribution belongs to the Kotz family of distributions (a particular class of symmetric and elliptical distributions discussed extensively in Fang et al. [31]) and is known to have several equivalent definitions in the literature. It can also include both the normal and the Laplace as special cases, as a function of the value of the non-normality parameter β dictating the tail-behaviour of the distribution. Given its simple implementation, in this paper we consider the pdf given in Solaro [32], which gives rise to the following log-likelihood function:
T ( ψ ) = 1 2 t = 1 T log | H t | + y t H t 1 y t β 2 + T log ( N ) + log Γ N 2 N 2 log ( π ) log Γ 1 + N β 1 + N β log ( 2 ) ,
where ψ = ( a 2 , b 2 , β ) . Note the restriction β > 0 . When β = 2 , the distribution reduces to the multivariate normal, while for β = 1 it corresponds to the multivariate Laplace. Whenever β < 2 ( > 2 ) , the distribution exhibits thicker (thinner) tails than the normal.
Multivariate skew-normal distribution The multivariate skew-normal is the first non-symmetric distribution we consider herein; it accounts for the skewness of the return distribution without taking into account its kurtosis (as it does not involve a tail parameter). Applying Definition 1 we derive the skew-normal density function, with corresponding log-likelihood function equal to
T ( ψ ) = 1 2 t = 1 T log | H t | + i = 1 N s i j = 1 N p i j , t y j , t + m i 2 ξ i 2 I i + T i = 1 N ( log ξ i + log s i ) log ( 1 + ξ i 2 ) + T N 2 log ( 2 ) log ( π ) ,
where p i j , t corresponds to the j t h element of the i t h row of the inverse cholesky factor of the matrix H t (the full derivation is provided in Appendix A.1), ξ i represents the asymmetry parameter of each marginal and I i is the indicator function defined in Equation (17). The vector of parameters of interest is ψ = ( a 2 , b 2 , ξ ) .
Multivariate skew-Student distribution Applying the same procedure as for the skew-normal, the log-likelihood function of the skew-Student distribution is given by the following expression:
T ( ψ ) = 1 2 t = 1 T log | H t | + ( ν + N ) log 1 + i = 1 N s i j = 1 N p i j , t y j , t + m i 2 ξ i 2 I i ν 2 + T N log ( 2 ) N 2 log ( π ) + log Γ ν + N 2 log Γ ν 2 N 2 log ( ν 2 ) + T i = 1 N ( log ξ i + log s i ) log ( 1 + ξ i 2 ) ,
where the parameter ν dictates the thickness of the tails and ξ i is again the asymmetry parameter of each marginal. Notice that the univariate means and standard deviations are functions of ξ i and ν and need not be estimated. Consequently, the skew-Student parametrization requires N + 1 parameters to be estimated in addition to those stemming from the RBEKK specification, i.e., ψ = ( a 2 , b 2 , ν , ξ ) .
Multivariate skew-MEP distribution The log-likelihood function to be maximized is given by
T ( ψ ) = 1 2 t = 1 T log | H t | + i = 1 N s i j = 1 N p i j , t y j , t + m i 2 ξ i 2 I i β 2 + T i = 1 N ( log ξ i + log s i ) log ( 1 + ξ i 2 ) + T log ( N ) + log Γ N 2 N 2 log ( π ) log Γ 1 + N β 1 + N β log ( 2 ) ,
where β is a parameter determining the tail-thickness of the density function, as in the symmetric case. Considering the skewness parameters implies the parameter vector ψ = ( a 2 , b 2 , β , ξ ) .

3. Empirical Application

3.1. Data and Forecasting Scheme

Our dataset (used in the paper of Noureldin et al. [33]) 5 comprises daily open-to-close returns of 10 stocks from the Dow Jones Industrial Average: Bank of America (BAC), JP Morgan (JPM), International Business Machines (IBM), Microsoft (MSFT), Exxon Mobil (XOM), Alcoa (AA), American Express (AXP), Du Pont (DD), General Electric (GE) and Coca Cola (KO). Each univariate vector of returns is calculated as y t = 100 × ( log p t log p t 1 ) and covers a period of 2200 days, from February 2001 to November 2009. Univariate descriptive statistics over the period of interest are provided in Table 1.
Table 1. Univariate descriptive statistics.
Table 1. Univariate descriptive statistics.
StockMeanStd.dev.SkewnessKurtosisKS TestJB Test
Estimation sample: 1 February 2001 to 23 January 2007 (1500 observations)
BAC0.091.09−0.187.450.000.00
JPM0.001.680.9031.020.000.00
IBM−0.041.240.015.960.010.00
MSFT−0.011.370.376.010.000.00
XOM−0.011.130.058.270.820.00
AA0.011.590.144.740.000.00
AXP−0.021.440.337.730.000.00
DD0.021.210.376.760.210.00
GE−0.011.340.137.900.020.00
KO0.010.990.165.530.000.00
Forecasting sample: 24 January 2007 to 30 October 2009 (700 observations)
BAC−0.183.950.379.360.000.00
JPM0.013.060.368.530.000.00
IBM0.081.45−0.026.310.000.00
MSFT0.021.600.085.900.000.00
XOM0.031.61−0.3911.310.000.00
AA−0.042.93−0.837.500.000.00
AXP0.043.060.226.960.000.00
DD−0.041.89−0.125.700.000.00
GE0.022.170.218.960.000.00
KO−0.031.220.077.680.060.00
Full sample: 1 February 2001 to 30 October 2009 (2200 observations)
BAC0.012.400.3321.720.000.00
JPM0.002.210.5716.900.000.00
IBM0.001.310.026.240.020.00
MSFT0.001.450.256.080.000.00
XOM0.001.30−0.2011.560.040.00
AA0.002.11−0.699.950.000.00
AXP0.002.090.3211.230.000.00
DD0.001.460.037.250.000.00
GE0.001.650.2210.850.000.00
KO0.001.070.116.890.000.00
Descriptive statistics of the stock return time series used in the empirical application. The three panels report the statistics for the in-sample period, the out-of-sample period and the full sample period, respectively. “KS test” and “JB test” denotes the Kolmogorov-Smirnov test and Jarque Bera test, with corresponding p-values in column.
Across the three panels, the values of skewness and kurtosis show that the assets are far from being unconditionally normally distributed, thus supporting the conjecture that more flexible distributional assumptions can be conducive to enhanced model performance.
To this extent, one-step ahead forecasts of the conditional portfolio variance (in the univariate case) and of the conditional covariance matrix of returns (in the multivariate one) are recursively obtained as:
σ ^ p , t + 1 2 = E ( σ p , t + 1 2 | t ) ,
H ^ t + 1 = E ( H t + 1 | t ) ,
where t is the information set at time t and σ p , t 2 , H t are defined as in Equations (2) and (5), respectively. Using a rolling-fixed-window scheme, the parameters are estimated over a window length of 1500 observations and used to predict the conditional variance process for the following 20 days. Each time the window is shifted forward by 20 observations and the parameters are re-estimated over the new period in order to compute the next set of forecasts. We iterate this process until the end of the dataset for a total of 35 parameter estimates and 700 one-step ahead forecasts. Table A1 in Appendix B reports the complete list of windows and forecast horizons along with their corresponding calendar dates.
For each model, the portfolio VaR forecast at α% confidence level is then obtained as
V a R t + 1 , α = σ ^ p , t + 1 q α .
For the symmetric distributions in our analysis (normal, Student and MEP), one can easily compute the long VaR of the portfolio by applying Equation (27) and the inverse of each CDF at α%. However, for the non-symmetric distributions this is not straightforward. In order to bypass this complication, for each non-symmetric distribution we apply a simple Monte-Carlo simulation approach, as widely used in VaR computations. Namely, we draw 10 , 000 random vectors (numbers) from each symmetric multivariate (univariate) standardized distribution z t and then we use the estimated skewness parameters to construct the corresponding skewed distribution z t . By assuming r j = H ^ t | t 1 1 / 2 z j ( r j = σ ^ t | t 1 z j ) as the true DGP, we obtain a set of 10,000 simulated returns over the period of interest. Finally, the simulated return distribution is used to derive the 5% and 1% quantiles for the one-step-ahead VaR.

3.2. Testing the Accuracy of VaR Forecasts

The models accuracy in predicting VaR is assessed using multiple statistical backtesting methods. A common starting point for this procedure is the so-called hit function, or indicator function, which is equal to
I t ( α ) = 1  if  r t V a R ( α ) 0  if  r t > V a R ( α )
i.e., it takes the value one if the ex-post portfolio loss exceeds the VaR predicted at time t 1 and the value zero otherwise. According to Christoffersen [34], in order to be accurate, the hit sequence has to satisfy the two properties of correct failure rate and independence of exceptions. The former implies that the probability of realizing a VaR violation should be equal to α × 100 % , while the latter further requires the violations to be independent of each other. These properties can be combined together into one single statement assessing that the hit function has to be an i.i.d. Bernoulli random variable with probability p, i.e., I t ( p ) i . i . d B ( p ) .
This represents the key foundation to many of the backtesting procedures developed in recent years and particularly to the accuracy tests being used in this paper. We focus on tests included in the following three categories:
  • Evaluation of the Frequency of Violations
  • Evaluation of the Independence of Violations
  • Evaluation of the Duration between Violations.
Their properties are briefly described below.
Frequency of Violations The first way of testing the VaR accuracy is to test the number or the frequency of margin exceedances. A test designed to this aim is the Kupiec test (Kupiec [35]), also known as the Unconditional Coverage (UC) test. Its null hypothesis is simply that the percentage of violated VaR forecasts or failure rate p is consistent with the given confidence level α, i.e., H 0 : p = α .
Denoting by F the length of the forecasting period and with v the number of violations occurred throughout this period, the log-likelihood ratio test statistic is defined as
U C = 2 ln p v ( 1 p ) F v p ^ v ( 1 p ^ ) F v ,
where p ^ = v / F is the maximum likelihood estimator under the alternative hypothesis. This ratio test statistic is asymptotically χ 2 ( 1 ) distributed and the null hypothesis is rejected if the critical value at the α % confidence level is exceeded.
A similar useful test is the TUFF (Time Until First Failure) test (Kupiec [35]). Under the null, the probability of an exception is equal to the inverse probability of the VaR confidence level, namely H 0 : p = p ^ = 1 / v . Its basic assumptions are similar to those of the Kupiec test and the t-statistic under the null is obtained as
T U F F = 2 ln p ( 1 p ) v 1 1 v 1 1 v ( v 1 ) .
The TUFF statistic is also asymptotically χ 2 ( 1 ) distributed.
Independence of Violations A limitation of the Kupiec test is that it is only concerned with the coverage of the VaR estimates without accounting for any clustering of the violations. This aspect is crucial for VaR practitioners, as large losses occurring in rapid succession are more likely to lead to disastrous events than individual exceptions.
The Independence test (IND) of Christoffersen [34] uses the same likelihood ratio framework as the previous tests but is designed to explicitly detect clustering in the VaR violations. Under the null hypothesis of independence, the IND test assumes that the probability of an exceedance on a given day t is not influenced by what happened the day before. Formally, H 0 : p 10 = p 11 , where p i j denotes the probability of an i event on day t 1 being followed by a j event on day t.
The relevant IND test statistic can be derived as
I N D = 2 ln p ^ v ( 1 p ^ ) F v p ^ 11 v 11 ( 1 p ^ 11 ) v 01 p ^ 10 v 10 ( 1 p ^ 10 ) v 00 ,
where v i j is the number of violations with value i at time t 1 followed by j at time t. Under the null, the IND statistic is also asymptotically distributed as a χ 2 ( 1 ) random variable.
Although the aforementioned test has received support in the literature, Christoffersen [34] noted that it was not complete on its own. For this reason, he proposed a joint test, the Conditional Coverage (CC) test, which combines the properties of both UC and IND tests.
Formally, the CC ratio statistic can be proven to be the sum of the UC and the IND statistics:
C C = 2 ( l n ( L 0 U C ) l n ( L 1 I N D ) ) = 2 ( l n ( L 0 U C ) l n ( L 1 U C ) + l n ( L 1 U C ) l n ( L 1 I N D ) ) = 2 ( l n ( L 0 U C ) l n ( L 1 U C ) + l n ( L 0 I N D ) l n ( L 1 I N D ) ) = 2 ( l n ( L 0 U C ) l n ( L 1 U C ) ) U C 2 ( l n ( L 0 I N D ) l n ( L 1 I N D ) ) I N D ,
where we added and subtracted the quantity ln ( L 1 ) U C and substituted ln ( L 0 ) I N D for ln ( L 1 ) U C . CC is also χ 2 distributed, but with two degrees of freedom since there are two separate statistics in the test. According to Campbell [36], in some cases it is possible that a VaR model passes the joint test while still failing either the independence test or the unconditional coverage test. Thus it is advisable to run them separately even when the joint test yields a positive result.
A second test belonging to this class is the Regression-based test of Engle and Manganelli [37], also known as Dynamic Quantile (DQ) test. Instead of directly considering the hit sequence, the test is based on its associated quantile process H t ( α ) = I t ( α ) α which assumes the following values:
H t ( α ) = 1 α  if  I t = 1 α  if  I t = 0 .
The idea of this approach is to regress current violations on past violations in order to test for different restrictions on the parameters of the model. That is, we estimate the linear regression model H t ( α ) = δ + k = 1 K β k H t k ( α ) + ϵ t and then we test the joint hypothesis H 0 ( D Q c c ) : δ = β 1 = . . . = β K = 0 . This assumption coincides with the null of Christoffersen’s CC test. It is also possible to split the test and separately test the independence hypothesis and the unconditional coverage hypothesis, respectively as H 0 ( D Q i n d ) : β 1 = . . . = β K = 0 and H 0 ( D Q u c ) : δ = 0 . ( D Q c c ) , ( D Q i n d ) and ( D Q u c ) are asymptotically χ 2 distributed with respectively { K + 1 } , K and one degrees of freedom.
Duration between Violations One of the drawbacks of Christoffersen’s CC test is that it is not capable of capturing dependence in all forms, since it only considers the dependence of observations between two successive days. To address this, Christofferson and Pelletier [38] introduced the Duration-Based test of independence (DBI), which is an improved test for both independence and coverage. Its basic intuition is that if exceptions are completely independent of each other, then the upcoming VaR violations should be independent of the time that has elapsed since the occurrence of the last exceedance (Campbell [36]). The duration (in days) between two exceptions is defined via the no-hit-duration D i = t i t i 1 , where t i is the day of i th violation.
A correctly specified model should have an expected conditional duration of 1 / p days and the no-hit duration should have no memory. The authors construct the ratio statistic considering different distributions for the null and the alternative hypotheses, namely the exponential, since it is the only memory-free (continuous) random distribution, and the Weibull, which allows for duration dependence. The likelihood ratio statistic is derived as
D B I = 2 ln L 0 L 1 = 2 ln p exp { p D } a b b D b 1 exp { ( a D ) b } ,
which has a χ 2 distribution with one degree of freedom.
Under the null hypothesis of independent violations, b = 1 and a is estimated via numerical maximization of l n ( L 1 ) . Whenever b < 1 , the Weibull function has a decreasing path which corresponds to an excessive number of very long durations (very calm period) while b > 1 corresponds to an excessive number of very short durations, namely very volatile periods.

4. Results

4.1. Parameter Estimates

Before turning to the out of sample analysis, it is worth first looking at some parameter estimates obtained by fitting the models on the data. Table 2 reports full sample estimation results over the period January 2001 to February 2009 for a total of 2200 daily observations.
Table 2. Full sample parameter estimates.
Table 2. Full sample parameter estimates.
NormalStudentMEPSkew-NormalSkew-StudentSkew-MEPNCT-APARCHNCT-GARCH
Univariate models
σ ¯ 0.0050.0040.0020.0050.0040.0100.0100.010
a 1 0 . 07 ( 0 . 016 ) 0 . 06 ( 0 . 013 ) 0 . 07 ( 0 . 016 ) 0 . 07 ( 0 . 029 ) 0 . 06 ( 0 . 013 ) 0 . 07 ( 0 . 015 ) 0.050.05
b 1 0 . 92 ( 0 . 017 ) 0 . 93 ( 0 . 013 ) 0 . 92 ( 0 . 017 ) 0 . 92 ( 0 . 031 ) 0 . 93 ( 0 . 014 ) 0 . 93 ( 0 . 017 ) 0.900.90
ν 8 . 44 ( 1 . 358 ) 8 . 42 ( 1 . 366 ) 7.205.20
γ −0.360−0.120
β 1 . 875 ( 0 . 041 ) 1 . 873 ( 0 . 038 )
ξ ¯ 0 . 938 ( 0 . 026 ) 0 . 935 ( 0 . 023 ) 0 . 946 ( 0 . 02 )
LogLik−2971−2946−2965−2968−2943−2963−2944-2963
AIC2.7032.6812.6982.7012.6792.6972.6802.696
Multivariate models
a 2 0 . 021 ( 0 . 001 ) 0 . 015 ( 0 . 001 ) 0 . 022 ( 0 . 001 ) 0 . 032 ( 0 . 002 ) 0 . 015 ( 0 . 001 ) 0 . 026 ( 0 . 001 )
b 2 0 . 976 ( 0 . 001 ) 0 . 983 ( 0 . 001 ) 0 . 976 ( 0 . 001 ) 0 . 966 ( 0 . 002 ) 0 . 983 ( 0 . 001 ) 0 . 972 ( 0 . 002 )
ν 8 . 42 ( 0 . 370 ) 8 . 30 ( 0 . 37 )
ξ ¯ 1 . 016 ( 0 . 035 ) 1 . 018 ( 0 . 022 ) 1 . 018 ( 0 . 034 )
β 1 . 919 ( 0 . 013 ) 2 . 036 ( 0 . 031 )
LogLik−32,154−31,352−32,193−32,121−31,330−32,067
AIC29.23228.50429.27829.21128.44929.154
Note: The table reports test statistics and robust standard errors obtained from full sample parameter estimation, for T = 2200. Note that ξ ¯ denotes the value of the skewness parameters averaged across univariate series with Mean Asymptotic Square Errors (MASE) reported in brackets. For both the NCT-GARCH and NCT-APARCH models ψ = ( μ , ν , γ ) , see Appendix D. The AIC is rescaled by T.
A common feature emerging from both univariate and multivariate panels is that the use of skewed distribution assumptions seems to be justified, as all asymmetric coefficients are significant at standard levels. Moreover, the direct comparison of models fit via the Akaike Information Criteria (AIC) highlights that the models incorporating skewed distributions consistently outperform their symmetric counterparts, with the skew-Student achieving the best fit. In both cases the Student has the lowest AIC among the symmetric densities, while the normal possesses the highest.
In the univariate setting, the NCT-APARCH model substantially improves over the NCT-GARCH due to the introduction of skewness into the volatility model, but it performs slightly worse than its closest competitor, the skew-Student GARCH model. This performance is even more impressive considering that the AIC penalizes the NCT-based models for only three parameters, given that remaining ones are fixed prior to maximization of the likelihood function according to the fast procedure of Krause and Paolella [16] (see Appendix D).
The evolution of the parameters across re-estimations τ = 1 , . . . , 35 allows us to discern their sensitivity to the financial crisis by comparing their dynamics against the key dates outlined in Table A1 in Appendix B. From Figure 1, which looks at the tail parameter estimates for the MEP and Student distributions along with their skewed counterparts, a number of commonalities are immediately apparent. Panel (a) reveals a small increase followed by a sharp drop in β between τ = 18 and 22 for both the univariate and multivariate MEP, indicating a thickening of the tails. Since the time frame corresponds to the windows including the events leading up to the collapse of Lehman Brothers on 15 September 2008, we interpret our results as β adjusting to incorporate the extreme negative events associated to this period.
Figure 1. Tail parameter evolution across re-estimations.
Figure 1. Tail parameter evolution across re-estimations.
Econometrics 04 00003 g001
Panel (b) exhibits the same dynamics as before albeit at a reduced magnitude. This occurs due to the introduction of skewness which is able to capture some of the negative returns associated to the crisis, thereby reducing the need for such a large increase in tail thickness. Evidence of such compensating effects between skewness and heavy-tails is given in Figure 2 which provides the evolution of the skew-parameter ξ across re-estimations (given as an average over the assets in the multivariate case for ease of exposition). Specifically, the dynamics of ξ for the univariate skew-MEP in Panel (a) appear to track the tail dynamics: as β decreases on average between τ equal 1 and 18, ξ exhibits a gradual increase, indicating a reduction of negative skewness. Moreover, the aforementioned drop is accompanied by ξ attaining its maximum value.
Figure 2. Skew parameters evolution across re-estimations.
Figure 2. Skew parameters evolution across re-estimations.
Econometrics 04 00003 g002
Turning to the Student and skew-Student distributions, the dynamics are somewhat different along a number of dimensions. First, there is now a marked difference between the univariate and multivariate cases. In the former, the dynamics are similar to the MEP and skew-MEP results wherein the tails thicken at around τ = 18 as evidenced by the sharp drop in ν. This is then followed by a short-lived increase after which the tails thicken gradually over the remaining estimation period. The reasoning for this, whereby the tails thicken to accommodate the negative returns associated to the crisis, is the same as the MEP-case.
Unlike the MEP and skew-MEP cases, where the multivariate specification exhibits thinner tails than the univariate but both exhibit congruent dynamics, the Student and skew-Student exhibit fewer similarities between specifications. In contrast, moving from the Student to skew-Student setting reveals almost identical tail parameter dynamics. Combined with the positive skew exhibited in Panel (b) of Figure 2, this indicates that in this case tail thickness does not adjust to incorporate the addition of skewness. The effect of these features on VaR forecast accuracy will be seen in Section 4.2.

4.2. VaR Backtesting Results

In the multivariate setting, the out-of-sample covariance matrix predictions are used to construct equally-weighted portfolios for the computation of the one-step-ahead VaR. Table 3 compares portfolios standard deviation obtained using the univariate and multivariate approaches over both the in- and out-of-sample periods.
Table 3. Portfolios descriptive statistics.
Table 3. Portfolios descriptive statistics.
Univariate Models
NormalStudentMEPSkew-NormalSkew-StudentSkew-MEPNCT-APARCHNCT-GARCH
In-sample: 1 February 2001 to 23 January 2007 (1500 observations)
σ ¯ p 0.87160.87220.86620.87090.87180.87800.82220.8720
min { σ p } 0.41590.41550.38990.41340.41410.44570.37270.3852
max { σ p } 2.78052.68272.82342.81472.69652.70952.52422.5826
Forecasting sample: 24 January 2007 to 30 October 2009 (700 observations)
σ ¯ p 1.46591.46011.48001.46561.45501.45731.45691.4644
min { σ p } 0.43980.44480.41390.43340.44270.47180.37450.3899
max { σ p } 3.95893.89673.97433.96563.80333.92814.23103.1567
Multivariate Models
NormalStudentMEPSkew-NormalSkew-StudentSkew-MEP
In-sample: 1 February 2001 to 23 January 2007 (1500 observations)
σ ¯ p 0.90210.91150.89830.90180.91150.8924
min { σ p } 0.52260.54740.50770.52120.54780.4840
max { σ p } 1.90671.79231.93911.90971.79351.9972
Forecasting sample: January 24, 2007 to October 30, 2009 (700 observations)
σ ¯ p 1.47211.46461.48751.47401.45491.4691
min { σ p } 0.52260.54740.51050.52330.54790.4908
max { σ p } 3.17333.06573.21083.17283.06743.2683
Note: The table reports average, minimum and maximum value of portfolio standard deviation over the in- and out-of-sample periods.
As already noted, the financial crisis features heavily in the summary statistics. Since this period is included in the forecasting sample (starting from observation 1921 according to Table A1), we notice a sharp increase in the portfolio standard deviation of all the models (see also the figures reported in Appendix C). On a general basis, heavy-tailed or skewed distributions do not lead to remarkable gains over the in-sample period, as all the models report very similar values on average. This pattern is partly reversed in the forecasting period: while the normal and skew-normal remain very close, the skew-Student and skew-MEP exhibit a lower portfolio standard deviation than their symmetric counterparts. Remarkably good is the performance of the models featuring the NCT distribution; the NCT-APARCH achieves the lowest standard deviation among the univariate alternatives and compares favorably also to many multivariate models, being only outperformed by the skew-Student RBEKK. It appears that the uncertainty due to the larger number of parameters to be estimated is not fully compensated by the gain coming from a better representation of the volatility dynamics in multivariate models.
Overall, according to this table, the univariate and multivariate approaches deliver quite similar portfolio summary statistics. If the focus was on the predicted portfolio variance alone, then the ideal choice would be to use a univariate volatility model coupled with either the skew-Student or the NCT distribution, as they are easy to estimate and computationally faster than the multivariate specifications. Ultimately, we are interested in the models accuracy in forecasting the one-step-ahead portfolio VaR, so we move to analyse the outcomes of the statistical backtesting procedures.
Table 4 and Table 5 report the results from the TUFF, UC, IND, CC and DBI tests in the univariate and multivariate cases, respectively.
All statistical tests are computed for the 5% and 1% VaR confidence level. For each portfolio we report test statistics along with their corresponding p-values in brackets. Since the applied tests measure the models accuracy in forecasting VaR along several dimensions (as detailed in Section 3.2), the overall results are summarized using a performance measure which considers the percentage of acceptances of the null hypothesis across the different tests at the standard 5 % significance level. To this end, rejections of the null are highlighted in bold with the total grade across distributions reported in Table 6.
Table 4. VaR backtesting results—Univariate.
Table 4. VaR backtesting results—Univariate.
NormSkew-NormStudentSkew-StudentMEPSkew-MEPNCT-APARCHNCT-GARCH
5% VaR
violation/frequency 52 ( 0 . 074 ) 42 ( 0 . 060 ) 53 ( 0 . 076 ) 29 ( 0 . 041 ) 52 ( 0 . 074 ) 40 ( 0 . 057 ) 53 ( 0 . 076 ) 25 ( 0 . 036 )
TUFF 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 0 . 022 ( 0 . 883 ) 5 . 991 ( 0 . 014 )
UC 7 . 611 ( 0 . 006 ) 1 . 389 ( 0 . 239 ) 8 . 476 ( 0 . 004 ) 1 . 147 ( 0 . 284 ) 7 . 611 ( 0 . 006 ) 0 . 720 ( 0 . 396 ) 3 . 326 ( 0 . 068 ) 8 . 476 ( 0 . 004 )
IND 8 . 829 ( 0 . 003 ) 2 . 646 ( 0 . 104 ) 8 . 815 ( 0 . 003 ) 1 . 155 ( 0 . 283 ) 7 . 870 ( 0 . 005 ) 1 . 676 ( 0 . 195 ) 5 . 151 ( 0 . 023 ) 9 . 861 ( 0 . 002 )
CC 16 . 440 ( 0 . 000 ) 4 . 035 ( 0 . 133 ) 17 . 290 ( 0 . 000 ) 2 . 302 ( 0 . 316 ) 15 . 481 ( 0 . 000 ) 2 . 397 ( 0 . 302 ) 8 . 477 ( 0 . 014 ) 18 . 337 ( 0 . 000 )
DBI 0 . 023 ( 0 . 880 ) 0 . 079 ( 0 . 778 ) 0 . 087 ( 0 . 768 ) 0 . 830 ( 0 . 362 ) 0 . 046 ( 0 . 831 ) 0 . 100 ( 0 . 751 ) 0 . 797 ( 0 . 372 ) 0 . 303 ( 0 . 582 )
1% VaR
violation/frequency 15 ( 0 . 021 ) 12 ( 0 . 017 ) 12 ( 0 . 017 ) 5 ( 0 . 007 ) 14 ( 0 . 020 ) 11 ( 0 . 016 ) 11 ( 0 . 016 ) 8 ( 0 . 011 )
TUFF 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 )
UC 6 . 957 ( 0 . 008 ) 2 . 972 ( 0 . 085 ) 2 . 972 ( 0 . 085 ) 0 . 641 ( 0 . 423 ) 5 . 479 ( 0 . 019 ) 1 . 967 ( 0 . 161 ) 0 . 138 ( 0 . 710 ) 1 . 967 ( 0 . 161 )
IND 7 . 638 ( 0 . 006 ) 3 . 406 ( 0 . 065 ) 3 . 406 ( 0 . 065 ) 0 . 707 ( 0 . 400 ) 6 . 072 ( 0 . 014 ) 2 . 330 ( 0 . 127 ) 0 . 326 ( 0 . 568 ) 2 . 330 ( 0 . 127 )
CC 14 . 595 ( 0 . 001 ) 6 . 378 ( 0 . 041 ) 6 . 378 ( 0 . 041 ) 1 . 348 ( 0 . 510 ) 11 . 551 ( 0 . 003 ) 4 . 297 ( 0 . 117 ) 0 . 464 ( 0 . 793 ) 4 . 297 ( 0 . 117 )
DBI 0 . 518 ( 0 . 472 ) 0 . 126 ( 0 . 723 ) 0 . 002 ( 0 . 969 ) 0 . 000 ( 0 . 985 ) 0 . 547 ( 0 . 459 ) 0 . 281 ( 0 . 596 ) 0 . 501 ( 0 . 479 ) 0 . 281 ( 0 . 596 )
Table 5. VaR backtesting results—Multivariate.
Table 5. VaR backtesting results—Multivariate.
NormSkew-NormStudentSkew-StudentMEPSkew-MEP
5% VaR
violation/frequency 53 ( 0 . 076 ) 51 ( 0 . 073 ) 56 ( 0 . 080 ) 38 ( 0 . 054 ) 52 ( 0 . 074 ) 44 ( 0 . 063 )
TUFF 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 ) 0 . 022 ( 0 . 883 ) 5 . 991 ( 0 . 014 ) 5 . 991 ( 0 . 014 )
UC 8 . 476 ( 0 . 004 ) 6 . 789 ( 0 . 009 ) 11 . 311 ( 0 . 001 ) 0 . 264 ( 0 . 608 ) 7 . 611 ( 0 . 006 ) 2 . 260 ( 0 . 133 )
IND 9 . 861 ( 0 . 002 ) 7 . 849 ( 0 . 005 ) 13 . 261 ( 0 . 000 ) 0 . 273 ( 0 . 601 ) 8 . 829 ( 0 . 003 ) 2 . 516 ( 0 . 113 )
CC 18 . 337 ( 0 . 000 ) 14 . 637 ( 0 . 001 ) 24 . 572 ( 0 . 000 ) 0 . 536 ( 0 . 765 ) 16 . 440 ( 0 . 000 ) 4 . 776 ( 0 . 092 )
DBI 0 . 858 ( 0 . 354 ) 0 . 818 ( 0 . 366 ) 1 . 052 ( 0 . 305 ) 1 . 501 ( 0 . 221 ) 0 . 797 ( 0 . 372 ) 1 . 593 ( 0 . 207 )
1% VaR
violation/frequency 19 ( 0 . 027 ) 18 ( 0 . 026 ) 17 ( 0 . 024 ) 8 ( 0 . 011 ) 17 ( 0 . 024 ) 15 ( 0 . 021 )
TUFF 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 ) 1 . 426 ( 0 . 232 )
UC 14 . 153 ( 0 . 000 ) 12 . 176 ( 0 . 000 ) 10 . 313 ( 0 . 001 ) 0 . 138 ( 0 . 710 ) 10 . 313 ( 0 . 001 ) 4 . 051 ( 0 . 044 )
IND 14 . 568 ( 0 . 000 ) 13 . 106 ( 0 . 000 ) 11 . 190 ( 0 . 001 ) 0 . 326 ( 0 . 568 ) 10 . 978 ( 0 . 001 ) 1 . 652 ( 0 . 199 )
CC 28 . 721 ( 0 . 000 ) 25 . 282 ( 0 . 000 ) 21 . 503 ( 0 . 000 ) 0 . 464 ( 0 . 793 ) 21 . 292 ( 0 . 000 ) 5 . 702 ( 0 . 057 )
DBI 3 . 109 ( 0 . 078 ) 0 . 902 ( 0 . 342 ) 3 . 798 ( 0 . 055 ) 0 . 282 ( 0 . 595 ) 1 . 573 ( 0 . 210 ) 0 . 774 ( 0 . 379 )
Table 6. Grade comparison—VaR backtesting.
Table 6. Grade comparison—VaR backtesting.
NormSkew-NormStudentSkew-StudentMEPSkew-MEPNCT-APARCHNCT-GARCH
5% VaR
Univariate grade20%80%20%80%20%80%60%20%
Multivariate grade20%20%20%100%20%80%
1% VaR
Univariate grade40%80%80%100%80%100%100%100%
Multivariate grade40%40%40%100%40%80%
The first distinguishing feature from the VaR backtesting results is the clear predominance of the skew-Student distribution. This holds for both the univariate and multivariate frameworks with the former producing VaR forecasts that outperform the NCT-APARCH at 5% VaR. Similarly, the skew-MEP distribution produces highly accurate VaR forecasts across the board. These findings exemplify a second feature of the results namely, the impact on VaR forecast accuracy of introducing skewness into a heavy-tailed distribution. Clearly, the performance of both the skew-Student and skew-MEP distributions improves compared to their symmetric counterparts, but this effect is less pronounced at 1% VaR. Indeed, the performance of the Student and MEP distributions improves when moving from the 5 % to the 1% VaR scenario, suggesting that despite the improvement arising from the introduction of skewness, heavy-tails remain useful in capturing larger swings in returns, i.e., those events located further out in the tails. As a final remark on the effect of skewness, we observe the large difference in performance between the NCT-APARCH and NCT-GARCH at 5% VaR. Though both feature a skewed distributional specification, recall that the NCT-APARCH also features asymmetry in the conditional volatility model which explains its superior performance. An analogous result is obtained in Giot and Laurent [11].
Analysis of the differences between the univariate and multivariate models reveals two key points. First, while the skew-Student and skew-MEP retain their dominance under both frameworks, the performance of their symmetric counterparts at 1% VaR is worse under the multivariate specification. Second, within the univariate framework, the skew-normal is capable of producing VaR forecasts comparable to the high performance skewed and heavy-tailed distributions at both VaR confidence levels. This does not hold in the multivariate setup where the skew-normal offers no improvements in VaR accuracy over the normal. Besides, we observe that in both frameworks the empirical failure rate (i.e., the frequency of violations) of the skewed distributions is closer to the nominal value than their symmetric counterparts, which are oversized. The NCT-GARCH represents the only exception, being considerably more conservative than the NCT-APARCH at both 5% and 1% VaR confidence levels.
With respect to the previous backtesting methods, the DQ test takes into account a more general temporal dependence between the series of violations and is considered the most reliable in assessing VaR accuracy 6. The DQ test results are reported in Table 7 and Table 8. In order to compare the different distributional assumptions, Table 9 summarises the percentage of null hypothesis acceptances over the two lag-lengths for each confidence level.
The DQ test results tell a similar story to the VaR backtesting procedures. As before, the skew-Student outperforms its competitors at both 5% and 1% VaR under both univariate and multivariate setups. The univariate skew-normal continues to produce VaR forecasts with an accuracy comparable to the top-performers. Again, this does not extend to the multivariate case.
Finally, forecast accuracy at 5% VaR again reveals a more pronounced improvement when moving from symmetric to skewed distributions than in the 1% case. Overall, introducing skewness into heavy-tailed distributions continues to offer the highest VaR forecast accuracy.
Table 7. Dynamic Quantile test results—Univariate models.
Table 7. Dynamic Quantile test results—Univariate models.
NormSkew-NormStudentSkew-StudentMEPSkew-MEPNCT-APARCHNCT-GARCH
5% VaR
K = 1
D Q U C 9 . 319 ( 0 . 002 ) 1 . 594 ( 0 . 207 ) 10 . 092 ( 0 . 001 ) 1 . 079 ( 0 . 299 ) 8 . 965 ( 0 . 003 ) 0 . 809 ( 0 . 368 ) 3 . 188 ( 0 . 074 ) 10 . 485 ( 0 . 001 )
D Q I N D 1 . 433 ( 0 . 231 ) 1 . 168 ( 0 . 280 ) 0 . 389 ( 0 . 533 ) 0 . 020 ( 0 . 886 ) 0 . 281 ( 0 . 596 ) 0 . 867 ( 0 . 352 ) 0 . 698 ( 0 . 403 ) 1 . 651 ( 0 . 199 )
D Q C C 10 . 189 ( 0 . 006 ) 2 . 665 ( 0 . 264 ) 10 . 201 ( 0 . 006 ) 1 . 087 ( 0 . 581 ) 9 . 036 ( 0 . 011 ) 1 . 636 ( 0 . 441 ) 3 . 680 ( 0 . 159 ) 11 . 463 ( 0 . 003 )
K = 3
D Q U C 7 . 028 ( 0 . 008 ) 1 . 196 ( 0 . 274 ) 6 . 821 ( 0 . 009 ) 1 . 240 ( 0 . 265 ) 6 . 193 ( 0 . 013 ) 0 . 525 ( 0 . 469 ) 3 . 024 ( 0 . 082 ) 8 . 068 ( 0 . 005 )
D Q I N D 0 . 633 ( 0 . 426 ) 0 . 251 ( 0 . 616 ) 4 . 536 ( 0 . 033 ) 0 . 712 ( 0 . 399 ) 3 . 738 ( 0 . 053 ) 0 . 042 ( 0 . 838 ) 0 . 102 ( 0 . 749 ) 0 . 382 ( 0 . 537 )
D Q C C 12 . 280 ( 0 . 015 ) 2 . 398 ( 0 . 663 ) 18 . 210 ( 0 . 001 ) 2 . 633 ( 0 . 621 ) 15 . 467 ( 0 . 004 ) 1 . 624 ( 0 . 805 ) 5 . 220 ( 0 . 265 ) 12 . 955 ( 0 . 011 )
1% VaR
K = 1
D Q U C 9 . 622 ( 0 . 002 ) 3 . 744 ( 0 . 053 ) 3 . 744 ( 0 . 055 ) 0 . 580 ( 0 . 446 ) 7 . 357 ( 0 . 007 ) 2 . 393 ( 0 . 122 ) 0 . 151 ( 0 . 698 ) 2 . 393 ( 0 . 122 )
D Q I N D 0 . 713 ( 0 . 398 ) 0 . 363 ( 0 . 547 ) 0 . 363 ( 0 . 557 ) 0 . 026 ( 0 . 872 ) 0 . 579 ( 0 . 447 ) 0 . 280 ( 0 . 597 ) 0 . 107 ( 0 . 744 ) 0 . 280 ( 0 . 597 )
D Q C C 9 . 985 ( 0 . 007 ) 3 . 991 ( 0 . 136 ) 3 . 991 ( 0 . 136 ) 0 . 598 ( 0 . 741 ) 7 . 680 ( 0 . 021 ) 2 . 603 ( 0 . 272 ) 0 . 254 ( 0 . 881 ) 2 . 603 ( 0 . 272 )
K = 3
D Q U C 9 . 078 ( 0 . 003 ) 3 . 376 ( 0 . 066 ) 3 . 376 ( 0 . 066 ) 0 . 373 ( 0 . 541 ) 6 . 848 ( 0 . 009 ) 2 . 118 ( 0 . 146 ) 0 . 126 ( 0 . 723 ) 2 . 118 ( 0 . 146 )
D Q I N D 0 . 013 ( 0 . 910 ) 0 . 528 ( 0 . 468 ) 0 . 528 ( 0 . 468 ) 0 . 617 ( 0 . 432 ) 0 . 100 ( 0 . 751 ) 0 . 876 ( 0 . 349 ) 2 . 605 ( 0 . 106 ) 0 . 876 ( 0 . 349 )
D Q C C 13 . 749 ( 0 . 008 ) 9 . 654 ( 0 . 047 ) 9 . 654 ( 0 . 047 ) 19 . 505 ( 0 . 001 ) 11 . 961 ( 0 . 018 ) 9 . 171 ( 0 . 057 ) 10 . 840 ( 0 . 028 ) 9 . 171 ( 0 . 057 )
Table 8. Dynamic Quantile test results—Multivariate.
Table 8. Dynamic Quantile test results—Multivariate.
NormSkew-normStudentSkew-StudentMEPSkew-MEP
5% VaR
K = 1
D Q U C 10 . 485 ( 0 . 001 ) 8 . 226 ( 0 . 004 ) 14 . 416 ( 0 . 000 ) 0 . 280 ( 0 . 597 ) 9 . 319 ( 0 . 002 ) 2 . 538 ( 0 . 111 )
D Q I N D 1 . 651 ( 0 . 199 ) 1 . 232 ( 0 . 267 ) 2 . 406 ( 0 . 121 ) 0 . 000 ( 0 . 993 ) 1 . 433 ( 0 . 231 ) 0 . 261 ( 0 . 610 )
D Q C C 11 . 463 ( 0 . 003 ) 8 . 990 ( 0 . 011 ) 15 . 751 ( 0 . 000 ) 0 . 280 ( 0 . 869 ) 10 . 189 ( 0 . 006 ) 2 . 727 ( 0 . 256 )
K = 3
D Q U C 7 . 733 ( 0 . 005 ) 6 . 096 ( 0 . 014 ) 10 . 767 ( 0 . 001 ) 0 . 277 ( 0 . 599 ) 7 . 028 ( 0 . 008 ) 1 . 607 ( 0 . 205 )
D Q I N D 1 . 006 ( 0 . 316 ) 0 . 914 ( 0 . 339 ) 0 . 690 ( 0 . 406 ) 0 . 789 ( 0 . 374 ) 0 . 633 ( 0 . 426 ) 3 . 148 ( 0 . 076 )
D Q C C 14 . 578 ( 0 . 006 ) 11 . 839 ( 0 . 019 ) 18 . 650 ( 0 . 001 ) 2 . 662 ( 0 . 616 ) 12 . 280 ( 0 . 015 ) 9 . 201 ( 0 . 056 )
1% VaR
K = 1
D Q U C 19 . 549 ( 0 . 000 ) 18 . 210 ( 0 . 000 ) 15 . 080 ( 0 . 000 ) 0 . 151 ( 0 . 698 ) 13 . 358 ( 0 . 000 ) 9 . 622 ( 0 . 002 )
D Q I N D 1 . 278 ( 0 . 258 ) 1 . 167 ( 0 . 280 ) 1 . 041 ( 0 . 308 ) 0 . 107 ( 0 . 744 ) 2 . 095 ( 0 . 148 ) 0 . 713 ( 0 . 398 )
D Q C C 22 . 121 ( 0 . 000 ) 18 . 684 ( 0 . 000 ) 15 . 521 ( 0 . 000 ) 0 . 254 ( 0 . 881 ) 16 . 575 ( 0 . 000 ) 9 . 985 ( 0 . 007 )
K = 3
D Q U C 13 . 935 ( 0 . 000 ) 14 . 056 ( 0 . 000 ) 9 . 885 ( 0 . 002 ) 0 . 126 ( 0 . 723 ) 10 . 157 ( 0 . 001 ) 6 . 809 ( 0 . 009 )
D Q I N D 20 . 104 ( 0 . 000 ) 6 . 375 ( 0 . 094 ) 15 . 843 ( 0 . 000 ) 2 . 605 ( 0 . 106 ) 14 . 404 ( 0 . 000 ) 4 . 056 ( 0 . 255 )
D Q C C 44 . 966 ( 0 . 000 ) 35 . 949 ( 0 . 000 ) 47 . 702 ( 0 . 000 ) 8 . 842 ( 0 . 065 ) 33 . 192 ( 0 . 000 ) 9 . 271 ( 0 . 054 )
Table 9. Grade comparison—DQ tests.
Table 9. Grade comparison—DQ tests.
NormSkew-NormStudentSkew-StudentMEPSkew-MEPNCT-APARCHNCT-GARCH
5% VaR
Univariate grade33%100%17%100%33%100%100%33%
Multivariate grade33%33%33%100%33%100%
1% VaR
Univariate grade33%83%83%83%33%100%83%100%
Multivariate grade17%33%17%100%17%50%

5. Conclusions

Given its importance in risk management, practitioners must be capable of forecasting the Value-at-Risk of their asset portfolios to a high degree of accuracy. This requires taking into account a number of properties of financial returns namely, non-normality, heavy-tails, skewness and the possibility of comovements between assets. In this article, we focus primarily on the effect of varying the distributional assumption used to forecast VaR. Moreover, we addressed the still open question of whether univariate or multivariate models are most appropriate for the problem of portfolio VaR forecasting.
The distributions treated in the paper comprised three symmetric and three skewed distributional assumptions (i.e., normal, Student, MEP and their skewed counterparts) which were coupled with the RBEKK model in the multivariate framework and GARCH in the univariate. In addition, we compared our specification with a novel method for fast estimation of the (univariate) NCT-GARCH and NCT-APARCH models. We then proceeded to the models’ accuracy in predicting equally-weighted portfolio VaR.
Employing a series of standard backtesting methods to compare the distribution-based model performance, the results reveal that the skew-Student specification produces the most accurate one-step ahead VaR forecast across all multivariate specifications and is able to compete with the high-performance NCT-APARCH in the univariate setup. This finding is echoed in the univariate MEP results wherein the skewed version outperforms the symmetric across all tests and confidence levels. By contrast, the multivariate skew-MEP exhibits thinner tails than the normal and thus, performs poorly at the 1% confidence level. More generally, The test results reveal a clear hierarchy of distributional assumptions within the univariate and multivariate setups, with that hierarchy being preserved when moving from one to the other. However, attempting to compare the performance of univariate to multivariate distributions in general does not reveal any key differences with the exception of certain cases in the skew-normal and skew-MEP specifications. Consequently, the additional computational burden of estimating multivariate models does not seem to be justified.
There are several possible avenues of research extending from this work. Given the focus on distributions, we limited our attention to relatively simple parametric models namely, the RBEKK and GARCH models. As the NCT-GARCH/APARCH results showed, capturing asymmetries in returns volatility improves forecast accuracy. Considering their multivariate versions would provide a useful contribution and allow for a detailed study of the optimal model-distribution-dimension combination. Another possibility would be to consider higher forecast horizons for the VaR in order to check if the inclusion of skewness and asymmetric forms of dependence can lead to significant improvements in the long run.

Acknowledgments

Manuela Braione and Nicolas K. Scholtes acknowledge support of the “Communauté française de Belgique” through contracts “Projet d’Actions de Recherche Concertées” “12/17-045” and “13/17-055”, respectively granted by the Académie universitaire Louvain. We thank seminar participants at the 13th Journée d’économétrie at the Université Paris Ouest Nanterre La Défense along with Luc Bauwens and Florian Ielpo for useful comments. We also thank two anonymous referees and the academic editors for their comments and advice, all of which led to considerable improvements in the paper.

Author Contributions

Manuela Braione and Nicolas K. Scholtes contributed equally to the writing of the MATLAB code as well as the preparation of the current manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendixes

A. Derivations

A.1. Transformation

The transformation z t = H t 1 / 2 y t is incorporated into the symmetric, standardised pdfs as follows:
κ κ = ( κ 1 , . . . , κ N ) ( κ 1 , . . . , κ N ) = ( s i z i + m i ) ξ i I i ( s i z i + m i ) ξ i I i = ( s i j = 1 N p i j y j + m i ) ξ i I i ( s i j = 1 N p i j y j + m i ) ξ i I i = i = 1 N s i j = 1 N p i j y j + m i ξ i 2 I i ,
where p i j corresponds to the j t h element of the i t h row of H t 1 / 2 . Note that the t subscript is dropped for simplicity. The matrix square root operation is carried out by applying the Cholesky decomposition of H t such that B B = H t . As a result, each z i is obtained by multiplying the row vector of H t 1 / 2 corresponding to asset i with the demeaned return vector (giving us the inner summation above) which is then multiplied by the univariate standard deviation and added to the univariate mean. The presence of skewness is factored in by the term ξ i I i , where the factor I i is defined as in Equation (17).

A.2. Distributions Moments

We report the first two moments of the univariate symmetric normal, Student and MEP distributions used to compute the log-likelihood function as given in Section 2.3.
Note that the following relation holds:
m i = M i , 1 ξ i 1 ξ i M i , 1 2 = m i 2 ξ i 2 ( ξ i 2 1 ) 2 .
Hence, substituting this result into Equation (14), the 2nd order moment of the skewed distributions can be obtained as a function of the first:
s i 2 = M i , 1 2 ξ i 2 1 ξ i 2 + 2 + M i , 2 ξ i 2 + 1 ξ i 2 1 = ξ i 2 ( ξ i 2 1 ) 2 ξ i 4 + 2 ξ i 2 1 ξ i 2 m i 2 + M i , 2 ξ i 4 ξ i 2 + 1 ξ i 2 = ξ i 2 ( ξ i 2 1 ) 2 ( ξ i 2 + 1 ) ( ξ i 2 1 ) ξ i 2 m i 2 + M i , 2 ξ i 2 ξ i 2 1 + 1 ξ i 2 ξ i 2 = m i 2 1 ξ i 2 ξ i 2 1 + M i , 2 ξ i 2 + 1 ξ i 2 1 .
Skew-Normal
M i , 1 = 0 2 2 π u exp 1 2 u 2 d u = 2 π 0 u exp 1 2 u 2 d u = 2 π M i , 2 = 0 2 2 π u 2 exp 1 2 u 2 d u = 2 π 0 u 2 exp 1 2 u 2 d u = 1
Skew-Student
M i , 1 = 2 Γ ν + 1 2 Γ ν 2 π ( ν 2 ) 0 u 1 + u 2 ν 2 1 + ν 2 d u = 2 ν 2 ν 1 2 Γ ν 1 2 π ( ν 1 ) Γ ν 2 = 2 ν 2 Γ ν + 1 2 π ( ν 1 ) Γ ν 2 = Γ ν 1 2 ν 2 π Γ ν 2 M i , 2 = 2 Γ ν + 1 2 Γ ν 2 π ( ν 2 ) 0 u 2 1 + u 2 ν 2 1 + ν 2 d u = ( ν 2 ) Γ ν 2 1 2 Γ ν 2 = ( ν 2 ) Γ ν 2 1 2 ν 2 2 Γ ν 2 1 = 1
Skew-MEP
M i , 1 = 2 1 β Γ 1 + 1 β 0 u exp 1 2 u β d u = 2 1 + 1 β Γ 2 + β β Γ 1 + 1 β M i , 2 = 2 1 β Γ 1 + 1 β 0 u 2 exp 1 2 u β d u = 4 1 β Γ 3 β β Γ 1 + 1 β

B. Tables

Table A1. Windows length and corresponding calendar time.
Table A1. Windows length and corresponding calendar time.
Rolling Fixed-WindowForecast Horizon
It.ObservationsDaysObservationsDays
11–15001 Febraury 2001–23 January 20071501–152024 January 2007–21 Febraury 2007
221–15202 March 2001–21 Febraury 20071521–154022 Febraury 2007–21 March 2007
341–154030 March 2001–21 March 20071541–156022 March 2007–19 April 2007
461–156030 April 2001–19 April 20071561–158020 April 2007–17 May 2007
581–158029 May2001–17 May 20071581–160018 May 2007–15 June 2007
6101–160026 June 2001– 15 June 20071601–162018 June 2007–16 July 2007
7121–162025 July 2001–16 July 20071621–164017 July 2007–13 August 2007
8141–164022 August 2001–13 August 20071641–166014 August 2007–11 September 2007
9161–166026 September 2001–11 September 20071661–168012 September 2007–9 October 2007
10181–168024 October 2001–9 October 20071681–170010 October 2007–6 November 2007
11201–170021 November 2001–6 Novembe 20071701–17207 November 2007–5 December 2007
12221–172020 December 2001–5 December 20071721–17406 December 2007–4 January 2008
13241–174022 January 2002–4 January 20081741–17607 January 2008–4 Febraury 2008
14261–176020 Febraury 2002–4 Febraury 20081761–17805 Febraury 2008– 4 March 2008
15281–178020 March 2002–4 March 20081781–18005 March 2008–2 April 2008
16301–180018 April 2002–2 April 20081801–18203 April 2008–30 April 2008
17321–182016 May 2002–30 May 20081821–18401 May 2008–29 May 2008
18341–184014 June 2002–29 May 20081841–186030 May 2008–26 June 2008
19361–186015 July 2002–26 June 20081861–188027 June 2008–25 July 2008
20381–188012 August 2002–25 July 20081881–190028 July 2008– 22 August 2008
21401–190010 September2002–22 August 20081901–192025 August 2008– 22 September 2008
22421–19208 October 2002–22 September 20081921–194023 September 2008–20 October 2008
23441–19405 November 2002–20 October 20081941–196021 October 2008–17 November 2008
24461-19604 December 2002–17 November 20081961–198018 November 2008–16 December 2008
25481–19803 January 2003–16 December 20081981–200017 December 2008–15 January 2009
26501–20003 Febraury 2003–15 January 20092001–202016 January 2009–13 Febraury 2009
27521–20204 March 2003–13 Febraury 20092021–204017 Febraury 2009–16 March 2009
28541–20401 April 2003–16 March 20092041–206017 March 2009–14 April 2009
29561–206030 April 2003–14 April 20092061–208015 April 2009–12 May 2009
30581–208029 May 2003–12 May 20092081–210013 May 2009–10 June 2009
31601–210026 June 2003–10 June 20092101–212011 June 2009–9 July 2009
32621–212025 July 2003–9 July 20092121–214010 July 2009–6 August 2009
33641–214022 August 2003–6 August 20092141–21607 August 2009–3 September 2009
34661–216022 September 2003–3 September 20092161–21804 September 2009–2 October 2009
35681-218020 October 2003–2 October 20092181–22005 October 2009–30 October 2009

C. Figures

Figure A1. Evolution across re-estimations of the GARCH/RBEKK parameter, a.
Figure A1. Evolution across re-estimations of the GARCH/RBEKK parameter, a.
Econometrics 04 00003 g003
Figure A2. Evolution across re-estimations of the GARCH/RBEKK parameter, b.
Figure A2. Evolution across re-estimations of the GARCH/RBEKK parameter, b.
Econometrics 04 00003 g004
Figure A3. Portfolio standard deviations.
Figure A3. Portfolio standard deviations.
Econometrics 04 00003 g005
Figure A4. VaR: Univariate models.
Figure A4. VaR: Univariate models.
Econometrics 04 00003 g006aEconometrics 04 00003 g006b
Figure A5. VaR: Multivariate models.
Figure A5. VaR: Multivariate models.
Econometrics 04 00003 g007

D. The Univariate NCT-APARCH Model

The main ingredients of the NCT-APARCH model discussed in the paper by Krause and Paolella [16] are, as the name may suggest, the use of a univariate noncentral t-distribution assumption for the error term and of an APARCH specification for the conditional variance equation.
The NCT density function is given as follows:
f Z t ; ν , γ = e γ 2 / 2 Γ ( ( ν + 1 ) / 2 ) ν ν / 2 π Γ ( ν / 2 ) 1 ν + t 2 ν + 1 2 × i = 0 ( t γ ) i i ! 2 t 2 + ν i / 2 Γ { ( ν + i + 1 ) / 2 } Γ { ( ν + 1 ) / 2 } ,
where ν > 0 denotes the degrees of freedom parameter and γ is the noncentrality parameter dictating the degree of asymmetry (with the Student distribution recovered when γ = 0 ).
The evolution of the conditional variance is modeled according to the APARCH model, which allows for both heavy-tails and asymmetry. It is defined as follows:
σ t δ = σ ¯ + a 1 ( | ϵ t 1 | γ 1 ϵ t 1 ) δ + b 1 σ t 1 δ
where a 1 > 0 , b 1 0 , δ > 0 and | γ 1 | < 1 . In order to speed up the estimation procedure, the authors suggest to fix some parameters before the maximization of the likelihood function. In this paper, we calibrated the model parameters on our dataset in order to achieve the highest in-sample likelihood value (this applies also for the model incorporating the GARCH specification). Specifically, we set σ ¯ = 0 . 01 , a 1 = 0 . 05 , b 1 = 0 . 90 , δ = 2 and γ 1 = 0 . 4 . The vector of parameters to be estimated, then, reduces to ψ = ( μ , ν , γ ) , where E ( y t ) = μ denotes the location coefficient which is fastly estimated using the method of trimmed mean. We refer to the original paper by Krause and Paolella [16] for further details on the implemented estimation procedure.

E. Backtesting VaR: Augmented Independence and Conditional Coverage Tests

Table A2 and Table A3 report test-statistics and corresponding p-values obtained from the Independence (IND) and Conditional Coverage (CC) tests of Christoffersen (see Section 4.2) computed with a number of lagged observations K > 1 .
The tests are built in an extended Markov framework that allows for higher, or K-th order, dependence in the VaR observations, thus highlighting possible clustering in the series of violations. The distribution of the generalized IND and CC tests is asymptotically χ ( 1 ) 2 and χ ( 2 ) 2 , respectively. For further details on the construction of the tests we remand the interested reader to theoretical framework developed in Pajhede [39].
In order to make these results directly comparable with those given in Table 7 and Table 8, we set the number of lags K = 3 and run the tests for the 5% and 1 % VaR confidence level.
Results for both univariate and multivariate specifications are reported below.
Table A2. Augmented IND and CC test results. Univariate models.
Table A2. Augmented IND and CC test results. Univariate models.
NormSkew-NormStudentSkew-StudentMEPSkew-MEPNCT-APARCHNCT-GARCH
5% VaR
IND (K = 3) 0 . 843 ( 0 . 358 ) 0 . 1494 ( 0 . 699 ) 2 . 294 ( 0 . 130 ) 1 . 0807 ( 0 . 299 ) 1 . 802 ( 0 . 179 ) 0 . 0091 ( 0 . 924 ) 0 . 182 ( 0 . 670 ) 0 . 576 ( 0 . 448 )
CC (K = 3) 7 . 779 ( 0 . 020 ) 1 . 233 ( 0 . 540 ) 10 . 062 ( 0 . 007 ) 2 . 596 ( 0 . 273 ) 8 . 739 ( 0 . 013 ) 0 . 511 ( 0 . 775 ) 3 . 419 ( 0 . 181 ) 8 . 344 ( 0 . 015 )
1% VaR
IND (K = 3) 0 . 003 ( 0 . 955 ) 0 . 2371 ( 0 . 626 ) 0 . 237 ( 0 . 626 ) 3 . 0345 ( 0 . 082 ) 0 . 038 ( 0 . 844 ) 0 . 408 ( 0 . 523 ) 1 . 288 ( 0 . 256 ) 0 . 408 ( 0 . 523 )
CC (K = 3) 7 . 030 ( 0 . 030 ) 3 . 253 ( 0 . 197 ) 3 . 253 ( 0 . 197 ) 3 . 658 ( 0 . 161 ) 5 . 579 ( 0 . 061 ) 2 . 410 ( 0 . 300 ) 1 . 436 ( 0 . 488 ) 2 . 410 ( 0 . 300 )
Table A3. Augmented IND and CC test results. Multivariate models.
Table A3. Augmented IND and CC test results. Multivariate models.
NormSkew-normStudentSkew-StudentMEPSkew-MEP
5% VaR
IND (K = 3) 1 . 204 ( 0 . 272 ) 1 . 105 ( 0 . 293 ) 0 . 940 ( 0 . 332 ) 1 . 142 ( 0 . 285 ) 0 . 843 ( 0 . 358 ) 3 . 453 ( 0 . 063 )
CC (K = 3) 8 . 972 ( 0 . 011 ) 7 . 251 ( 0 . 027 ) 11 . 451 ( 0 . 003 ) 1 . 434 ( 0 . 488 ) 7 . 779 ( 0 . 020 ) 5 . 327 ( 0 . 070 )
1% VaR
IND (K = 3) 6 . 349 ( 0 . 012 ) 1 . 965 ( 0 . 161 ) 4 . 765 ( 0 . 029 ) 1 . 288 ( 0 . 256 ) 4 . 896 ( 0 . 027 ) 3 . 234 ( 0 . 072 )
CC (K = 3) 20 . 608 ( 0 . 000 ) 14 . 237 ( 0 . 001 ) 15 . 166 ( 0 . 001 ) 1 . 436 ( 0 . 488 ) 15 . 298 ( 0 . 000 ) 5 . 671 ( 0 . 058 )
Overall, adding more lags in the regression equation does not seem to crucially affect the outcome of the tests, as the new results are still in line with those obtained under K = 1 in Table 4 and Table 5. The main difference registered for both univariate and multivariate approaches is in the outcome of the IND test, which now all models in the tables pass at the standard 5 % level for both the 5% and 1% VaR. Apparently, violations are not clustered in time for more than one or two lags.
As for the CC test, results concerning the multivariate models are basically the same obtained by setting K = 1 , while in the univariate case we observe an improvement in the performance of the models (excluding the normal) for the 1 % VaR.
Ultimately, univariate distributions featuring skewness and heavy tails confirm their predominance over the remaining alternatives in terms of better predictive ability. In the multivariate case this is particularly true for the skew-Student and the skew-MEP, as the skew-normal is not leading to remarkable improvements over the corresponding normal assumption.

References

  1. R. Engle. “Autoregressive Conditional Heteroskedasticity with Estimates of United Kindgom Heteroskedasticity.” Econometrica 50 (1982): 987–1007. [Google Scholar] [CrossRef]
  2. T. Bollerslev. “Generalized Autoregressive Conditional Heteroskedasticity.” J. Econom. 31 (1986): 307–327. [Google Scholar] [CrossRef]
  3. T. Angelidis, A. Benos, and S. Degiannakis. “The Use of GARCH Models in VaR Estimation.” Stat. Methodol. 1 (2004): 105–128. [Google Scholar] [CrossRef]
  4. D.B. Nelson. “Conditional heteroskedasticity in asset returns: A new approach.” Econometrica 59 (1991): 347–370. [Google Scholar] [CrossRef]
  5. S. Mittnik, and M. Paolella. “Conditional density and value-at-risk prediction of Asian currency exchange rates.” J. Forecast. 19 (2000): 313–333. [Google Scholar] [CrossRef]
  6. Z. Ding, C.W. Granger, and R.F. Engle. “A long memory property of stock market returns and a new model.” J. Empir. Financ. 1 (1993): 83–106. [Google Scholar] [CrossRef]
  7. K. Kuester, S. Mittnik, and M. Paolella. “Value-at-risk prediction: A comparison of alternative strategies.” J. Financ. Econom. 4 (2006): 53–89. [Google Scholar] [CrossRef]
  8. F. Longin, and B. Solnik. “Is the correlation in international equity returns constant: 1960–1990? ” J. Int. Money Financ. 14 (1995): 3–26. [Google Scholar] [CrossRef]
  9. C. Brooks, and G. Persand. “Value-at-risk and market crashes.” J. Risk 2 (2000): 5–26. [Google Scholar]
  10. L. Bauwens, S. Laurent, and J. Rombouts. “Multivariate GARCH models: A Survey.” J. Appl. Econom. 21 (2006): 79–109. [Google Scholar] [CrossRef]
  11. P. Giot, and S. Laurent. “Value-at-risk for long and short trading positions.” J. Appl. Econom. 18 (2003): 641–663. [Google Scholar] [CrossRef]
  12. A.A. Santos, F.J. Nogales, and E. Ruiz. “Comparing univariate and multivariate models to forecast portfolio value-at-risk.” J. Financ. Econom. 11 (2013): 400–441. [Google Scholar] [CrossRef]
  13. L. Bauwens, and S. Laurent. “A New Class of Multivariate Skew Densities, with Application to GARCH Models.” J. Bus. Econ. Stat. 23 (2005): 346–354. [Google Scholar] [CrossRef]
  14. C. Ley, and D. Paindaveine. “Multivariate skewing mechanisms: A unified perspective based on the transformation approach.” Stat. Probab. Lett. 80 (2010): 1685–1694. [Google Scholar] [CrossRef]
  15. D. Noureldin, N. Shephard, and K. Sheppard. “Multivariate rotated ARCH models.” J. Econom. 179 (2014): 16–30. [Google Scholar] [CrossRef]
  16. J. Krause, and M.S. Paolella. “A fast, accurate method for value-at-risk and expected shortfall.” Econometrics 2 (2014): 98–122. [Google Scholar] [CrossRef] [Green Version]
  17. P.F. Christoffersen. “Evaluating interval forecasts.” Int. Econ. Rev. 39 (1998): 841–862. [Google Scholar] [CrossRef]
  18. E. Dumitrescu, C. Hurlin, and V. Pham. “Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests.” Finance 33 (1995): 79–111. [Google Scholar]
  19. P.H. Kupiec. “Techniques for verifying the accuracy of risk measurement models.” J. Deriv. 3 (1995): 73–84. [Google Scholar] [CrossRef]
  20. R.F. Engle, and S. Manganelli. “CAViaR: Conditional autoregressive value at risk by regression quantiles.” J. Bus. Econ. Stat. 22 (2004): 367–381. [Google Scholar] [CrossRef]
  21. V. DeMiguel, L. Garlappi, and R. Uppal. “Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? ” Rev. Financ. Stud. 22 (2009): 1915–1953. [Google Scholar] [CrossRef]
  22. J. Tu, and G. Zhou. “Markowitz meets Talmud: A combination of sophisticated and naive diversification strategies.” J. Financ. Econ. 99 (2011): 204–215. [Google Scholar] [CrossRef] [Green Version]
  23. S.J. Brown, I. Hwang, and F. In. “Why Optimal Diversification Cannot Outperform Naive Diversification: Evidence from Tail Risk Exposure.” , 2013. [Google Scholar] [CrossRef]
  24. C. Fugazza, M. Guidolin, and G. Nicodano. “Equally Weighted vs. Long-Run Optimal Portfolios.” Eur. Financ. Manag. 21 (2014): 742–789. [Google Scholar] [CrossRef]
  25. L. Bauwens, W. Omrane, and E. Rengifo. Intra-Daily FX Optimal Portfolio Allocation. CORE Discussion Paper 2006/10; Louvain-la-Neuve, Belgium: CORE, 2006. [Google Scholar]
  26. R.F. Engle, and K.F. Kroner. “Multivariate simultaneous generalized ARCH.” Econometr. Theory 11 (1995): 122–150. [Google Scholar] [CrossRef]
  27. O. Barndorff-Nielsen. “Exponentially decreasing distributions for the logarithm of particle size.” Proc. R. Soc. Lond. A Math. Phys. Eng. Sci. 353 (1977): 401–419. [Google Scholar] [CrossRef]
  28. M.S. Paolella, and P. Polak. “COMFORT: A Common Market Factor Non-Gaussian Returns Model.” J. Econom. 187 (2015): 593–605. [Google Scholar] [CrossRef]
  29. C. Fernández, and M.F. Steel. “On Bayesian modeling of fat tails and skewness.” J. Am. Stat. Assoc. 93 (1998): 359–371. [Google Scholar]
  30. T. Bollerslev, and J. Wooldridge. “Quasi-Maximum Likelihood Estimation and Inference in Dynamic Models with Time-Varying Covariances.” Econom. Rev. 1 (1992): 143–172. [Google Scholar] [CrossRef]
  31. K. Fang, S. Kotz, and K. Ng. Symmetric Multivariate and Related Distributions. London, UK: Chapman and Hall, 1990. [Google Scholar]
  32. N. Solaro. “Random variate generation from multivariate exponential power distribution.” Stat. Appl. II 2 (2004): 25–44. [Google Scholar]
  33. D. Noureldin, N. Shephard, and K. Sheppard. “Multivariate high-frequency-based volatility (HEAVY) models.” J. Appl. Econom. 27 (2012): 907–933. [Google Scholar] [CrossRef]
  34. P. Christoffersen. “Evaluating Interval Forecasts.” Int. Eco. Rev. 39 (1998): 841–862. [Google Scholar] [CrossRef]
  35. P. Kupiec. “Techniques for Verifying the Accuracy of Risk Management Models.” J. Deriv. 23 (1995): 73–84. [Google Scholar] [CrossRef]
  36. S. Campbell. “A Review of Backtesting and Backtesting Procedures.” J. Risk 9 (2006): 1–17. [Google Scholar]
  37. R. Engle, and S. Manganelli. “CaViaR: Conditional Autoregressive Value at Risk by Regression Quantiles.” J. Bus. Econ. Stat. 22 (2004): 367–381. [Google Scholar] [CrossRef]
  38. P. Christofferson, and D. Pelletier. “Backtesting Value-at-Risk: A Duration-Based Approach.” J. Financ. Econom. 2 (2004): 84–108. [Google Scholar] [CrossRef]
  39. T. Pajhede. Backtesting Value-at-Risk: The generalized Markov Test. U. Copenhagen Economics Discussion Paper 15-18; Copenhagen, Denmark: University of Copenhagen, 2015. [Google Scholar]
  • 1.The interested reader is referred to Ley and Paindaveine [14] who provide a detailed overview of this approach.
  • 2.We thank the authors for kindly providing us their MATLAB codes.
  • 3.In Appendix D we provide a brief overwiew of the noncentral t distribution and its likelihood derivation.
  • 4.In the univariate case it corresponds to ψ = ( a 1 , b 1 ) .
  • 6.We thank an anonymous referee who pointed out the possibility of better comparing the outcomes of the DQ and Christoffersen’s CC and IND tests by allowing the latter to be computed in an extended framework than the standard one described in Section 3.2. As results did not lead to significant improvements, this issue is briefly covered in Appendix E.

Share and Cite

MDPI and ACS Style

Braione, M.; Scholtes, N.K. Forecasting Value-at-Risk under Different Distributional Assumptions. Econometrics 2016, 4, 3. https://0-doi-org.brum.beds.ac.uk/10.3390/econometrics4010003

AMA Style

Braione M, Scholtes NK. Forecasting Value-at-Risk under Different Distributional Assumptions. Econometrics. 2016; 4(1):3. https://0-doi-org.brum.beds.ac.uk/10.3390/econometrics4010003

Chicago/Turabian Style

Braione, Manuela, and Nicolas K. Scholtes. 2016. "Forecasting Value-at-Risk under Different Distributional Assumptions" Econometrics 4, no. 1: 3. https://0-doi-org.brum.beds.ac.uk/10.3390/econometrics4010003

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop