Permutation Entropy and Information Recovery in Nonlinear Dynamic Economic Time Series

Abstract

The focus of this paper is an information theoretic-symbolic logic approach to extract information from complex economic systems and unlock its dynamic content. Permutation Entropy (PE) is used to capture the permutation patterns-ordinal relations among the individual values of a given time series; to obtain a probability distribution of the accessible patterns; and to quantify the degree of complexity of an economic behavior system. Ordinal patterns are used to describe the intrinsic patterns, which are hidden in the dynamics of the economic system. Empirical applications involving the Dow Jones Industrial Average are presented to indicate the information recovery value and the applicability of the PE method. The results demonstrate the ability of the PE method to detect the extent of complexity (irregularity) and to discriminate and classify admissible and forbidden states.

1. Introduction

The 1000-point collapse of the Dow Jones Industrial Average on 6 May 2010 “… was a small indicator of how complex and chaotic, in the formal sense, these systems have become …” Ben Bernanke, Interview with the International Herald Tribune, 17 May 2010

Economic behavioral processes and systems have the interesting characteristic of being stochastic, dynamic, seldom in equilibrium and not subject to a unique time invariant econometric model. Although the study of such systems has received a great deal of attention and various researchers have sought to develop economic tools to capture the complexity of real-world economic phenomena (LeBaron and Tesfatsion 2008), their underlying complex dynamics behavior is not well understood and thus, it is difficult to model econometrically. Understanding and predicting the complex behavior of such stochastic economic behavior systems may best be considered in a probabilistic context and its dynamics should be measured against a statistical economic equilibrium. Although complexity requires various forms (see Rosser 1999), we follow a “broad tent” definition within economics in this paper, which was given by Day and Mizrach (1994). Accordingly, we define complexity as some form of erratic dynamic behavior arising in nonlinear dynamical systems where this does not asymptotically tend to a fixed point or to a non-oscillating growth or decline due to endogenous causes arising from within itself.

Over time, economists have been interested in the informational content of time series data (see e.g., Juselius and Johansen 2006) and in recognizing economic downturns (see e.g., Barsky and de Long 1993). Many methods have been proposed for the proper analysis of the time series probability space and the particular characteristics of the data. Indeed, time series models are traditionally expressed in terms of reductionist parametric models and rely on estimation and inference methods that seek to understand the behavior of the whole from that of its parts. Recovering information about the unknowns from indirect noisy observations based on observable data is usually based on parametrized functions and a range of observed data sampling processes, which involves a finite set of indirect noisy observations that fall in the effect domain. Higher order moments are commonly assumed to be well-behaved, which is an assumption often violated in real-world data, where heavy-tailed distributions prevail. The resulting econometric formulations usually do not capture important non-linear model components and important microscopic details, leading to a blurred and limited vision at the micro and macro levels. In this paper, our interest is in the causal domain (Judge 2016).

Early attempts to quantitatively assess some of the information in the economic and financial time series focused on reducing the observed system outcomes of a possible disequilibrium world to a stationary world in the form of univariate or multivariate moving average-autoregressive ARMA representations. Each of these linear parametric time series models contain the stationary and equilibrium characteristics of the data. In one way or another, the future is viewed as a function of time, where lags that are functions of the known past are artificially reversed and added to the present value of the function—the future is now. The function of dynamics in this process that is associated with stationary probability is to connect temporal information from the behavioral environment to system outcomes later on in time. Despite the many productive efforts in this area of econometric reductionist modeling with a time series, the hidden dynamic nonlinear nonstationary temporal patterns underlying the time-dated outcomes have often remained hidden and have not provided a reliable basis for understanding the current economic behavioral processes and systems (Stiglitz 2018).

Against this background, we focus on an information theoretic-symbolic logic complexity approach in this paper, which resembles the method taken in ordinal time series analysis (cf. Bandt 2005; Bandt and Shiha 2007; Cao et al. 2004; Hou et al. 2017; Keller and Sinn 2005; Kowalski et al. 2012; Zanin et al. 2012; Zunino et al. 2009, 2010b). This aims to unlock the complex and hidden dynamical behaviors contained in the nonlinear time series. This entails two tasks. We first capture the permutation patterns–ordinal relations among the individual values of a given time series, which concerns the temporal information from the dynamical properties of the economic system. After this, we extract an ordinal pattern probability distribution, whose elements are the frequencies associated with the admissible permutation patterns. Thus, this permits estimation of the complexity measure.

Unlike the reductionist time series econometric approaches, this theoretic-symbolic basis can be applied to any type of time series (regular, chaotic, noisy, experimental or reality based), which has weak stationary assumptions, is conceptually simple and computationally fast. The complexity of a given time series is quantified by extracting qualitative information (temporal structural diversity), avoiding restrictive parametric model assumptions and recognizing the temporal ordering structure (time causality) of the given time series. More precisely, we adopt the Permutation Entropy (PE) method, which relies upon the notions of entropy and symbolic dynamics. Compared to other entropy measures, such as the Kolmogorov–Sinai entropy, PE does not require a long time series. The notion of entropy was initially introduced in communication theory by Shannon (1948) and reflects the degree of uncertainty (information content) associated with an unknown probability distribution. The relevant reviews of entropy measures and their econometric applications include Golan et al. (1996), Ullah (1996) and Judge and Mittelhammer (2012a, 2012b). The symbolic dynamics, which is a mathematical approach first proposed by Morse and Hedlund (1938) to analyze general dynamical systems by discretizing space into a number of symbol sequences with a length of D (blocks) that captures the order relations (permutation ranks) between the values, is used in this paper as a tool to provide a simplified picture of complicated nonequilibrium economic behavior systems. Schittenkopf et al. (2002) suggest that symbolic information processing represents a promising approach to prediction tasks regarding the hypothesis of efficient capital markets. Using symbolic dynamics, they predicted the daily change in the volatility of German and the British stock markets. Mensi et al. (2014), Matilla-García and Marín (2010), Matilla-García (2007) and Stutzer (1980) constitute some other applications of the concept of symbolic dynamics in the economics literature. Ordinal patterns–permutations that arrange the values of a time series according to their order are used by Bandt (2005) and Groth (2005) as a robust method with respect to nonlinear perturbations, which describe the intrinsic patterns that are hidden in dynamics economic systems.

Looking Ahead

The definition of ordinal patterns adopted in this paper follows Bandt and Pompe (2002) and the information content about the time series is conveyed in the form of a probability density function (PDF). Empirical applications involving a complex system, such as the most widely quoted high-frequency intraday transaction Dow Jones Industrial Average (DJIA) (Serletis 2016), are conducted to indicate the information recovery value and the applicability of the PE method. Alvarez-Ramirez and Rodríguez (2011) already analyzed the dynamics of the DJIA. Nevertheless, they used the approximate entropy, which is a non-ordinal probabilistic approach developed by Pincus (1991) and rooted in the Eckmann and Ruelle (1985). To the best of our knowledge, our study is the first to employ PE involving DJIA for indicating the information recovery value and applicability of the PE method. In Section 2, we introduce the PE methodology and include an illustration of the calculation of the order relations (permutations) with simple examples. In Section 3, we describe the information-theoretic entropy framework associated with the permutation patterns and the PE metric measure. In Section 4, an empirical application of the PE to DJIA data is exhibited. We conclude with a summary of our findings and implications in Section 5.

2. Permutation Entropy and Ordinal Patterns

The PE method is characterized by its conceptual simplicity and computational speed. It does not presuppose any model based assumptions, including whether the model is nonlinear and ordinal or is invariant under any monotonic transformation of the data. This comes in metric (see Bandt and Pompe 2002) and topological (see Bandt et al. 2002) versions. Although the topological approach can be intuitively appealing and lends itself to suggestive graphical representations, this paper focuses on the metric version, which is a more powerful and effective approach when the underlying fundamental research questions relate to complexity and predictability as well as discovering the relation between deterministic and stochastic systems (Barnett et al. 2012).

PE provides a basis for the analysis of a nonlinear time series and permits a way of describing the underlying dynamic state of the system in probability distribution form. The objective in using this technique is to extract qualitative information from the nonlinear time series in the form of temporal dynamics. The PE idea is based on the permutation patterns–ordinal relations among values of a time series, which concerns temporal information derived from the dynamic properties of the economic system. The complexity of an economic system is reflected in terms of accessible ordinal patterns hidden in the process system (for example, see the study of ordinal patterns by Amigó et al. 2010).

To provide a probability distribution of the temporal dynamics that is linked to the sample space, Bandt and Pompe (2002) proposed a method that takes time causality into account by comparing time related observations–ordinal patterns in a time series. They consider the order relation between time series instead of the individual values. Permutation patterns–partitions–vectors (symbol sequences) are developed by comparing the order of neighboring observations. A vector of the $D$th subsequent values is constructed, where $D$ is the embedding dimension that determines how much information is contained in each vector with which an ordinal pattern (permutation) is associated. The values of each vector are sorted in ascending order and a permutation of $D!$ partitions are created. Patterns of occurrence do not have the same probability and thus, information is revealed concerning the underlying dynamic system. Thus, using a member of the Cressie-Read (CR) family with each time series, it is feasible to associate a probability distribution whose frequencies are related with the possible permutation patterns (see Section 3). The CR family is a general and flexible family of the power divergence-goodness-of-fit measures proposed by Cressie and Read (1984) and Read and Cressie (1988). This general entropy measure was used by Cressie and Read (1984) as a non-parametric measure of the discrepancy between the distributions p and q so that it is referred as a power divergence measure.

More specifically, in order to use the Bandt and Pompe (2002) PE methodology for evaluating the probability distribution $P$ associated with the time series dynamical system under study, we start by considering the partitions of the pertinent $D$-dimensional space that will hopefully “reveal” relevant details of the ordinal structure of a given one-dimensional time series $\mathcal{S}\left(t\right)=\left\{x_t;t=1,\cdots ,T\right\},$ with an embedding dimension $D>1$ and time delay $\tau$ between $x_t$ values in the symbol sequences. In this paper, $\tau$ is kept fixed to the unity to avoid missing any patterns. Bandt and Pompe (2002), and Bandt (2005) justify using $\tau =1$ when analyzing today’s unemployment rate or the Dow Jones index. For a review on the choice of this parameter, see Riedl et al. (2013).

In this paper, we are interested in “ordinal patterns”, of order $D$, which is generated by:

$$\left(s\right)\mapsto \left(x_{s-\left(D-1\right)},x_{s-\left(D-2\right)},\cdots ,x_{s-1},x_s\right),$$

(1)

This assigns to each time $s$, which is the $D$-dimensional vector of values at times $s,s-1,\cdots ,s-\left(D-1\right)$. Clearly, a greater $D$ value results in more information on the past being incorporated into our $D$-dimensional vectors. By the “ordinal pattern” related to the time $\left(s\right)$, we mean the permutation $\pi =\left(r_0,r_1,\cdots ,r_{D-1}\right)$ of $\left[0, 1,\cdots , D-1\right]$, which is defined by:

$$x_{s-r_{D-1}}\le x_{s-r_{D-2}}\le \cdots \le x_{s-r_1}\le x_{s-r_0}.$$

(2)

In order to get a unique result, we can set $r_i<r_{i-1}$, if $x_{s-r_i}=x_{s-r_{i-1}}$. This is justified if the values of $x_t$ have a continuous distribution so that equal values are very unusual. Otherwise, it is possible to break these equalities by adding small random perturbations. Thus, for all the $D!$ possible permutations $\pi$ of order $D$, the associated relative frequencies can naturally be computed by the number of times. This particular order sequence is found in the time series divided by the total number of permutation sequences. The probability distribution $P=\left\{p\left({\pi }_i\right)\right\}$ is defined by:

$$p\left({\pi }_i\right)=\frac{♯\left\{s|s\le T-\left(D-1\right);\left(s\right) has type {\pi }_i\right\}}{T-\left(D-1\right)},$$

(3)

where the symbol $♯$ stands for the “number” of elements in it.

The procedure for the calculation of the ordinal patterns (permutations) is illustrated here and in an Appendix A with simple examples. First, assume that we start with the time series $\left\{1,3,5,4,2,5,\dots \right\}$ and we set the embedding dimension $D=4$. In this case, the state space is divided into $4!$ partitions and $24$ mutually exclusive permutation symbols are considered. The first 4-dimensional vector is $\left(1,3,5,4\right)$. According to Equation (2), this vector corresponds with (x_s−3, x_s−2, x_s−1, x_s). Following Equation (3), we found that $x_{s-3}\le x_{s-2}\le x_s\le x_{s-1}$. After this, the ordinal pattern that allows us to fulfill Equation (2) is $\left[3,2,0,1\right]$. The second 4-dimensional vector is $\left(3,5,4,2\right)$ and $\left[0,3,1,2\right]$ will be its associated permutation and so on. A graphical example for $D=3$ is presented in Appendix A.

For the computation of the probability distribution $P$ represented in Equation (3) and presented in Appendix B, we use the very fast and computationally efficient pentropy algorithm developed by Clower and Henry (2019), in which the equal values have been numerically broken by adding white noise to the series, with the Gaussian noise being smaller than the smallest distance between values.

The PE methodology is not restricted to the time series that is representative of low dimensional dynamical systems. Indeed, it can be applied to any type of time series (regular, chaotic, noisy, experimental or reality based), with a very weak stationary assumption: for $k \le D$, the probability of finding $x_t<x_{t+k}$ should not depend on $t$ (Bandt and Pompe 2002). It is assumed that enough data are available for a correct embedding procedure. The embedding dimension $D$ plays an important role in the evaluation of the appropriate probability distribution because $D$ determines the number of accessible $D!$ states. Furthermore, it is conditioned by the minimum acceptable length $T\gg D!$ of the time series that one needs in order to extract reliable statistics. Moreover, the choice of $D$ is relevant in detecting the dynamic structure of the data (Kantz and Schreiber 2004).

In terms of the statistical inference, there are simple, consistent and powerful tests of independence that provide a basis for using PE as a measure of serial dependence (see Matilla-García and Marín 2008, 2009; Canovas and Guillamon 2009). The resulting test statistic has been used to explore possible serial dependences in several financial returns and sovereign markets, such as the DJIA, S&P 500 and CDS (see Sensoy et al. 2017). Lee (2012) investigates the potential of the PE as an early warning signal temporal dependence measure in the financial markets. Other PE developments include tests to assess spurious seasonality effects in the time series (Bariviera et al. 2018), correlation type functions for a long time series (Bandt 2016) and combined complexity–entropy metrics as an empirical device to characterize the complex time series (Zunino et al. 2010a; Ribeiro et al. 2017).

3. Information Theoretic Estimation and Inference Base

As indicated previously, the time series analysis requires the solution of a stochastic inverse problem in order to identify the underlying economic dynamic system based on indirect noisy effects. One natural solution to cope with the estimation and inference problems noted above is to make use of the estimation and inference methods that are designed to deal with the stochastic ill-posed inverse problem and the disequilibrium nature of econometric models. In this context, the CR family of power divergences encompassing a family of likelihood functionals provides one basis for linking the data and the sampling model of the process. This permits to exploit the statistical machinery of information theory to gain insights related to the behavior of an economic process from a sample of data from a system that may not be in equilibrium. In developing this information theoretic econometric approach to estimation and inference, the CR single parameter family of the informational functionals (see Judge and Mittelhammer 2012a, 2012b) represents a way to link the family of possible likelihood functions to the sampling model of the process. Information functionals of this type have an intuitive interpretation reflecting uncertainty as it relates to a model of the process and a model of the data for processes that are possibly out of equilibrium. In addition to its optimality base, an advantage of this approach is that it permits the possibility of non-standard distributions.

The Cressie–Read Family of Power Divergence Measures and the PE metric

In identifying the estimation and inference measures that may be used as a basis for characterizing the data sampling process for indirect noisy observed data outcomes, we begin with the CR (1984)’s multi-parametric family of goodness-of-fit-power divergence measures defined as:

$$I\left(p,q,\gamma \right)=\frac{1}{\gamma \left(\gamma +1\right)}{\displaystyle \sum _{i=1}^n{p}_i}\left[{\left(\frac{p_i}{q_i}\right)}^{\gamma }-1\right]$$

(4)

In Equation (4), $\gamma \in \left(-\infty ,\infty \right)$ is a scalar power parameter that indexes the members of the CR family, $p_i$’s represent the subject probabilities and $q_i$’s are interpreted as reference probabilities. Being probabilities, the usual probability distribution characteristics of $p_i,q_i\in \left[0,1\right]\forall i,{\displaystyle \sum _{i=1}^n{p}_i=1}$ and ${\displaystyle \sum _{i=1}^n{q}_i=1}$ are assumed to remain the same. The choice of the reference distribution is equivalent when choosing the empirical distribution function (EDF).

The CR family of power divergences leads to a broad family of likelihood functions. In the context of extremum metrics, the maximum likelihood (ML) is embedded in the CR family of the power divergence statistics. As γ varies, the resulting CR family of estimators that minimize power divergence exhibit qualitatively different sampling behavior. This class of estimation procedures is referred in the literature to as Minimum Power Divergence (MPD) estimation (see Gorban et al. 2010; Judge and Mittelhammer 2012a, 2012b). An empirical application of the new ML-MPD binary response estimator developed by Mittelhammer and Judge (2011) is given by Henry et al. (2018).

The CR family of the divergence measures in Equation (4) permits us to exploit the statistical machinery of information theory to gain an insight into the static PDF behavior of economic systems and processes. In an extremum metrics context, the CR entropy power divergence family represents a basis for deriving empirical probabilities associated with the indirect noisy micro and macro data. As demonstrated by Gorban et al. (2010), the CR and entropy families are equivalent over the defined ranges of the divergence measures. Using a uniform refence distribution, such that $q=n^{-1}{1}_n$ and applying the $\underset{\gamma \to 0}{\lim }CR\left(\gamma \right)\equiv \underset{\gamma \to 0}{\lim }\mathrm{I}\left(p,q,\gamma \right)$, it is possible to create a probability distribution, whose elements $p_i\equiv p\left({\pi }_i\right)$ are the frequencies associated with the ith permutation pattern, i = 1,2, …, $D$!. The information content of such a distribution for the $D$! distinct assessable states is what Bandt and Pompe (2002) defined as the PE of order $D$ of the time series and is given by:

$$P{E}_D=-{\displaystyle \sum _{i=1}^{D!}{p}_i{\log }_2{p}_i}$$

(5)

with a normalized version defined by:

$$P{E}_{D,norm}=-\frac{1}{{\log }_{2}D!}{\displaystyle \sum _{i=1}^{D!}{p}_i{\log }_2{p}_i}$$

(6)

to the interval [0, 1]. It is important to notice that $P{E}_D\in \left[0,{\log }_{2}D!\right]$, where the maximum entropy value is realized when all $D!$ possible permutations have an equal probability of occurrence. A PE = 0 is reached for a completely predictable, regular system (i.e., a monotonically increasing or decreasing time series). When $P{E}_D<{\log }_{2}D!$, certain types of dynamics are realized from the time series. Independent of parameter choice and the length of the time series, the normalization of Equation (5) is always bounded between 0 and 1. The $P{E}_{D,norm}$ gives a measure of the departure of the time series from a complete random time series. The closer to 1 the $P{E}_{D,norm}$, the more noise and stochastic the time series. Alternatively, a smaller $P{E}_{D,norm}$ results in the time series being more regular, more deterministic, more predictable and more market efficient (Lo 2008). It is important to note that the base of the logarithm used is 2 since the information conveyed by such distribution in Equation (5) and its estimation is encoded in bits (shorthand for “binary digits”).

4. Estimation and Empirical Applications

In this section, we investigate the applicability and usefulness of the PE metric in unlocking the complex and hidden dynamical behavior in temporal patterns (temporal structure diversity) in the high-frequency DJIA time series system. Before presenting our empirical results, we briefly discuss a key issue concerning the identification of dynamical changes and event detection when using the traditional PE procedure. To demonstrate these results, we use the daily returns on the entire DJIA data series and for two restricted post-World War II periods. We calculate the returns by continuously compounding the nominal daily DJIA price indices as the first difference of the natural logarithms of daily prices. Denoting $p_t$ as the price index for DJIA as time t, we define the following:

$$r_t=\ln \left(p_t\right)-\ln \left(p_{t-1}\right)$$

(7)

as the continuously compounded return for the DJIA price index at time t. Our particular focus is on the DJIA time period of 2007–2009. Computational implications are provided in Appendix D.

4.1. PE Information Recovery Estimation

In discussing the empirical implementation of the PE approach presented in previous sections, a basic but important limitation of the traditional procedure introduced by (Bandt and Pompe 2002) in 2002 concerns the examination of dynamical changes and event detection of abrupt changes over time in a time series. Regardless of the chosen embedding dimension and time delay parameter values, the original procedure of calculating the entropy based on the permutation patterns yields one single PE point estimate for the entire time series. The complexity of the whole dynamic system and its extracted PDF is represented by a single measure that captures the temporal information contained in the time series. As our purpose also involves the detection of dynamic changes over time, we use a rolling window analysis procedure to compute the time-varying estimates of PE, which capture the underlying dynamics and uncertainty. For some empirical applications of this rolling window procedure in the PE context, see Cao et al. (2004), Staniek and Lehnertz (2007), and Hou et al. (2017).

The procedure that we use partitions the entire one-dimensional time series $\mathcal{S}\left(t\right)=\left\{x_t;t=1,\cdots ,M\right\}$ into a number of overlapping segments of short length m, which shifts a h-step ahead and yields a data matrix with dimensions of $m\times \left(T-m+1\right)$. The size of the chosen fixed rolling window contains the number of consecutive time series observations for each m-dimensional vector. Provided that the windows are rolled through the sample $\mathcal{S}\left(t\right)$ one observation at a time, there are $T-m+1$ time-varying PE estimates to compute, where Equation (5) is applied to each segment. It is worth mentioning that the PE results are qualitatively unchanged when the window size is increased by a scalar multiple. Finally, we should mention that in Section 4.2 and Section 4.3, we start by estimating a single PE point estimate for the time series under consideration. After this, we apply the rolling window procedure described previously for each time series to compute the time-varying estimates of PE.

4.2. Analysis of the Full DJIA Time Series: 1901–2016

We begin our empirical analyses by using the full DJIA time series. These data are displayed in Figure 1 and contain the time series observations from 1901 to 2016.

As displayed in Figure 1, we can see the long run movement of daily $p_t$ and $r_t$ over the past 115 years. There is an upward trend for $p_t$ but $r_t$ is rather stable around the mean $\mu =0.00019$. From Figure 1a, the DJIA rose from 107.23 points in 1919 to a level of 248.5 points in 1929, just before the stock market crash—Depression in 1929. These facts are summarized in Carter et al. (2006). This economic event was of unprecedented dimensions and it reflected a unique failure of the industrial economy, where economic activity in the U.S. declined from the middle of 1929 through March 1933. Another major market action started around 1980 and the most dramatic changes in behavior occurred between 2007 and 2009.

Using the one-dimensional 115-year DJIA returns time series from Figure 1b, the traditional procedure introduced by Band and Pompe in 2002 (see Equation (5)) calculated a fixed time delay $\tau =1$, an embedding dimension of 4 and the (normalized) PE point estimate of 0.998. This entropy point estimate of 0.998 associated with the DJIA data (Figure 1b) reveals an extremely high degree of uncertainty and randomness (disorder). This is a manifestation of overall U.S. stock market efficiency and the complex information processing and dynamics hidden in the DJIA system. Conceptually, it denotes the average or expected uncertainty or surprisal value of the 4! distinct and mutually exclusive admissible ordinal patterns (see Appendix B), which arises naturally from the time series without model-based assumptions. It is important to note that the averaging or expectation operation involved in the entropy estimation in Equation (5) is taken with respect to the probability distribution on the permutations (Judge and Mittelhammer 2012a). This finding of a PE point estimate of 0.998 for DJIA nominal returns agrees with that of Hou et al. (2017), who estimates that the PE of DJIA is close to 1 during the 2008 financial crisis.

For the one-dimensional DJIA time series in Figure 1b, the estimation of the single PE point estimate ordinal sequences of 4-dimensional vectors were chosen to capture the information contained in the permutations of the distinct states. With a smaller embedding dimension, the degree of information would be very limited. More importantly, based on our investigations, we note that as $D$ increases, some ordinal patterns are missing and the computation time increases, causing memory restrictions (see Zanin (2008) and Zunino et al. (2009) on forbidden patterns). Our numerical PE results indicate that although the magnitude of the normalized PE reduces as the order $D$ of PE increases, the choice of the entropy order $D$ has very marginal influence on the final PE results (see Appendix C). According to Bandt and Pompe (2002), this justifies the use of a low entropy order.

4.3 Rolling Window Analysis

To detect the temporal changes of the complexity of the one-dimensional DJIA time series in Figure 1b and to identify the market events of interest using the PE concept, we apply the rolling window (RW) approach described in Section 4.1. In using the RW approach, m should be considerably larger than $D$ in order to estimate PE accurately (Bandt and Pompe 2002). Following Matilla-García and Marín (2008), the possible largest embedding dimension for m = 750 (three-year window) that satisfies the constraint that $5D!\le m$ is 5. Using a one day offset splits the time series into 750-day segments. This results in 30,746 segments of 750 trading days. On each segment, we applied the same PE function given in Equation (6) with $D\in \left\{4,5\right\}$ and $\tau =1$ and observed that although the series of PEs based on $D=5$ falls below (mean PE of 0.977) the other series with $D=4$ (mean PE of 0.990), both time-varying PE series behave in a similar way over time in the U.S. stock market. Figure 2 displays the normalized permutation entropy rolling estimates series for $D=5$ of daily DJIA nominal returns. This result provides some insight into the broad trends in the U.S. fiscal policy. The turmoil in the U.S. stock market and collapse of stock prices during the Depression at the end of 1929 is apparent in Figure 2. It is important to note an increase in the temporary uncertainty (rise in PE and complexity) and the steady demand for manufactured products during World War I. The PE over time reveals important volatility that corresponds to U.S. fiscal policy. The sharp increase in PE or equivalently the decrease in informational content of the series during the 1980s may be linked to deregulation during the Reagan administration. These diagnostics suggest potential questions that require rigorous analysis in order to posit a causal link. However, the value of examining the partitioned sequencing within the DJIA continuously compounded return series is immediately apparent in suggesting worthwhile questions for economic analysis.

4.4. Post-World War II Analysis

In this subsection, we restrict our analysis to the post-World War II period, which is a period characterized by speculative phases and by the recent and extraordinarily complex stock market recession event. This analysis is intended to give an intuitive grasp of the application of the PE approach across different and shorter analysis periods.

We begin by recovering the information content for a single PE point estimate for $D=4$ and its associated static ordinal pattern probability distribution over the restricted time period of 2000–2016 (see Figure 3). After this, we use the RW approach described in Section 4.1 to analyze this time period with a special focus on the 2007–2009 period. This time period involves observations where dramatic changes in behavior during economic fluctuations are a prevalent feature of the data.

Using a time delay of 1, an embedding dimension of 4 and the much shorter one-dimensional DJIA returns time series from 2000 to 2016 (Figure 3b), the resulting normalized PE quantity is 0.997. In Appendix B, we present the relative frequencies of each admissible permutation associated with this time series. During the 2007–2009 period of bear market conditions, surrounding the stock market recession, the normalized PE measure attains a value of 0.992. Clearly, these post-War World II PE results recognize the high degrees of uncertainty and lower degree of return predictability or, equivalently, the higher degree of weak-form market efficiency involved in the DJIA series during the period with major dramatic effects, since the Great Depression in the 1930s.

Compared to the longer period (1901–2016), the post-War World II period (2000–2016)’s PE results are slightly smaller in magnitude across the different embedding dimensions ($D$ = 4, 5 and 6). However, they are consistently similar when $D=4$ even when including the shortest period of 2007–2009 (see Appendix C).

4.5. Rolling Window Analysis

In Figure 4, we use an entropy-based procedure that implements the RW method and a length m of 750 that shifts one day at a time (h-step = 1) over this shorter post-War World II period. The variation of 3532 rolling 36-month (normalized) PEs for a fixed time delay $\tau =1$ and embedding dimension $D$ = 5 is displayed in Figure 4. The complexity and degree of disorder of the frequencies $p(\pi )$ of the $D$ ordinal patterns is quantified by the informational content of the distribution.

An interesting aspect of Figure 4 is that PEs attain the historically lowest level before the Recession, with a normalized PE of 0.970. Thus, the daily DJIA dynamics became more predictable during these periods and after the bear market conditions, where PEs start dropping at the end of 2010. The normalized PE, right after the DJIA hits an intraday peak (the left dash red vertical line in Figure 4) of 14,164.53 on 9 October 2007, clearly exhibits a relative increase with no significant drop in its distribution during the bear market conditions. During this period, the downward trend in PE is evident for the remaining of the stock market recession up to a turning point in the stock market. September 2008 is revealed as the deepest stage in the financial crisis, whose critical financial intermediaries failed or were bailed out. The PE method provides a precise new way to view, predict and analyze financial data.

5. Concluding Remarks

In this paper, we recognize that the economic behavioral processes and systems are seldom in equilibrium and the new methods of modeling and information recovery are needed to explain the hidden dynamic economic world of interest. To reflect this dynamic situation, we propose a fast and robust method for extracting qualitative information from non-linear dynamic economic time series observations, which focuses from an ordinal viewpoint. Ordinal patterns are used to describe the intrinsic patterns hidden in the dynamics of economic systems. The concept of PE considers the order relation between the values of a time series instead of the actual values and permits one to obtain a distribution of the ordinal accessible patterns and quantify the complexity of a system. Since PE is nonlinear, ordinal and model-free, it can be applied to a range of regular, noisy and chaotic time series. The empirical applications on the DJIA are given to demonstrate that the PE method permits the identification of dynamic structure and an ability to discriminate and classify accessible and forbidden states. Looking ahead, we hope to conduct a comparison with other alternative measures of divergence such as $\gamma =-1$ (the empirical likelihood) or a convex combination of $\gamma =0\mathrm{and}-1$. We also hope to extend the use of nonlinear dynamics in the time series to predict the future out of sample behavior of an economic system. We also hope to use the PE concept to analyze the connection between the micro and macro dynamic economic systems and multivariate PE to measure the complexity of multivariate economic systems.