Information Length Analysis of Linear Autonomous Stochastic Processes

Abstract

When studying the behaviour of complex dynamical systems, a statistical formulation can provide useful insights. In particular, information geometry is a promising tool for this purpose. In this paper, we investigate the information length for n-dimensional linear autonomous stochastic processes, providing a basic theoretical framework that can be applied to a large set of problems in engineering and physics. A specific application is made to a harmonically bound particle system with the natural oscillation frequency $\omega$, subject to a damping $\gamma$ and a Gaussian white-noise. We explore how the information length depends on $\omega$ and $\gamma$, elucidating the role of critical damping $\gamma =2\omega$ in information geometry. Furthermore, in the long time limit, we show that the information length reflects the linear geometry associated with the Gaussian statistics in a linear stochastic process.

1. Introduction

Stochastic processes are common in nature or laboratories, and play a major role across traditional disciplinary boundaries (e.g., see [1,2]). These stochastic processes often exhibit complex temporal behaviour and even the emergence of order (self-organization). The latter can also be artificially designed to complete an orderly task (guided self-organization) [3,4,5,6]. In order to study and compare the dynamics of different stochastic processes and self-organization, it is valuable to utilize a measurement which is independent of any specifics of a system [7,8,9,10,11] (e.g., physical variables, units, dimensions, etc.). This can be achieved by using information theory based on probability density functions (PDFs) and working in terms of information content or information change, e.g., by quantifying the statistical difference between two states [12,13,14]. Mathematically, we do this by assigning a metric to probability and by using the notion of ‘length’ or ‘distance’ in the statistical space.

One method of measuring the information content in a system is utilizing the Fisher information, which represents the degree of certainty, or order. The opposite is entropy, which is a popular concept for the uncertainty or amount of disorder. Comparing entropy at different times then gives a measure of the difference in information content between the two states, which is known as relative entropy (e.g., see [15]). Another example is the Wasserstein metric [16,17], which provides an exact solution to the Fokker-Planck equation for a gradient flow subject to the minimization of the energy functional defined as the sum of the entropy and potential energy [18,19,20]. This metric has units of a physical length in comparison with other metrics, for instance the dimensionless statistical distance based on the Fisher information metric [21,22,23]. Interestingly, there is a link between the Fisher information and the Wasserstein distance [24]. Furthermore, the relative entropy can be expressed by the integral of the Fisher information along the same path [25].

Although quite useful, the relative entropy lacks the locality of a metric as it concerns only about the difference between given two PDFs. For instance, when these two PDFs represent the two states at different times, the relative entropy between them tells us nothing about how one PDF evolves to the other PDF over time or what intermediate states a system passes through between the two PDFs. As a result, it can only inform us of the changes that affect the overall system evolution [26]. To overcome this limitation, the information length $\mathcal{L}\left(t\right)$ was proposed in recent works, which quantifies the total number of different states that the system evolves through in time [27,28]. This means that the information length is a measure that depends on the evolution path between two states (PDFs). Its formulation allows us to measure local changes in the evolution of the system as well as providing an intriguing link between stochastic processes and geometry [26].

For instance, the relation between the information length ${\mathcal{L}}_{\infty }=\mathcal{L}(t\to \infty )$ and the mean value of the initial PDF for the fixed values of all other parameters was invoked as a new way of mapping out an attractor structure in a relaxation problem where any initial PDF relaxes into its equilibrium PDF in the long time limit. Specifically, for the Ornstein-Uhlenbeck (O-U) process driven by a Gaussian white-noise (which is a linearly damped, relaxation problem), ${\mathcal{L}}_{\infty }$ increases linearly with the distance between the mean position of an initial PDF and the stable equilibrium point for further details, see [28,29], with its minimum value zero at the stable equilibrium point. This linear dependence manifests that the information length preserves the linear geometry of the underlying Gaussian process, which is lost in other metrics [26]. For a nonlinear stochastic process with nonlinear damping, ${\mathcal{L}}_{\infty }$ still takes its minimum value at the stable equilibrium point but exhibits a power-law dependence on the distance between the mean value of an initial PDF and the stable equilibrium point. In contrast, for a chaotic attractor, ${\mathcal{L}}_{\infty }$ changes abruptly under an infinitesimal change of the mean value of an initial PDF, reminiscent of the sensitive dependence on initial conditions of the Lyapunov exponent [30]. These results suggest that ${\mathcal{L}}_{\infty }$ elucidates how different (non)linear forces affect (information) geometry.

With the above background in mind, this paper aims to extend the analysis of the information length of the O-U process to an arbitrary n-th order linear autonomous stochastic processes, providing a basic theoretical framework to be utilized in a large set of problems in both engineering and physics. In particular, we provide a useful analytical result that defines the information diagnostics as a function of the covariance matrix and the mean vector of the system, which enormously reduces the computational cost of numerical simulations of high-order systems.

This is followed by a specific application to a harmonically bound particle system (Kramers equation) for the position x and velocity $v=\frac{dx}{dt}$, with the natural oscillation frequency $\omega$, subject to a damping constant $\gamma$ and a Gaussian white-noise (short-correlated). We find an exact time-dependent joint PDF $p(x,v,t)$ starting from an initial Gaussian PDF which has a finite-width. Note that as far as we are aware of, our result $p(x,v,t)$ is original since in literature, the calculation was done only for the case of a delta-function initial PDF. Since this process is governed by the two variables, x and v, we investigate how ${\mathcal{L}}_{\infty }$ depends on their initial mean values $\langle x_0\rangle$ and $\langle v_0\rangle$. Here, the angular brackets denote the average. Furthermore, the two characteristic time scales associated with $\omega$ and $\gamma$ raise the interesting question as to their role in ${\mathcal{L}}_{\infty }$. Thus, we explore how the information length depends on $\omega$ and $\gamma$. Our principle results are as follows: (i) ${\mathcal{L}}_{\infty }$ tends to increase linearly with either the deviation of initial mean position $\langle x_0\rangle$ or the initial mean velocity $\langle v_0\rangle$ from their equilibrium values $\langle x\left(0\right)\rangle =\langle v\left(0\right)\rangle =0$; (ii) a linear geometry is thus preserved for our linearly coupled stochastic processes driven by a Gaussian noise; (iii) ${\mathcal{L}}_{\infty }$ tends to take its minimum value near the critical damping $\gamma =2\omega$ for the same initial conditions and other parameters.

The remainder of this paper is organized as follows: Section 2 presents the basic concept of information length and the formulation of our problem. In Section 3, our main theoretical results are provided (see also Appendix A). In Section 4, we apply the results in Section 3 to analyze a harmonically bound particle system with the natural oscillation frequency $\omega$ subject to a damping constant $\gamma$ and a Gaussian white-noise. Finally, Section 5 contains our concluding remarks.

To help readers, we here provide a summary of our notations: $\mathbb{R}$ and $\mathbb{C}$ are the sets of real and complex numbers, respectively. $\mathbf{x}\in {\mathbb{R}}^n$ represents a column vector $\mathbf{x}$ of real numbers of dimension n, $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ represents a real matrix of dimension $n\times n$, $\mathbf{tr}\left(\mathbf{A}\right)$ corresponds to the trace of the matrix $\mathbf{A}$, ${\mathbf{A}}^T$ and ${\mathbf{A}}^{-1}$ are the transpose and inverse of matrix $\mathbf{A}$, respectively. (Bold-face letters are used to represent vectors and matrices.) In some places, ${\partial }_t$ or the prime both are used for the partial derivative with respect to time. Besides, $\mathbf{i}=\sqrt{-1}$ and for $s\in \mathbb{C}$, ${\mathcal{L}}^{-1}\left[F\left(s\right)\right]=\frac{1}{2\pi \mathbf{i}}{lim}_{b\to \infty }{\int }_{a-b\mathbf{i}}^{a+b\mathbf{i}}{e}^{st}F\left(s\right)\mathrm{d}s$ corresponds to the inverse Laplace transform of the complex function $F\left(s\right)$. Finally, the average of a random vector $\mathbf{x}$ is denoted by $\langle \mathbf{x}\rangle$.

2. Preliminaries

2.1. Information Length

As noted in Section 1, the information length [26,27,31] is a dimensionless measurement of the total number of statistically different states that a system passes through in time in non-equilibrium processes. We cannot overemphasize that it is a measure that depends on the evolution of the system, being a useful index for understanding the information geometry underlying non-equilibrium processes. For example, for a time-dependent PDF $p(x,t)$ of one stochastic variable x, the information length $\mathcal{L}\left(t\right)$ is the total information change between time 0 and t, and is defined by

$$\mathcal{L}\left(t\right)={\int }_0^t\frac{\mathrm{d}{t}_1}{\tau \left(t_1\right)}={\int }_0^t\mathrm{d}{t}_1\sqrt{\mathcal{E}\left(t_1\right)}={\int }_0^t\mathrm{d}{t}_1\sqrt{{\int }_{-\infty }^{\infty }\mathrm{d}x\frac{1}{p(x,t_1)}{\left[\frac{\partial p(x,t_1)}{\partial t_1}\right]}^2}.$$

(1)

Here, $\mathcal{E}\left(t_1\right)={\int }_{-\infty }^{\infty }\mathrm{d}x\frac{1}{p(x,t_1)}{\left[\frac{\partial p(x,t_1)}{\partial t_1}\right]}^2$ is the square of the information velocity (recalling we are working with the unit where the distance given by the information length has no dimension). As we can see, to define the information length, we compute the dynamic time unit $\tau \left(t\right)=\frac{1}{\sqrt{\mathcal{E}}}$, which quantifies the correlation time over which the PDF $p(x,t)$ changes. Besides, $\tau$ serves as the time unit in the statistical space. Alternatively, the information velocity $\frac{1}{\tau \left(t_1\right)}$ quantifies the (average) rate of change of information in time.

2.2. Problem Formulation

We consider the following linear autonomous process

$$\dot{\mathbf{x}}\left(t\right)=\mathbf{A}\mathbf{x}\left(t\right)+\Gamma \left(t\right).$$

(2)

Here, $\mathbf{A}$ is an $n\times n$ constant real matrix; $\Gamma \in {\mathbb{R}}^n$ is a stochastic driving given by a n dimensional vector of $\delta$-correlated Gaussian noises ${\Gamma }_i$ ($i=1,2,...n$), with the following statistical property

$$\langle {\Gamma }_i\left(t\right)\rangle =0,\langle {\Gamma }_i\left(t\right){\Gamma }_j\left(t_1\right)\rangle =2D_{ij}\delta (t-t_1),D_{ij}=D_{ji},\forall i,j=1,\cdots ,n.$$

(3)

Note that $D_{ii}$ represents the strength of the i-th stochastic noise while $D_{ij}$ for $i\ne j$ denotes the correlation between i-th and j-th noises (i.e., random fluctuations). Then, from the joint PDF $p(\mathbf{x},t)$, we define the information length $\mathcal{L}$ of system (2) by the following integral

$$\mathcal{L}\left(t\right)={\int }_0^t\mathrm{d}{t}_1\sqrt{{\int }_{-\infty }^{\infty }\mathrm{d}\mathbf{x}\frac{1}{p(\mathbf{x},t_1)}{\left[\frac{\partial p(\mathbf{x},t_1)}{\partial t_1}\right]}^2}={\int }_0^t\mathrm{d}{t}_1\sqrt{\mathcal{E}},$$

(4)

where $\mathcal{E}={\int }_{-\infty }^{\infty }\mathrm{d}\mathbf{x}\frac{1}{p(\mathbf{x},t_1)}{\left[\frac{\partial p(\mathbf{x},t_1)}{\partial t_1}\right]}^2$ is the square of the information velocity.

The first goal of this paper is to provide theoretical results for the information length (4) for the system (2) and (3). This is done in the following Section 3.

3. General Analytical Results

In the section, we provide the analytical results for Problem 2.1, summarizing the main steps required to calculate information length (4). To this end, we assume that an initial PDF is Gaussian and then take the advantage of the fact that a linear process driven by a Gaussian noise with an initial Gaussian PDF is always Gaussian. The joint PDF for (2) and (3) is thus Gaussian, whose form is provided below.

Proposition 1

(Joint probability). The system (2) and (3) for a Gaussian random variable $\mathbf{x}$ at any time t has the following joint PDF

$$p(\mathbf{x},t)=\frac{1}{\sqrt{det(2\pi \Sigma )}}{e}^{-\frac{1}{2}{\left(\mathbf{x}-\langle \mathbf{x}\left(t\right)\rangle \right)}^T{\Sigma }^{-1}\left(\mathbf{x}-\langle \mathbf{x}\left(t\right)\rangle \right)},$$

(5)

where

$$\begin{array}{ccc}\hfill \langle \mathbf{x}\left(t\right)\rangle & =& e^{\mathbf{A}t}\langle \mathbf{x}\left(0\right)\rangle ,\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill \Sigma \left(t\right)& =& e^{\mathbf{A}t}〈\delta \mathbf{x}\left(0\right)\delta \mathbf{x}{\left(0\right)}^T〉e^{{\mathbf{A}}^{T}t}+2{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\mathbf{D}{e}^{{\mathbf{A}}^T(t-t_1)}\mathrm{d}{t}_1,\hfill \end{array}$$

(7)

and $\mathbf{D}\in {\mathbb{R}}^{n\times n}$ is the matrix of elements $D_{ij}$. Here, $\langle \mathbf{x}\left(t\right)\rangle$ is the mean value of $\mathbf{x}\left(t\right)$ while Σ is the covariance matrix.

Proof. 

For a Gaussian PDF of $\mathbf{x}$, all we need to calculate are the mean and covariance of $\mathbf{x}$ and substitute them in the general expression for multi-variable Gaussian distribution (5). To this end, we first write down the solution of Equation (2) as follows

$$\mathbf{x}\left(t\right)=e^{\mathbf{A}t}\mathbf{x}\left(0\right)+{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\Gamma \left(t_1\right)\mathrm{d}{t}_1.$$

(8)

By taking the average of Equation (8), we find the mean value of $\mathbf{x}\left(t\right)$ of (8) as follows

$$\langle \mathbf{x}\left(t\right)\rangle =\langle e^{\mathbf{A}t}\mathbf{x}\left(0\right)\rangle +{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\langle \Gamma \left(t_1\right)\rangle \mathrm{d}{t}_1=e^{\mathbf{A}t}\langle \mathbf{x}\left(0\right)\rangle ,$$

(9)

which is Equation (6). On the other hand, to find covariance $\Sigma \left(t\right)$, we let $\mathbf{x}=\langle \mathbf{x}\rangle +\delta \mathbf{x}$, and use the property $\langle \delta \mathbf{x}\left(0\right)\Gamma \left(t\right)\rangle =0$ to find

$$\begin{array}{ccc}\hfill \Sigma \left(t\right)& =& 〈\delta \mathbf{x}\delta {\mathbf{x}}^T〉\hfill \\ & =& 〈\left(e^{\mathbf{A}t}\delta \mathbf{x}\left(0\right)+{\int }_0^t{e}^{\mathbf{A}(t-t_2)}\langle \Gamma \left(t_2\right)\rangle \mathrm{d}{t}_2\right){\left(e^{\mathbf{A}t}\delta \mathbf{x}\left(0\right)+{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\langle \Gamma \left(t_1\right)\rangle \mathrm{d}{t}_1\right)}^T〉\hfill \\ & =& 〈\left(e^{\mathbf{A}t}\delta \mathbf{x}\left(0\right)+{\int }_0^t{e}^{\mathbf{A}(t-t_2)}\Gamma \left(t_2\right)\mathrm{d}{t}_2\right)\left(\delta \mathbf{x}{\left(0\right)}^T{e}^{{\mathbf{A}}^{T}t}+{\int }_0^t\Gamma {\left(t_1\right)}^T{\left(e^{\mathbf{A}(t-t_1)}\right)}^T\mathrm{d}{t}_1\right)〉\hfill \\ & =& e^{\mathbf{A}t}〈\delta \mathbf{x}\left(0\right)\delta \mathbf{x}{\left(0\right)}^T〉e^{{\mathbf{A}}^{T}t}+〈\left({\int }_0^t{e}^{\mathbf{A}(t-t_2)}\Gamma \left(t_2\right)\mathrm{d}{t}_2\right)\left({\int }_0^t\Gamma {\left(t_1\right)}^T{e}^{{\mathbf{A}}^T(t-t_1)}\mathrm{d}{t}_1\right)〉\hfill \\ & =& e^{\mathbf{A}t}〈\delta \mathbf{x}\left(0\right)\delta \mathbf{x}{\left(0\right)}^T〉e^{{\mathbf{A}}^{T}t}+{\int }_0^t{\int }_0^t{e}^{\mathbf{A}(t-t_2)}\langle \Gamma \left(t_2\right)\Gamma {\left(t_1\right)}^T\rangle e^{{\mathbf{A}}^T(t-t_1)}\mathrm{d}{t}_2\mathrm{d}{t}_1\hfill \\ & =& e^{\mathbf{A}t}〈\delta \mathbf{x}\left(0\right)\delta \mathbf{x}{\left(0\right)}^T〉e^{{\mathbf{A}}^{T}t}+2{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\mathbf{D}{e}^{{\mathbf{A}}^T(t-t_1)}\mathrm{d}{t}_1.\hfill \end{array}$$

(10)

Here $\delta \mathbf{x}\left(0\right)=\delta \mathbf{x}(t=0)$ is the initial fluctuation at $t=0$. Equation (10) thus proves Equation (7). Substitution of Equations (6) and (7) in Equation (5) thus gives us a joint PDF $p(\mathbf{x},t)$ ☐

Next, in order to calculate the information length from the joint PDF $p(\mathbf{x},t)$ in Equation (5), we now use the following Theorem:

Theorem 1

(Information Length). The information length of the joint PDF of system (2) and (3) is given by the following integral

$$\mathcal{L}\left(t\right)={\int }_0^t\mathrm{d}{t}_1\sqrt{\mathcal{E}\left(t_1\right)}=\frac{1}{\sqrt{2}}{\int }_0^t\mathrm{d}{t}_1\sqrt{{\partial }_{t_1}\left[\mathbf{t}\mathbf{r}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right]+2{\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle }^T\mathbf{Q}\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle +\mathbf{tr}\left({\mathbf{Q}}^{″}\Sigma \right)},$$

(11)

where $\mathbf{Q}={\Sigma }^{-1}$ (recall, a prime denotes $\frac{\partial }{\partial t}$).

Proof. 

To prove this theorem, we use the PDF (5) in (4). To simplify the expression, we let

$$\mathbf{w}\equiv \delta \mathbf{x}=\mathbf{x}-\langle \mathbf{x}\left(t\right)\rangle ,\mathbf{Q}={\Sigma }^{-1}.$$

We then compute step by step $\frac{{\left[{\partial }_{t_1}p(\mathbf{x},t_1)\right]}^2}{p(\mathbf{x},t_1)}$ as follows:

$$\begin{array}{ccc}\hfill {\partial }_{t_1}p(\mathbf{x},t_1)& =& \frac{\partial }{\partial t_1}\left[{\left(det(2\pi \Sigma )\right)}^{-\frac{1}{2}}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\right]\hfill \\ \hfill & =& -\frac{1}{2}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\left(det(2\pi \Sigma )\right)}^{-\frac{3}{2}}{\partial }_{t_1}\left(det(2\pi \Sigma )\right)-\frac{1}{2}{\left(det(2\pi \Sigma )\right)}^{-\frac{1}{2}}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\partial }_{t_1}\left({\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right),\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\left[{\partial }_{t_1}p(\mathbf{x},t_1)\right]}^2& =& \frac{1}{4}{e}^{-{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{(det(2\pi \Sigma ))}^{-3}{\left[{\partial }_{t_1}det(2\pi \Sigma )\right]}^2+\frac{1}{4}{\left(det(2\pi \Sigma )\right)}^{-1}{e}^{-{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\left({\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]\right)}^2\hfill \\ \hfill & & +\frac{1}{2}{\left(det(2\pi \Sigma )\right)}^{-2}{\partial }_{t_1}\left[det(2\pi \Sigma )\right]{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]e^{-{\mathbf{w}}^T\mathbf{Q}\mathbf{w}},\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill \frac{{\left[{\partial }_{t_1}p(\mathbf{x},t_1)\right]}^2}{p(\mathbf{x},t_1)}& =& \frac{1}{4}{\left(det(2\pi \Sigma )\right)}^{-\frac{5}{2}}{\left[{\partial }_{t_1}det(2\pi \Sigma )\right]}^2{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}+\frac{1}{4}{\left(det(2\pi \Sigma )\right)}^{-\frac{1}{2}}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\left({\partial }_{t_1}\left({\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right)\right)}^2\hfill \\ \hfill & & +\frac{1}{2}{\left(det(2\pi \Sigma )\right)}^{-\frac{3}{2}}{\partial }_{t_1}\left(det(2\pi \Sigma )\right){\partial }_{t_1}\left({\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right)e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}.\hfill \end{array}$$

(14)

Now, using Equation (12) in Equation (14), we compute the integral $\mathcal{E}\left(t_1\right)={\int }_{-\infty }^{\infty }\mathbf{x}\left(\frac{{\left[{\partial }_{t_1}p(\mathbf{x},t_1)\right]}^2}{p(\mathbf{x},t_1)}\right)$ as follows

$$\begin{array}{ccc}\hfill \mathcal{E}\left(t_1\right)& =& {\int }_{-\infty }^{\infty }p(\mathbf{x},t_1)\left(\frac{{\left(det(2\pi \Sigma )\right)}^{-2}}{4}{\left[{\partial }_{t_1}\left(det(2\pi \Sigma )\right)\right]}^2+\frac{{\left({\partial }_{t_1}\left({\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right)\right)}^2}{4}+\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]}{2det(2\pi \Sigma )}\right)\mathrm{d}\mathbf{x}\hfill \\ \hfill & =& 〈{\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{2det(2\pi \Sigma )}\right)}^2〉+〈{\left(\frac{{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]}{2}\right)}^2〉+〈\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]}{2det(2\pi \Sigma )}〉.\hfill \end{array}$$

(15)

To calculate the three averages in (15), we use the properties ${\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\mathbf{w}=\sqrt{det(2\pi \Sigma )}$ [32], ${\partial }_{t_1}\left[e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\right]=-\frac{1}{2}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]$ and ${\partial }_{t_1{t}_1}\left[e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\right]=-\frac{1}{2}{\partial }_{t_1{t}_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}+\frac{1}{4}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}{\left({\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]\right)}^2$. We then have

$$\begin{array}{ccc}\hfill \mathcal{E}\left(t_1\right)& =& {\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{2det(2\pi \Sigma )}\right)}^2+{\int }_{-\infty }^{\infty }p(\mathbf{x},t_1){\left(\frac{{\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]}{2}\right)}^2\mathrm{d}\mathbf{x}+\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{2det(2\pi \Sigma )}{\int }_{-\infty }^{\infty }p(\mathbf{x},t_1){\partial }_{t_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]\mathrm{d}\mathbf{x}\hfill \\ \hfill & =& \frac{1}{4}{\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{det(2\pi \Sigma )}\right)}^2+\frac{1}{4{\left(det(2\pi \Sigma )\right)}^{\frac{1}{2}}}{\int }_{-\infty }^{\infty }\left[4{\partial }_{t_1{t}_1}{e}^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}+2{\partial }_{t_1{t}_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\right]\mathrm{d}\mathbf{x}\hfill \\ \hfill & & -2\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{2{\left(det(2\pi \Sigma )\right)}^{\frac{3}{2}}}{\int }_{-\infty }^{\infty }{\partial }_{t_1}\left[e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\right]\mathrm{d}\mathbf{x}\hfill \\ \hfill & =& \frac{1}{4}{\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{det(2\pi \Sigma )}\right)}^2+\frac{{\partial }_{t_1{t}_1}\left[\sqrt{det(2\pi \Sigma )}\right]}{\sqrt{det(2\pi \Sigma )}}+\frac{1}{2\sqrt{det(2\pi \Sigma )}}{\int }_{-\infty }^{\infty }{\partial }_{t_1{t}_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]e^{-\frac{1}{2}{\mathbf{w}}^T\mathbf{Q}\mathbf{w}}\mathrm{d}\mathbf{x}\hfill \\ \hfill & & -\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{{\left(det(2\pi \Sigma )\right)}^{\frac{3}{2}}}{\partial }_{t_1}\sqrt{det(2\pi \Sigma )}.\hfill \end{array}$$

(16)

Here

$${\partial }_{t_1{t}_1}\left[{\mathbf{w}}^T\mathbf{Q}\mathbf{w}\right]=\sum _{i,j=1}^n\left[{\partial }_{t_1{t}_1}\left(q_{ij}{w}_i{w}_j\right)\right]=\sum _{i,j=1}^n\left[4q_{ij}^{\prime }{w}_i^{\prime }{w}_j+\underset{\mathrm{independent}\mathrm{of}x}{\underbrace{2q_{ij}{w}_i^{\prime }{w}_j^{\prime }}}+2q_{ij}{w}_i^{″}{w}_j+\underset{{\mathbf{w}}^T{\mathbf{Q}}^{″}\mathbf{w}}{\underbrace{q_{ij}^{″}{w}_i{w}_j}}\right].$$

(17)

We recall that ${\omega }_i^{\prime },q_{ij}^{\prime }$ and ${\omega }_i^{″},q_{ij}^{″}$ denote the first and second derivative over time of the elements ${\omega }_i$ and $q_{ij}$. By substituting (17) in (16) and making some arrangements, we obtain

$$\begin{array}{ccc}\hfill \mathcal{E}\left(t_1\right)& =& \frac{1}{4}{\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{det(2\pi \Sigma )}\right)}^2+\frac{{\partial }_{t_1{t}_1}\left[\sqrt{det(2\pi \Sigma )}\right]}{\sqrt{det(2\pi \Sigma )}}+\frac{1}{2}{\overline{)〈4\sum _{i,j=1}^n{q}_{ij}^{\prime }{w}_i^{\prime }{w}_j〉}}^0+\frac{1}{2}{\overline{)〈\sum _{i,j=1}^{n}2q_{ij}{w}_i^{″}{w}_j〉}}^0\hfill \\ \hfill & & +\frac{1}{2}〈\sum _{i,j=1}^{n}2q_{ij}{w}_i^{\prime }{w}_j^{\prime }〉+\frac{1}{2}〈{\mathbf{w}}^T{\mathbf{Q}}^{″}\mathbf{w}〉-\frac{1}{2}{\left(\frac{{\partial }_{t_1}\left[det(2\pi \Sigma )\right]}{det(2\pi \Sigma )}\right)}^2.\hfill \end{array}$$

(18)

Now with the help of the following relations $〈{\mathbf{w}}^T{\mathbf{Q}}^{″}\mathbf{w}〉=\mathbf{tr}\left({\mathbf{Q}}^{″}\Sigma \right)$ [33], ${\partial }_{t_1}det(\Sigma )=det(\Sigma )\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )$ [34], and

${\partial }_{t_1{t}_1}\sqrt{det(2\pi \Sigma )}=\frac{1}{4}\sqrt{det(2\pi \Sigma )}{\left(\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right)}^2+\frac{1}{2}\sqrt{det(2\pi \Sigma )}{\partial }_{t_1}\left(\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right)$, we then have

$$\begin{array}{ccc}\hfill \mathcal{E}\left(t_1\right)& =& -\frac{1}{4}{\left(\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right)}^2+\frac{1}{2}{\partial }_{t_1}\left[\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right]+\frac{1}{4}{\left(\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right)}^2\hfill \\ \hfill & & +{\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle }^T\mathbf{Q}\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle +\frac{1}{2}\mathbf{tr}\left({\mathbf{Q}}^{″}\Sigma \right)\hfill \\ & =& \frac{1}{2}{\partial }_{t_1}\left[\mathbf{tr}(\mathbf{Q}{\partial }_{t_1}\Sigma )\right]+{\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle }^T\mathbf{Q}\langle {\mathbf{x}}^{\prime }\left(t_1\right)\rangle +\frac{1}{2}\mathbf{tr}\left({\mathbf{Q}}^{″}\Sigma \right).\hfill \end{array}$$

(19)

Equation (19) thus proves Equation (11). ☐

Given important properties of the covariance matrix eigenvalues (see, e.g., [35]), it is useful to express Equation (19) and the information length as a function of these covariance matrix eigenvalues. This is done in the following Corollary.

Corollary 1.

Let ${\phi }_i\left(t\right)$’s ($i=1,...n$) be the eigenvalues of the covariance matrix Σ, and $\overline{\mathbf{x}}={\langle {\mathbf{x}}^{\prime }\left(t\right)\rangle }^T\mathbf{P}$ where $\mathbf{P}$ is an orthonormal matrix whose column vectors are linearly independent eigenvectors of $Q={\Sigma }^{-1}$. We can rewrite the information length (11) as

$$\mathcal{L}\left(t\right)={\int }_0^t\mathrm{d}{t}_1\sqrt{\mathcal{E}\left(t_1\right)}={\int }_0^t\mathrm{d}{t}_1\sqrt{\sum _{i=1}^n\left(\frac{{{\phi }^{\prime }}_i^2\left(t_1\right)+2{\phi }_i\left(t_1\right){\overline{x}}_i^2}{2{\phi }_i^2\left(t_1\right)}\right)}.$$

(20)

Proof. 

The proof follows straightforwardly from the fact that $\Sigma$ is a symmetric matrix which can be diagonalised by finding the orthonormal matrix $\mathbf{P}$ such that ${\mathbf{P}}^T{\Sigma }^{-1}\mathbf{P}=\Phi$. Here $\Phi$ is the diagonal matrix whose entries are the eigenvalues $\frac{1}{{\phi }_i\left(t\right)}$$\forall i=1,2,\cdots ,n$ (recall that ${\phi }_i\left(t\right)$ is i-th the eigenvalue of ${\Sigma }^{-1}$). This gives us

$$\mathcal{E}\left(t\right)=\frac{1}{2}\sum _{i=1}^n\left({\partial }_t\left[\frac{{\phi }_i^{\prime }\left(t\right)}{{\phi }_i\left(t\right)}\right]+{\phi }_i\left(t\right){\partial }_{tt}\left[\frac{1}{\phi \left(t\right)}\right]+2\frac{{\overline{x}}_i^2}{{\phi }_i\left(t\right)}\right)=\sum _{i=1}^n\left(\frac{{{\phi }^{\prime }}_i^2\left(t\right)+2{\phi }_i\left(t\right){\overline{x}}_i^2}{2{\phi }_i^2\left(t\right)}\right).$$

(21)

This finishes the proof. ☐

It is useful to check that Equation (20) reproduces the previous result for the O-U process [36]

$$\mathcal{E}=\frac{{\beta }^{\prime 2}}{2{\beta }^2}+2\beta {\langle x^{\prime }\rangle }^2,$$

(22)

where $\beta =\frac{1}{2\langle {(x-\langle x\rangle )}^2\rangle }$ is the inverse temperature. Here, ${\beta }^{\prime }$ denotes the time derivative of $\beta$. To show this, we note that for the O-U process, the covariance matrix is a scalar ($n=1$) with the value $\Sigma =\frac{1}{2\beta }=\phi \left(t\right)$ and thus $\mathbf{Q}=\frac{1}{\phi \left(t\right)}=2\beta$ while $\langle {\mathbf{x}}^{\prime }\left(t\right)\rangle =\langle x^{\prime }\rangle$. Thus,

$$\mathcal{E}\left(t\right)=\frac{1}{2}{\left(\frac{-\frac{1}{2}{\beta }^{-2}{\beta }^{\prime }}{\frac{1}{2\beta }}\right)}^2+2\beta {\langle x^{\prime }\rangle }^2=\frac{{\beta }^{\prime 2}}{2{\beta }^2}+2\beta {\langle x^{\prime }\rangle }^2.$$

In sum, for the O-U process, the square of the information velocity (shown in expression (22)) increases with the ‘roughness’ of the process, as quantified by the squared ratio of the rate of change of the inverse temperature (or precision) and the precision – plus a term that depends upon this precision times the variance of the state velocity.

4. Kramers Equation

In this section we apply our results in Section 3 to the Kramers equation for a harmonically bound particle [19,37]. As noted in Introduction, we investigate the behaviour of the information length when varying various parameters and initial conditions to elucidate how the information geometry is affected by the damping, oscillations, strength of the stochastic noises and initial mean values.

Consider the Kramers equation

$$\begin{array}{ccc}\hfill \frac{\mathrm{d}x}{\mathrm{d}t}& =& v\hfill \\ \hfill \frac{\mathrm{d}v}{\mathrm{d}t}& =& -\gamma v-{\omega }^{2}x+\xi \left(t\right).\hfill \end{array}$$

(23)

Here, $\omega$ is a natural frequency and $\gamma$ is the damping constant, both positive real numbers. $\xi$ is a Gaussian white-noise acting on v with the zero mean value $\langle \xi \left(t\right)\rangle =0$, with the statistical property

$$\langle \xi \left(t\right)\xi \left(t_1\right)\rangle =2D\delta (t-t_1).$$

(24)

Comparing Equations (23) and (24) with Equations (2) and (3), we note that $x_1=x$, $x_2=v$, ${\xi }_1=0$, ${\xi }_2=\xi$, $D_{11}=0$, $D_{12}=0$, and $D_{22}=D$ while the matrix $\mathbf{A}$ for (23) has the element $A_{11}=0,A_{12}=1,A_{21}=-{\omega }^2,A_{22}=-\gamma$. Thus, the eigenvalues of $\mathbf{A}$ are ${\lambda }_{1,2}=-\frac{1}{2}\left(\gamma \pm \sqrt{{\gamma }^2-4{\omega }^2}\right)$.

To find the information length for the system (23), we use Proposition 1 and Theorem 1. First, Proposition 1 requires the computation of the exponential matrix $e^{\mathbf{A}t}$ involving a rather long algebra with the help of [38]. The result is:

$$\begin{array}{c}\hfill e^{\mathbf{A}t}={\mathcal{L}}^{-1}\left[{(s\mathbf{I}-\mathbf{A})}^{-1}\right]={\mathcal{L}}^{-1}\left[\left[\begin{array}{cc}\frac{s+\gamma }{(s-{\lambda }_1)(s-{\lambda }_2)}& \frac{1}{(s-{\lambda }_1)(s-{\lambda }_2)}\\ -\frac{{\omega }^2}{(s-{\lambda }_1)(s-{\lambda }_2)}& \frac{s}{(s-{\lambda }_1)(s-{\lambda }_2)}\end{array}\right]\right]=\left[\begin{array}{cc}\frac{e^{{\lambda }_{1}t}(\gamma +{\lambda }_1)-e^{{\lambda }_{2}t}(\gamma +{\lambda }_2)}{{\lambda }_1-{\lambda }_2}& \frac{e^{{\lambda }_{1}t}-e^{{\lambda }_{2}t}}{{\lambda }_1-{\lambda }_2}\\ -\frac{\left(e^{{\lambda }_{1}t}-e^{{\lambda }_{2}t}\right){\omega }^2}{{\lambda }_1-{\lambda }_2}& \frac{e^{{\lambda }_{1}t}{\lambda }_1-e^{{\lambda }_{2}t}{\lambda }_2}{{\lambda }_1-{\lambda }_2}\end{array}\right].\end{array}$$

(25)

Here, $\mathbf{I}\in {\mathbb{R}}^{n\times n}$ is the identity matrix. Similarly, we can show

$$\begin{array}{c}2{\int }_0^t{e}^{\mathbf{A}(t-t_1)}\mathbf{D}{e}^{{\mathbf{A}}^T(t-t_1)}\mathrm{d}{t}_1=\hfill \\ \hfill \left[\begin{array}{cc}\frac{D\left(\frac{-1+e^{2{\lambda }_{1}t}}{{\lambda }_1}+\frac{-{\lambda }_1-4e^{({\lambda }_1+{\lambda }_2)t}{\lambda }_2+3{\lambda }_2+e^{2{\lambda }_{2}t}({\lambda }_1+{\lambda }_2)}{{\lambda }_2({\lambda }_1+{\lambda }_2)}\right)}{{({\lambda }_1-{\lambda }_2)}^2}& \frac{D{\left(e^{{\lambda }_{1}t}-e^{{\lambda }_{2}t}\right)}^2}{{({\lambda }_1-{\lambda }_2)}^2}\\ \frac{D{\left(e^{{\lambda }_{1}t}-e^{{\lambda }_{2}t}\right)}^2}{{({\lambda }_1-{\lambda }_2)}^2}& \frac{D\left(\left(-1+e^{2{\lambda }_{2}t}\right){\lambda }_2+{\lambda }_1\left(-\frac{4e^{({\lambda }_1+{\lambda }_2)t}{\lambda }_2}{{\lambda }_1+{\lambda }_2}+\frac{4{\lambda }_2}{{\lambda }_1+{\lambda }_2}+e^{2{\lambda }_{1}t}-1\right)\right)}{{({\lambda }_1-{\lambda }_2)}^2}\end{array}\right].\end{array}$$

(26)

Using Equations (25) and (26) in Equations (6) and (7), we have the time-dependent (joint) PDF (5) at any time t for our system (23) and (24). To calculate Equation (11) with the help of Equations (25) and (26), we perform numerical simulations (integrations) for various parameters in Equations (23) and (24) as well as initial conditions. Note that while we have simulated many different cases, for illustration, we show some representative cases by varying D, $\omega$, $\gamma$ and $\langle x\left(0\right)\rangle$, $\langle v\left(0\right)\rangle$ in Section 4.1, Section 4.2 and Section 4.3 and Appendix A, respectively, for the same initial covariance matrix $\Sigma \left(0\right)$ with elements ${\Sigma }_{11}\left(0\right)={\Sigma }_{22}\left(0\right)=0.01$ and ${\Sigma }_{12}\left(0\right)={\Sigma }_{21}\left(0\right)=0$. Note that the initial marginal distributions of $p\left(x\right(0\left)\right)$ and $p\left(v\right(0\left)\right)$ are Gaussian with the same variance $0.01$. Results in the limit $\omega \to 0$ are presented in Section 4.4.

4.1. Varying D

Figure 1 shows the results when varying D as $D\in (0.0005,0.04)$ for the fixed parameters $\gamma =2$ and $\omega =1$. The initial joint PDFs are Gaussian with the fixed mean values $\langle x\left(0\right)\rangle =-0.5$, $\langle v\left(0\right)\rangle =0.7$; as noted above, the covariance matrix $\Sigma \left(0\right)$ with elements ${\Sigma }_{11}\left(0\right)={\Sigma }_{22}\left(0\right)=0.01$ and ${\Sigma }_{12}\left(0\right)={\Sigma }_{21}\left(0\right)=0$. Consequently, at $t=0$, the marginal distributions of $p\left(x\right(0\left)\right)$ and $p\left(v\right(0\left)\right)$ are Gaussian PDFs with the same variance $0.01$ and the mean values $\langle x\left(0\right)\rangle =-0.5$ and $\langle v\left(0\right)\rangle =0.7$, respectively.

Figure 1a,b show the snapshots of time-dependent joint PDF $p(\mathbf{x},t)$ (in contour plots) for the two different values of $D=0.0005$ and $D=0.04$, respectively. The black solid represents the phase portrait of the mean value of $\langle x\left(t\right)\rangle$ and $\langle v\left(t\right)\rangle$ while the red arrows display the direction of time increase. Note that in Figure 1b, only some of the initial snapshots of the PDFs are shown for clarity, given the great amount of overlapping between different PDFs. Figure 1c,d show the time-evolution of the information velocity $\mathcal{E}\left(t\right)$ and information length $\mathcal{L}\left(t\right)$, respectively, for different values of $D\in (0.0005,0.04)$. It can be seen that the system approaches a stationary (equilibrium) state for $t\gtrsim 20$ for all values of D, $\mathcal{L}\left(t\right)$ approaching constant values (recall $\mathcal{L}\left(t\right)$ does not change in a stationary state). Therefore, we approximate the total information length as ${\mathcal{L}}_{\infty }=\mathcal{L}(t=50)$, for instance. Finally, the total information length ${\mathcal{L}}_{\infty }=\mathcal{L}(t=50)$ is shown in Figure 1e. We determine the dependence of ${\mathcal{L}}_{\infty }$ on D by fitting an exponential function as ${\mathcal{L}}_{\infty }\left(D\right)=7.84e^{-329.05D}+11.21e^{-11.86D}$ (shown in red solid line).

4.2. Varying $\omega$ or $\gamma$

We now explore how results depend on the two parameters $\omega$ and $\gamma$, associated with oscillation and damping, respectively. To this end, we use $D=0.0005$ and the same initial conditions as in Figure 1 but vary $\omega \in (0,2)$ and $\gamma \in (0,6)$ in Figure 2 and Figure 3, respectively. Specifically, in different panels of these figures, we show the snapshots of the joint PDF $p(\mathbf{x},t)$, the time-evolutions of $\mathcal{E}\left(t\right)$ and $\mathcal{L}\left(t\right)$ for different values of $\omega \in (0,2)$ and $\gamma \in (0,6)$, and ${\mathcal{L}}_{\infty }$ against either $\omega$ or $\gamma$. From Figure 2e and Figure 3e, we can see that the system is in a stationary state for sufficiently large $t=10$ and $t=100$, respectively. Thus, we use ${\mathcal{L}}_{\infty }=\mathcal{L}(t=10)=\mathcal{L}\left(10\right)$ in Figure 2f,g and ${\mathcal{L}}_{\infty }=\mathcal{L}(t=100)=\mathcal{L}\left(10\right)$ in Figure 3f,g.

Notably, Figure 2f,g (shown on linear-linear and log-linear scales on $x-y$ axes, respectively) exhibit an interesting a non-monotonic dependence of ${\mathcal{L}}_{\infty }$ on $\omega$ for the fixed $\gamma =2$, with the presence of a distinct minimum in ${\mathcal{L}}_{\infty }$ at certain $\omega$. Similarly, Figure 3f,g (shown in linear-linear and log-log scales on $x-y$ axes, respectively) also shows a non-monotonic dependence of ${\mathcal{L}}_{\infty }$ on $\gamma$ for the fixed $\omega =1$. These non-monotonic dependences are more clearly seen in Figure 2g and Figure 3g. A close inspection of these figures then reveals that the minimum value of ${\mathcal{L}}_{\infty }$ occurs close to the critical damping (CD) $\gamma \sim 2\omega$; specifically, this happens at $\omega \sim 1$ for $\gamma =2$ in Figure 2f,g while at $\gamma \sim 2$ for $\omega =1$ in Figure 3f,g. We thus fit ${\mathcal{L}}_{\infty }$ against $\omega$ or $\gamma$ depending on whether $\omega$ or $\gamma$ is smaller/larger than its critical value as follows:

$$\begin{array}{ccc}\hfill {\mathcal{L}}_{10}\left(\omega \right)& =& -0.03e^{4.34\omega }+19.63e^{0.06\omega }\forall \omega \in (0,1),\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill {\mathcal{L}}_{10}\left(\omega \right)& =& 19.52e^{-0.12\omega }+0.11e^{2.48\omega }\forall \omega \in (1,2),\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill {\mathcal{L}}_{100}\left(\gamma \right)& =& 413.22e^{-12.4\gamma }+95.39e^{-1.02\gamma }\forall \gamma \in (0,2),\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill {\mathcal{L}}_{100}\left(\gamma \right)& =& 3.23\gamma \forall \gamma \in (2,6).\hfill \end{array}$$

(30)

The fitted curves in Equations (27)–(30) are superimposed in Figure 2f and Figure 3f, respectively. It is important to notice from Equations (27)–(30) that ${\mathcal{L}}_{\infty }$ tends to increase as either $\omega \to \infty$ for a finite, fixed $\gamma$ ($<\infty$) or $\gamma \to \infty$ for a finite, fixed $\omega$ ($<\infty$).

Finally, we note that for the critical damping $\gamma =2\omega$, the eigenvalue becomes a real double root with the value ${\lambda }_{1,2}\to -\omega$. Thus, in this limit, we have that

$$\langle \mathbf{x}\left(t\right)\rangle =\left[\begin{array}{c}{e}^{-t\omega }(x\left(0\right)+t(v\left(0\right)+(\gamma -\omega )x\left(0\right)))\\ e^{-t\omega }\left(-tx\left(0\right){\omega }^2-tv\left(0\right)\omega +v\left(0\right)\right)\end{array}\right],$$

(31)

and $\Sigma \left(t\right)$ is composed by the following elements

$$\begin{array}{ccc}\hfill {\Sigma }_{11}\left(t\right)& =& \frac{e^{-2t\omega }\left(2{\omega }^3\left({\Sigma }_{11}{(\gamma t-t\omega +1)}^2+t^2(({\Sigma }_{12}+{\Sigma }_{21})(\gamma -\omega )+{\Sigma }_{22})+t({\Sigma }_{12}+{\Sigma }_{21})\right)+D\left(-2t\omega (t\omega +1)+e^{2t\omega }-1\right)\right)}{2{\omega }^3},\hfill \\ \hfill {\Sigma }_{12}\left(t\right)& =& e^{-2t\omega }\left(t\left(-{\omega }^2({\Sigma }_{11}\gamma t+{\Sigma }_{11}+{\Sigma }_{21}t)+{\Sigma }_{11}t{\omega }^3-{\Sigma }_{22}t\omega +{\Sigma }_{22}+Dt\right)-{\Sigma }_{12}(t\omega -1)(\gamma t-t\omega +1)\right),\hfill \\ \hfill {\Sigma }_{21}\left(t\right)& =& e^{-2t\omega }\left(t\left(-{\omega }^2({\Sigma }_{11}\gamma t+{\Sigma }_{11}+{\Sigma }_{12}t)+{\Sigma }_{11}t{\omega }^3-{\Sigma }_{22}t\omega +{\Sigma }_{22}+Dt\right)-{\Sigma }_{21}(t\omega -1)(\gamma t-t\omega +1)\right),\hfill \\ \hfill {\Sigma }_{22}\left(t\right)& =& \frac{e^{-2t\omega }\left(2t{\omega }^2\left(t{\omega }^2({\Sigma }_{11}\omega +{\Sigma }_{12}+{\Sigma }_{21})-\omega ({\Sigma }_{12}+{\Sigma }_{21})+{\Sigma }_{22}(t\omega -2)\right)+2{\Sigma }_{22}\omega +D\left(-2t\omega (t\omega -1)+e^{2t\omega }-1\right)\right)}{2\omega }.\hfill \end{array}$$

(32)

Equations (31) and (32) are used in Section 4.1 (Figure 1).

4.3. Varying $\langle x\left(0\right)\rangle$ or $\langle v\left(0\right)\rangle$

To elucidate the information geometry associated with the Kramer equation (Equations (23) and (24)), we now investigate how ${\mathcal{L}}_{\infty }$ behaves near the equilibrium point $\langle x\left(0\right)\rangle =\langle v\left(0\right)\rangle =0$. To this end, we scan over $\langle x\left(0\right)\rangle$ for $\langle v\left(0\right)\rangle =0$ in Figure 4a–e while scanning over $\langle v\left(0\right)\rangle$ for $\langle x0)\rangle =0$ in Figure 4f–i. For our illustrations in Figure 4, we use the same initial covariance matrix $\Sigma \left(0\right)$ as in Figure 1, Figure 2 and Figure 3, $D=0.0005$ and $\omega =1$ and a few different values of $\gamma$ (above/below/at the critical value $\gamma =2$). We note that the information geometry near a non-equilibrium point is studied in Appendix A.

Specifically, snapshots of $p(\mathbf{x},t)$ are shown in Figure 4a–f for $\gamma =2.5$ (above its critical value $\gamma =2=2\omega$) while those in Figure 4c–g are for $\gamma =0.1$ below the critical value 2. By approximating ${\mathcal{L}}_{\infty }=\mathcal{L}(t=100)$, we then show how ${\mathcal{L}}_{\infty }$ depends on $\langle x\left(0\right)\rangle$ and $\langle v\left(0\right)\rangle$ for different values of $\gamma$ in Figure 4d,e and Figure 4h,i, respectively.

Figure 4d,e show the presence of a minimum in ${\mathcal{L}}_{\infty }$ at the equilibrium $\langle x\left(0\right)\rangle =0$ (recall $\langle v\left(0\right)\rangle =0$); ${\mathcal{L}}_{\infty }$ is a linear function of $\langle x\left(0\right)\rangle$ for $\langle x\left(0\right)\rangle \gg 0.1$, which can be described as ${\mathcal{L}}_{\infty }(x\left(0\right),\gamma )=h\left(\gamma \right)\left|\langle x\left(0\right)\rangle \right|+f\left(\gamma \right)$. Here, $h\left(\gamma \right)$ and $f\left(\gamma \right)$ are constant functions depending on $\gamma$ for a fixed $\omega$ which represent the slope and the y-axis intercept, respectively. A non-zero value of ${\mathcal{L}}_{\infty }$ at $\langle x\left(0\right)\rangle =0$ is caused by the adjustment (oscillation and damping) of the width of the PDFs in time due to the disparity between the width of the initial and equilibrium PDFs (see Figure 4b). In other words, even though the mean values remain in equilibrium for all time ${[\langle x\left(0\right)\rangle ,\langle v\left(0\right)\rangle ]}^T={lim}_{t\to \infty }\langle \mathbf{x}\left(t\right)\rangle ={[0,0]}^T$, the information length (11) depends on the covariance matrix $\Sigma$ which changes from its initial value to the final equilibrium value as follows

$$\Sigma \left(0\right)=\left[\begin{array}{cc}0.01& 0\\ 0& 0.01\end{array}\right]\mathrm{to}\underset{t\to \infty }{lim}\Sigma \left(t\right)=\left[\begin{array}{cc}\frac{D}{\gamma {\omega }^2}& 0\\ 0& \frac{D}{\gamma }\end{array}\right].$$

On the other hand, ${\mathcal{L}}_{\infty }$ against $\langle x\left(0\right)\rangle$ shows parabolic behaviour for small $\langle x\left(0\right)\rangle <0.1$ in Figure 4e. This is caused by the finite width $0.1=\sqrt{{\Sigma }_{11}\left(0\right)}=\sqrt{{\Sigma }_{22}\left(0\right)}$ of the initial $p(x,0)$; we see that $\langle x\left(0\right)\rangle <0.1$ is within the uncertainty of the initial $p(x,0)$.

Similarly, Figure 4h,i exhibit a minimum in ${\mathcal{L}}_{\infty }$ at the equilibrium $\langle v\left(0\right)\rangle =0$ (recall $\langle x\left(0\right)\rangle =0$ in this case); ${\mathcal{L}}_{\infty }$ is a linear function of $\langle v\left(0\right)\rangle$ for $\langle v\left(0\right)\rangle \gg 0.1$ described by ${\mathcal{L}}_{\infty }(v\left(0\right),\gamma )=H\left(\gamma \right)|\langle v\left(0\right)|+F\left(\gamma \right)$ (again parabolic for $\langle v\left(0\right)\rangle <0.1$, see Figure 4i). Here again, $H\left(\gamma \right)$ and $F\left(\gamma \right)$ are constant functions depending on $\gamma$ for a fixed $\omega$ which represent the slope and the y-axis intercept, respectively.

Finally, Figure 4j shows in logarithmic scale that the minimum value of ${\mathcal{L}}_{\infty }$ at $\langle x\left(0\right)\rangle =\langle v\left(0\right)\rangle$ monotonically increases with $\gamma$.

4.4. The Limit Where $\omega \to 0$

When the natural frequency $\omega =0$ (i.e., damped-driven system like the O-U process [36]) in Equation (23), the two eigenvalues of the matrix $\mathbf{A}$ become ${\lambda }_1\to -\gamma$ and ${\lambda }_2\to 0$. It then easily follows that

$$\langle \mathbf{x}\left(t\right)\rangle =\left[\begin{array}{c}\frac{v\left(0\right)-e^{-\gamma t}v\left(0\right)}{\gamma }+x\left(0\right)\\ e^{-\gamma t}v\left(0\right)\end{array}\right],$$

(33)

and $\Sigma \left(t\right)$ is composed by the elements

$$\begin{array}{ccc}\hfill {\Sigma }_{11}\left(t\right)& =& \frac{e^{-2\gamma t}\left(-D+{\Sigma }_{22}\left(0\right)\gamma +e^{\gamma t}(4D-\gamma (2{\Sigma }_{22}\left(0\right)+({\Sigma }_{12}\left(0\right)+{\Sigma }_{21}\left(0\right))\gamma ))+e^{2\gamma t}(\gamma ({\Sigma }_{22}\left(0\right)+\gamma ({\Sigma }_{12}\left(0\right)+{\Sigma }_{21}\left(0\right)+{\Sigma }_{11}\left(0\right)\gamma ))+D(2\gamma t-3))\right)}{{\gamma }^3},\hfill \\ \hfill {\Sigma }_{12}\left(t\right)& =& \frac{e^{-2\gamma t}\left(D{\left(-1+e^{\gamma t}\right)}^2-{\Sigma }_{22}\left(0\right)\gamma +e^{\gamma t}\gamma ({\Sigma }_{22}\left(0\right)+{\Sigma }_{12}\left(0\right)\gamma )\right)}{{\gamma }^2},\hfill \\ \hfill {\Sigma }_{21}\left(t\right)& =& \frac{e^{-2\gamma t}\left(D{\left(-1+e^{\gamma t}\right)}^2-{\Sigma }_{22}\left(0\right)\gamma +e^{\gamma t}\gamma ({\Sigma }_{22}\left(0\right)+{\Sigma }_{21}\left(0\right)\gamma )\right)}{{\gamma }^2},\hfill \\ \hfill {\Sigma }_{22}\left(t\right)& =& \frac{e^{-2\gamma t}\left(D\left(-1+e^{2\gamma t}\right)+{\Sigma }_{22}\left(0\right)\gamma \right)}{\gamma }.\hfill \end{array}$$

(34)

To investigate the case of $\omega \to 0$, we consider the scan over $D\in (0.0005,0.04)$ for the same parameter value $\gamma =2$, and the initial conditions as in Figure 1, apart from using $\omega =0$ instead of $\omega =1$. Figure 5 presents the results – snapshots of $p(\mathbf{x},t)$, time evolutions of $\mathcal{E}\left(t\right)$, $\mathcal{L}\left(t\right)$, and ${\mathcal{L}}_{\infty }=\mathcal{L}(t=50)$ against D in Figure 5a–e. In particular, in Figure 5e, we identify the dependence of ${\mathcal{L}}_{\infty }$ on D by fitting the results to the curve ${\mathcal{L}}_{=}8.99e^{-324.19D}+10.83e^{-12.24D}$.

5. Concluding Remarks

We have presented theoretical results of time-dependent PDFs and the information length for n-th order linear autonomous stochastic processes, which can be applied to a variety of practical problems. In particular, the information length diagnostics was found as a function of the mean and covariance matrices; the latter was further expressed in terms of the covariance matrix eigenvalues. A Specific application was made to a harmonically bound particle system with the natural oscillation frequency $\omega$, subject to a damping $\gamma$ and a Gaussian white-noise (Kramer equation). We investigated how the information length depends on $\omega$ and $\gamma$, elucidating the role of critical damping $\gamma =2\omega$ in information geometry. The fact that the information length tends to take its minimum value near the critical damping can be viewed as the simplification of dynamics and thus the decrease in information change due to the reduction of the two characteristic time scales associated with $\omega$ and $\gamma$ to the one value. On the other hand, the information length in the long time limit was shown to preserve the linear geometry associated with the Gaussian statistics in a linear stochastic process, as in the case of the O-U process.

Future works would include the exploration of our results when applied to high-dimensional processes and the extension of our work to a more general (e.g., finite-correlated) stochastic noise, non-autonomous systems or non-linearly coupled systems. In particular, it will be of interest to look for a geodesic solution in non-autonomous systems [9] with the help of an external force, optimization or guiding self-organization (multi-agent systems) as well as elucidating the role of critical damping and resonances in self-organization. In addition, it would also be interesting to utilize the results introduced in [39] to predict the bound on the evolution of any observable for the Kramers problem (23), and compare it with a natural observable in such a system, the energy, for instance.