A Dynamic Model of Information and Entropy

Abstract

We discuss the possibility of a relativistic relationship between information and entropy, closely analogous to the classical Maxwell electro-magnetic wave equations. Inherent to the analysis is the description of information as residing in points of non-analyticity; yet ultimately also exhibiting a distributed characteristic: additionally analogous, therefore, to the wave-particle duality of light. At cosmological scales our vector differential equations predict conservation of information in black holes, whereas regular- and Z-DNA molecules correspond to helical solutions at microscopic levels. We further propose that regular- and Z-DNA are equivalent to the alternative words chosen from an alphabet to maintain the equilibrium of an information transmission system.

1. Introduction

The close relationship between information and entropy is well recognised [1,2], e.g., Brillouin considered a negative change in entropy to be equivalent to a change in information (negentropy [3]), and Landauer considered the erasure of information $\mathsf{∆}I$ to be associated with an increase in entropy $\mathsf{∆}S$, via the relationship $-\mathsf{∆}I=\mathsf{∆}S$ [1,4]. In previous work, we qualitatively indicated the dynamic relationship between information and entropy, stating that the movement of information is accompanied by the increase of entropy in time [5,6], and suggesting that information propagates at the speed of light in vacuum c [5,6,7,8,9]. Opinions on the physical nature of information have tended to be contradictory. One view is that information is inherent to points of non-analyticity (discontinuities) [5,7,9] that do not allow “prediction”, whereas other researchers consider the information to be more distributed (delocalised) in nature [10,11,12]. Such considerations are akin to the paradoxes arising from a wave-particle duality, with the question of which of the two complements best characterises information. In the following analysis, by initially adopting the localised, point of non-analyticity definition of information, we ultimately find that information and entropy may indeed also exhibit wavelike properties, and may be described by a pair of coupled wave equations, analogous to the Maxwell equations for electromagnetic (EM) radiation, and a distant echo of the simple difference relationship above.

2. Analysis

We start our analysis by consideration of a meromorphic [13] function $\psi \left(z\right)$ in a reduced 1+1 (i.e., one space dimension and one time dimension) complex space-time $z=x+ict$, with a simple, isolated pole at $z_0=x_0+ic{t}_0$, as indicated in Figure 1:

$$\psi \left(z\right)=\sqrt{\frac{c{t}_0}{\pi }}\frac{1}{z-z_0}$$

(1)

Figure 1. Information I (integration over imaginary ct-axis) and entropy S (integration over real x-axis) due to a point of non-analyticity (pole) at the space-time position z₀.

Figure 1. Information I (integration over imaginary ct-axis) and entropy S (integration over real x-axis) due to a point of non-analyticity (pole) at the space-time position z₀.

The point of non-analyticity $z_0$ travels at the speed of light in vacuum c [5,6,7,8,9]. We note that $\psi \left(z\right)$ acts as an inverse-square law function, e.g., as proportional to the field around an isolated charge, or the gravitational field around a point mass. It is also square-integrable, such that ${\int }_{-\infty }^{\infty }{\rho |}_{t=0}dx=1$, and $ic{\int }_{-\infty }^{\infty }{\rho |}_{x=0}dt=1$, where $\rho ={\left|\psi \left(z\right)\right|}^2$, and obeys the Paley-Wiener criterion for causality [14]. We now calculate the differential information [5,15] (entropy) of the function $\psi \left(z\right)$. By convention, in this paper, we define the logarithmic integration along the spatial x-axis to yield the entropy S of the function; whereas the same integration along the imaginary t-axis yields the information I of the function. This can be qualitatively understood to arise from the entropy of a system being related to its spatial permutations (e.g., the spatial fluxions of a volume of gas), whereas the information of a system is related to its temporal distribution (e.g., the arrival of a data train of light pulses down an optical fibre.) Considering the differential entropy first, substituting Equation (1) with $t=0$ and calculating the sum of the residues of the appropriate closed contour integral in the z-plane, S is given by:

$$S=k\underset{-\infty }{\overset{\infty }{{\displaystyle \int }}}\rho {\log }_2\rho dx=k{\log }_{2}c{t}_0$$

(2)

where k is the Boltzmann constant. We have defined S using a positive sign in front of the integral, as opposed to the conventional negative sign [15], so that both the entropy and information of a space-time function are calculated in a unified fashion. In addition, this has the effect of aligning equation (2) with the 2nd Law of Thermodynamics, since S increases monotonically with increasing time $t_0$. Performing the appropriate closed contour integral in the z-plane with $x=0$ we find that the differential information is given by:

$$I=k\underset{-\infty }{\overset{\infty }{{\displaystyle \int }}}\rho {\log }_2\rho cdt=k{\log }_2{x}_0$$

(3)

Note that in (2) and (3) some constant background values of integration have been ignored, since they disappear in the following differential analyses. We see that the entropy and information are given by surprisingly simple expressions. The information is the logarithm of the space-time distance $x_0$ of the point of non-analyticity from the temporal t-axis; whilst the entropy content is simply the logarithm of the distance (i.e., time $c{t}_0$) of the pole from the spatial x-axis. The overall info-entropy $\mathrm{\Sigma }$ of the function $\psi \left(z\right)$ is given by the summation of the two orthogonal logarithmic integrals, $\mathrm{\Sigma }=I+iS$, where the entropy S is in quadrature to the information I, since I depends on the real axis quantity $x_0$, whereas S depends on the imaginary axis quantity $c{t}_0$. Since the choice of reference axes is arbitrary, we could have chosen an alternative set of co-ordinate axes to calculate the information and entropy. For example, consider a co-ordinate set of axes $x^{\prime }-ct\prime$, rotated an angle $\theta$ about the origin with respect to the original $x-ct$ axes. In this case, the unitary transformation describing the position of the pole $z_0^{\prime }$ in the new co-ordinate framework is:

$$\left(\begin{array}{c}c{t}_0^{\prime }\\ x_0^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}c{t}_0\\ x_0\end{array}\right)$$

(4)

Using (2) and (3), the resulting values of the information and entropy for the new frame of reference are given by:

$$I\prime =k{\log }_2{x}_0^{\prime }$$

(5a)

$$S\prime =k{\log }_{2}c{t}_0^{\prime }$$

(5b)

with overall info-entropy $\mathrm{\Sigma }\prime$ of the function again given by the summation of the two quadrature logarithmic integrals, $\mathrm{\Sigma }\prime =I\prime +iS\prime$. We next perform a dynamic calculus on the equations (5), with respect to the original $x-ct$ axes, using (4) to calculate:

$$\frac{\partial I\prime }{\partial x}=\frac{k}{x_0^{\prime }}\frac{\partial x_0^{\prime }}{\partial x}=\frac{k}{x_0^{\prime }}\frac{\partial x_0^{\prime }}{\partial x_0}=\frac{k}{x_0^{\prime }}\cos \theta$$

(6a)

$$\frac{1}{c}\frac{\partial I\prime }{\partial t}=\frac{k}{x_0^{\prime }}\frac{1}{c}\frac{\partial x_0^{\prime }}{\partial t}=\frac{k}{x_0^{\prime }}\frac{1}{c}\frac{\partial x_0^{\prime }}{\partial t_0}=\frac{k}{x_0^{\prime }}\sin \theta$$

(6b)

and:

$$\frac{1}{c}\frac{\partial S\prime }{\partial t}=\frac{k}{t_0^{\prime }}\frac{1}{c}\frac{\partial t_0^{\prime }}{\partial t}=\frac{k}{c{t}_0^{\prime }}\frac{\partial t_0^{\prime }}{\partial t_0}=\frac{k}{c{t}_0^{\prime }}\cos \theta$$

(7a)

$$\frac{\partial S\prime }{\partial x}=\frac{k}{c{t}_0^{\prime }}\frac{\partial t_0^{\prime }}{\partial x}=\frac{k}{c{t}_0^{\prime }}\frac{1}{c}\frac{\partial t_0^{\prime }}{\partial x_0}=\frac{-k}{c{t}_0^{\prime }}\sin \theta$$

(7b)

where we have ignored the common factor $\log 2$, and we have assumed the equality of the calculus operators $\partial /\partial x=\partial /\partial x_0$ and $\partial /\partial t=\partial /\partial t_0$, since the trajectory of the pole in the z-plane is a straight line with $dx=d{x}_0$, and $dt=d{t}_0$. Since the point of non-analyticity is moving at the speed of light in vacuum c [5,7,9], such that $x_0=c{t}_0$, we must also have $x_0^{\prime }=c{t}_0^{\prime }$, as c is the same constant in any frame of reference [16]. We can now see that equations (6) and (7) can be equated to yield a pair of Cauchy-Riemann equations [17]:

$$\frac{\partial I}{\partial x}=\frac{1}{c}\frac{\partial S}{\partial t}$$

(8a)

$$\frac{\partial S}{\partial x}=-\frac{1}{c}\frac{\partial I}{\partial t}$$

(8b)

where we have dropped the primes, since equations (8) are true for all frames of reference. Hence, the info-entropy function $\mathrm{\Sigma }$ is analytic (i.e., holographic in nature [13]), and the Cauchy-Riemann equations (8) indicate that information and entropy (as we have defined them) may propagate as waves travelling at the speed c. In the Appendix we extend the analysis from 1+1 complex space-time to the full 3+1 (three-space and one-time dimensions) case, so that it can be straightforwardly shown that Equations (8) generalise to:

$$\nabla \times \underset{\_}{I}=\frac{1}{c}\frac{d\underset{\_}{S}}{dt}$$

(9a)

$$\nabla \times \underset{\_}{\mathrm{S}}=\frac{-1}{\text{c}}\frac{\text{d}\underset{\_}{I}}{\text{dt}}$$

(9b)

where $\underset{\_}{I}$ and $\underset{\_}{S}$ are in 3D vector form. Further basic calculus manipulations reveal that the scalar gradient (divergence) of our information and entropy fields are both equal to zero:

$$\nabla \cdot \underset{\_}{I}=0$$

(10a)

$$\nabla \cdot \underset{\_}{\mathrm{S}}=0$$

(10b)

Together, Equations (9) and (10) form a set of equations analogous to the Maxwell equations [16] (with no sources or isolated charges, and in an isotropic medium). Equations (9) can be combined to yield wave equations for the information and entropy fields:

$${\nabla }^2\underset{\_}{I}-\frac{1}{c^2}\frac{d^2\underset{\_}{I}}{d{t}^2}=0$$

(11a)

$${\nabla }^2\underset{\_}{S}-\frac{1}{c^2}\frac{d^2\underset{\_}{S}}{d{t}^2}=0$$

(11b)

3. Discussion

As defined previously, the information and entropy fields are mutually orthogonal, since $\underset{\_}{I}\cdot \underset{\_}{S}=0$. Analogous to the Poynting vector describing energy flow of an EM wave, the direction of propagation of the wave is given by $\underset{\_}{I}\times \underset{\_}{S}$, e.g., see Figure 2. The dynamic relationship between I and S implies that information flow tends to be accompanied by entropy flux [5,6], such that the movement of data is dissipative in order to satisfy the 2nd Law of Thermodynamics. Analytic functions such as I and S only have points of inflection, so that, again, the 2nd Law with respect to the S-field is a natural consequence of its monotonicity. However, in analogy to an EM-wave, Equations (9) and (10) in combination imply that the sum of information and entropy ${\left|\underset{\_}{I}\right|}^2+{\left|\underset{\_}{S}\right|}^2$ may be a conserved quantity. Hence, the 2nd Law of Thermodynamics should perhaps be viewed as a conservation law for info-entropy, similar to the 1st Law for energy. This has implications for the treatment of information falling into a black hole.

Figure 2. (a) Right-handed polarisation helical info-entropy wave, propagating in positive $x_i$-direction given by $\underset{\_}{I}\times \underset{\_}{S}$. (b) Left-handed polarisation helical info-entropy wave travelling in same direction.

Figure 2. (a) Right-handed polarisation helical info-entropy wave, propagating in positive $x_i$-direction given by $\underset{\_}{I}\times \underset{\_}{S}$. (b) Left-handed polarisation helical info-entropy wave travelling in same direction.

The reason that the differential Equations (6) and (7) obey the laws of relativity is that they are simple spatial reciprocal quantities, with their ratios equivalent to velocities, such that the relativistic laws are applicable. This reciprocal space aspect implies that calculation of Fourier-space info-entropy quantities result in quantities proportional to real space, which are therefore also relativistically-invariant and holographic.

Equation (9a) shows that a right-handed helical spatial information distribution may be associated with an increase of entropy with time (Figure 2a); in contrast to a left-handed chirality where entropy may decrease with time (Figure 2b). A link is therefore possibly revealed between the high information density of right-handed DNA [18] and the overall increase of entropy with time in our universe [19]. A molecule such as DNA is an efficient system for information storage. However, our dynamic model suggests that information would radiate away, unless a localisation mechanism were present, e.g., the existence of a standing wave. Such waves require forward and backward wave propagation of the same polarisation. We see that the complementary anti-parallel (C2 spacegroup) symmetry of DNA’s double helix means that information waves of the same polarisation (chirality) would travel in both directions along the helix axis to form a standing wave, thus localising the information. Small left-handed molecules (e.g., amino acids, enzymes and proteins, as well as short sections of Z-DNA [20]) are known to exist alongside DNA within the cell nucleus. Their chirality may be thus associated with a decrease of entropy with time. In conjunction with right-handed molecules they can be understood to possibly regulate the thermodynamic function of the DNA molecule. As is well known, highly dynamic systems require negative feedback to direct and control their output, without which they become unstable. We can draw parallels to the high entropic potential of DNA, with left-handed molecules (as well as right-handed molecules) potentially acting as damping agents to control its thermodynamic action.

4. Conclusions

In conclusion, our model grounds information and entropy onto a physical, relativistic basis, and provides thermodynamic insights into the double-helix geometry of DNA. It offers the prediction that non-DNA-based life forms will still tend to have helical structures, and suggests that black holes conserve info-entropy.