Information in Biological Systems and the Fluctuation Theorem

Abstract

Some critical trends in information theory, its role in living systems and utilization in fluctuation theory are discussed. The mutual information of thermodynamic coupling is incorporated into the generalized fluctuation theorem by using information theory and nonequilibrium thermodynamics. Thermodynamically coupled dissipative structures in living systems are capable of degrading more energy, and processing complex information through developmental and environmental constraints. The generalized fluctuation theorem can quantify the hysteresis observed in the amount of the irreversible work in nonequilibrium regimes in the presence of information and thermodynamic coupling.

1. Introduction

Definition and quantification of information have created broad discussions; particularly, “information theory” in living systems is an evolving field [1–14]. Evolution is an axiomatic consequence of organismic information obeying the second law of thermodynamics; as entropy increases, the information within a biological system becomes more complex and diversifies at bifurcation points [15–18]. Behavior of living systems is decomposable into functional modes and the ensemble of modes at an agent’s disposal constitutes functional repertoire. The modes may be subjected to operational signals and a mechanism that sequentially selects a mode so that it controls the functional dynamics [18,19].

In conventional thermodynamics, the amount of entropy is independent of how the process is regarded as being divided into irreducible subsystems; the entropy of a system can be calculated from the entropies of its subsystems. If the subsystems are statistically independent that is not correlated by the mutual information, the entropy is additive. Shannon’s entropy of the total system is given by the sum of those of the subsystems [2,3,11], and is restricted to random variables taking discrete values. In small thermodynamics systems, such as at mesoscopic or nanoscopic scale, the mutual information can serve as a resource of the work or the free energy through information processing such as feedback control. Use of the mutual information as a source of entropy decrease accompanies no heat dissipation [3]. This is contrary to the conventional thermodynamics where the decrease in entropy is only because of heat dissipation.

An over damped motion x(τ) of a system in contact with a heat bath and a single continuous degree of freedom can be described by the Langevin equation ẋ = μF(x,y) + ζ. The systematic force F(x,λ) can arise from a conservative potential and/or be applied to the system directly as a nonconservative force, while ξ is the stochastic force, which is not affected by a time-dependent force, and μ is a positive constant. The Langevin dynamics generates trajectories x(τ) starting at x₀. For an arbitrary number of degrees of freedom, x and F become vectors. The Langevin equation is the generic equation of motion for a single fluctuating thermodynamic quantity such as the concentrations of the chemical species in the vicinity of equilibrium [4,16,17].

For a system in contact with a heat bath, symmetry of the probability distribution of entropy production in the steady state is known as the fluctuation theorem. Crook’s fluctuation theorem compares probability distributions for the work required in the original process with that of the time-reversed process. The probabilistic approach has gained broader appeal due to the advances in experimental techniques, such as atomic force microscopes and optical tweezers, for tracking and manipulation of single particles and molecules [20–25]. This study discusses some emerging efforts in information theory, its role in living systems, and its utilization in fluctuation theory. Relationship of thermodynamic coupling with the information theory and its consequence on the fluctuation theorem has been emphasized.

2. Information Theory

Information may be defined as the capacity to reduce statistical uncertainty in the communication of messages between a sender and a receiver [3–6]. Consider the number of ways W in which N distinguishable entities can be assigned to M distinguishable states such that there are n_i entities in state i:

$$W=\frac{N!}{n_1!n_2!\dots n_M!}$$

(1)

Probability p_i is related to maximum entropy H in the limit of large N and n_i. The asymptotic result from Stirling’s approximation (ln N! ≈ N ln N) yields:

$$\frac{1}{N}\text{ln}\hspace{0.17em}W=-{\sum }_i^M{p}_i\hspace{0.17em}\text{ln}\hspace{0.17em}{p}_i=H$$

(2)

where the occupation frequency of state i is p_i = n_i/N [25].

In Shannon’s theory, entropy represents the amount of uncertainty one particular observer has about the state of the system [3,5]. For a variable X with the x₁, x₂,…, x_N of its N possible states, the probability of finding X in state x_i would be p_i and Shannon’s entropy H of X is $H(X)=-{\sum }_i^N{p}_i\hspace{0.17em}\text{ln}\hspace{0.17em}{p}_i$. If nothing is known about X, we have H(X) = ln N, which is the maximum value that H(X) can be; this occurs if all the states are equally likely p_i = 1/N. However, for example, if X = x₅ then the uncertainty about X becomes smaller, and therefore H(X) represents the closest description of X. The probability distribution using prior knowledge or measurements can teach us something about a system. The difference between the maximal and the actual entropy after our measurements or analysis is the amount of information we have for the system. As it measures the difference of uncertainty, information is a relative quantity [5]. Using the problem of ion channel thermodynamics, for example, complex thermodynamic models can be built by successively adding physical information [23].

If we define another random variable Y with its states y₁, y₂,…,y_M and probabilities p₁, p₂,…, p_M, then the joint entropy H(X,Y) measures our uncertainty about the joint system XY in N·M states. If X and Y are somehow connected, such as two molecules that can bind to each other, the information that one molecule has about the other is:

$$I(X:Y)=H(X)+H(Y)-H(XY)$$

(3)

Here “:” shows that information is symmetric; X and Y equally knows each other. If the state of Y is known, then the so called “conditional entropy” becomes:

$$H(X/Y)=H(XY)-H(Y)$$

(4)

For independent variables X and Y, we get H(XY) = H(X) + H(Y). With the conditional entropy, Equation (3) becomes:

$$I(X:Y)=H(X)-H(X/Y)$$

(5)

The equation above shows that information measures deviation from independence that is the amount by which the entropy of X or Y is reduced by knowing the other [5]. Maximization of the information entropy (IE) determines the probability of a particular state of the system. This leads to the relation between the probability of a nonequilibrium system and the number of microscopic trajectories [23–26].

2.1. Information and Thermodynamics

Maximum entropy production (MEP) may be an organizational principle applicable to physical and biological systems. Nonequilibrium thermodynamics and MEP can be used to derive overall evolutionary trends of Earth’s past; for example, they can identify the role that life plays in driving thermodynamic states far from equilibrium [3,6–13,27–35]. Various derivations of MEP by using the maximum information entropy (MIE) procedure by Jaynes [36] exist in the literature [2]. In these derivations the information entropy (IE) is not defined by a probability measure on phase space, but on path space for the stationary nonequilibrium systems [26].

Implementation of the MEP principle also includes the steepest entropy ascent formalism based on quantum thermodynamics approach [37–39]. Beretta [37] has examined the steepest entropy ascent formalism for transition between states and adopted a geometrical view of the dynamics in probability space based on identifying a unique path of locally maximal entropy increase.

Consider M sites with a variable n_i(t) (i = 1,2,…,M) at each site with t = 0,1,…,τ. The flux (time asymmetric) occurring randomly at every time step, J_ij = − J_ji from i to j depends on a parameter c_ij = c_ji, such that J_ij(t) = ± c_ij with stochastic sign. A microscopic path a is a set of values ± c_ij so that n_i,a (t+1) −n_i,a(t)= −∑_j J_ij,a(t). The path dependent time average flux is J̄_ij,a = (1/τ)∑_t J_ij(t) and n_i(0) does not depend on the complete path. With the microscopic path dependent probability p_a, the path ensemble average is 〈J̄_ij〉 = ∑_ap_aJ̄_ij,a. By using Jaynes’ information theory and maximizing the path information entropy [2,24,25]:

$$S_I=-{\sum }_a{p}_a\hspace{0.17em}\text{ln}\hspace{0.17em}{p}_a$$

(6)

with the constraints [2,25]:

$$1={\sum }_a{p}_a$$

(7)

$$\langle n_i(0)\rangle ={\sum }_a{p}_a{n}_{i,a}(0)$$

(8)

$$N_{ij}=-{\sum }_a{p}_a{\overline{J}}_{ij,a}$$

(9)

the most likely probability on the path space is estimated as:

$$p_a=\frac{1}{Z}\text{exp}\hspace{0.17em}{A}_a$$

(10)

where N_ij is the numerical value of the time and path ensemble average of the flux J_ij, A_a is the path action A_a = ∑_i λ_i n_ia(0) + ∑_ij n_ij J̄_ij,a in which λ_i and n_ij = − n_ji are the Lagrange multipliers of constraints (8) and (9), respectively, and Z is the partition function [2,24,25].

However, a trajectory of a quantity possessed by a system may fluctuate wildly (if far from equilibrium) or weakly; than they would not have the same probabilities as long as they have the same initial and final states. Here the path trajectory a is a sequence of p over some time interval with M sites:

$$a\equiv [p(0),p(dt),p(2dt),\dots ,(Mdt)]$$

(11)

where dt is the coarse graining corresponding to the time scale of experimental observations [4].

The partition sum and the constitutive equation of motion have the relations:

$$\frac{\partial \hspace{0.17em}\text{ln}\hspace{0.17em}Z}{\partial n_{ij}}=N_{ij}$$

(12)

$$X_{ij}=\frac{n_{ij}}{\tau }$$

(13)

The forward and backward components of the time and ensemble averaged fluxes are:

$$N_{ij}=N_{ij}^f-N_{ij}^b=\frac{c_{ij}{e}^m}{e^m+e^{-m}}-\frac{c_{ij}{e}^{-m}}{e^m+e^{-m}}$$

(14)

where m = X_ijc_ij and $2m=\text{ln}\hspace{0.17em}\left(N_{ij}^f/N_{ij}^b\right)$.

The entropy production of a microscopic path a is [2]:

$${\sigma }_a={\sum }_{ij}{X}_{ij}{N}_{ij}$$

(15)

By using the most likely probability on the path space p_a (given in Equation (10)) in Equation (6), the maximum information entropy as a function of the force X becomes:

$$S_{I,\text{max}}(X)=\text{ln}\hspace{0.17em}Z(X)-\langle A(x)\rangle \approx \text{ln}\hspace{0.17em}W\left(\langle A(X)\rangle \right)$$

(16)

where W (〈A(X)〉) is the density of paths, W is the number of paths, and (〈A(X)〉) is the average path action.

The entropy curvature (response) matrix and the probability distribution for the time averaged flux J̄ are [2]:

$$A_{ij,kl}(N)=\frac{\partial X_{ij}}{\partial N_{kl}}=-\frac{{\partial }^2{S}_{I,\text{max}}(X(N))}{\tau \partial N_{ij}\partial N_{kl}}$$

(17)

$$p(\overline{J})\propto \text{exp}\left(-\frac{\tau }{2}{\sum }_{ij,kl}[{\overline{J}}_{ij}-N_{ij}]A_{ij,kl}(N)[{\overline{J}}_{ij}-N_{kl}]\right)$$

(18)

In near equilibrium regime that is for small X, the maximum path information becomes:

$$S_{I,\text{max}}(X)=\text{ln}(W)\approx \text{ln}\hspace{0.17em}{2}^{\tau (M^2-M)}+\tau \sigma /2$$

(19)

The first part on the right side of the equation above is the logarithm of the total number of paths for uniform probability distribution, while the second term is the entropy production. In the MEP, the assumption is that the number of paths W should be an increasing function of the averaged action [24–26]. Here, for the higher value of entropy production, the value of S_I is lowered [2].

MEP principle states that if thermodynamic force X_i is preset, then the true thermodynamic flow J_i satisfying the condition for the local rate of entropy production σ_τ:

$${\sigma }_{\tau }={\sum }_i{J}_i{X}_i\ge 0$$

(20)

yields the maximum value of the σ_τ(J). This can be written using the Lagrange multiplier λ and fixed force:

$${\delta }_j{[{\sigma }_{\tau }(J_k)-\lambda ({\sigma }_{\tau }(J_k)-{\sum }_i{J}_i{X}_i)]}_X=0$$

(21)

In the meantime, the relationship between the fluxes and forces:

$$X_i=\frac{\sigma (J)}{{\sum }_i{J}_i(\partial \sigma /\partial J_i)}(\partial \sigma /\partial J_i)$$

(22)

indicates that this relationship can be both linear and nonlinear [23].

The same entropy production can be both maximum and minimum depending on the constraints used in the entropy production variation. However, it is widely published that the MEP principle may be a critical link in the explanation of the direction of the biological evolution under the imposed constraints of the environment [22,23,26]. If X is fixed, the MEP leads to maximum J that is the selection of fastest process. MEP principle has proved to be valuable for understanding and describing of various nonequilibrium systems in physics, biology, and environment. The local equilibrium hypothesis of a nonequilibrium system and the representation of the entropy production as a bilinear form of flows and forces are mandatory conditions for the use of MEP principle [23–31].

In the cortex, populations of neurons continuously receive input signals from other neurons, interpret their ongoing activity, and generate output signals destined for other neurons. This information processing and transmission is limited by the repertoire of different activated configurations available to the population. The extent of this repertoire may be estimated by its entropy H characterizing the information capacity as the upper limit on the aspects of information processing of the population. When the information transmitted from the input to the output by a population that has only two states in its repertoire (H = 1 bit), then regardless the information the input contains, the output information content cannot exceed 1 bit. Therefore, a network with low entropy population may limit information transmission. Activity in the cortex depends on the ratio of fast excitatory E_x to inhibitory I_n synaptic signals to neurons. The E_x/I_n ratio remains fixed at an average in various events during highly fluctuating activity levels; yet a small E_x/I_n ratio (caused, for example, by weak excitation drive) may reduce the correlations as well as the overall level of activity [7].

For a number of unique binary patterns, the entropy is:

$$H=-{\sum }_{i=1}^n{p}_i\hspace{0.17em}{\text{log}}_2\hspace{0.17em}{p}_i$$

(23)

where p_i is the probability that pattern i occurs. Equation (23) estimates the occurrence probability for each pattern. Maximization of entropy may be an organizing principle of neural information processing systems [7]. The information capacity I_C in binary units may be expressed as a function of the probability p:

$$I_C=\frac{1}{\text{ln}\hspace{0.17em}2}\left(\sum _{j=1}^{\mathrm{\Omega }}{p}_j\hspace{0.17em}\text{ln}\hspace{0.17em}{p}_j-\sum _{j=1}^{\mathrm{\Omega }}{p}_j^o\hspace{0.17em}\text{ln}\hspace{0.17em}{p}_j^o\right)$$

(24)

where Ω is the number of possibilities, p∘ is the probability at equilibrium (i.e., no knowledge), and p is the probability when some information are available about the system. Information here is used as a measure of structure [1,16].

3. Biological Systems

3.1. Genetic Information Use in Protein Synthesis

Genes are sequences of deoxyribonucleic acids (DNA) nucleotides composed of four bases (guanine G, adenine A, cytosine C, thymine T). Genes carry and transmit the information to specify 20 amino acids for protein synthesis. The genome refers collectively to the total genetic information coded in a cell. Ribonucleic acid (RNA) molecules transfer information from DNA to the site of protein synthesis [8–14]:

$$\text{DNA}\stackrel{\text{transcription}}{\to }\text{mRNA}\stackrel{\text{translation}}{\to }\text{Protein}$$

(25)

During transcription, one of the strands of the DNA acts as a template and determines the sequence of RNA nucleotides. RNA also contains the base uracil U in place of thymine T. The transcribed RNA binds to ribosomes in the cytoplasm where proteins are synthesized from the encoded information. Only exon segments actually code for amino acids, and a continuous sequence of exons becomes mRNA, which binds to a ribosome. In translation, each ribosome is composed of proteins and ribosomal RNA called rRNA. Transfer RNA (tRNA) is the link between an amino acid and its mRNA codon. Each amino acid brought to a tRNA molecule is linked by a peptide bond to the end of the growing protein chain. The completed protein then undergoes folding. Signals from within or outside the cell can turn on or off the transcription of genes. Homeostatic control systems regulate compensatory responses in which the set point of a variable is controlled. In case of a continuing perturbation, the error signal (negative or positive feedback) may lead to resetting the set point [8–14].

A stimulus is a detectable change in the internal or external environment. A receptor detects the stimulus and produces a signal that is relayed through various pathways to the integrating center, which receives signals from many receptors. The output of the integrating center reflects the net effect of the total afferent input. The output is sent through the efferent pathway to an effector.

Some of the intercellular chemical messengers are hormones, neurotransmitters, and paracrine agents, which are synthesized by cells and diffuse to target cells upon receipt of an appropriate stimulus. Autocrine agents are local chemical messengers that act upon the same cells that secreted them. Many homeostatic systems add and remove different chemicals in response to a particular stress; it is mostly reversible without the involvement of genetic change.

Receptors are glycoproteins on plasma membranes of target cells and bind specific messengers. The binding leads to response through the signal transduction pathways that would vary in different cell types. For example, neurons generate electric signals that pass from one cell to another and release chemical messengers called neurotransmitters to communicate with other cells. A synapse is a junction between two neurons and its influence can be either excitatory or inhibitory. If many presynaptic cells affect a single postsynaptic cell it allows information from many sources to influence the activity of one cell. On the other hand, if a single presynaptic cell affects many postsynaptic cells it allows one information source to affect multiple pathways [8,10].

3.2. Information and Biological Systems

DNA is a code, and codes from sequence alone do not reveal information. DNA replication starts with an initiator protein, which unwinds a short stretch where another protein known as “helicase” attaches to and breaks apart the hydrogen bonds between the bases on the DNA strands. The nonconditional entropy for DNA sequence or proteins is about two bits per base; a random protein would have log₂(20) = 4.32 bits of entropy per site. Due to repetitions, pair, and triplet correlations the actual entropy would be lower [5]. This entropy per symbol only allows us to estimate our uncertainty about the sequence identity; it will not reveal the “function” of the genes. Estimates of the entropy reduction and information aspects of a living cell are becoming possible in the context of cell’s metabolic activity and hence its maintenance of the nonequilibrium state [10–14,40,41].

In equilibrium thermodynamics, isolated systems have the maximum entropy and there are no correlations between the states; hence there is no information. The information as the amount of correlations between two systems stored in living system (biological genomes) points out that they are far away from equilibrium. Consequently, the information theory becomes a part of nonequilibrium thermodynamics in living cells. Information measures the amount of entropy shared between two systems. Also, information enables us to make predictions about other systems, since only in reference to another ensemble entropy can become information. Therefore, what is described by the correlations between the sequences stores information not the sequence itself. On the other hand, what information a genomic sequence represents depends on the interpreter environment. Therefore, if a sequence means something it can create a function (response) necessary for its environment [6,14].

The information theory introduced “functional information” that leads to self-organizing capabilities of living systems, and “instructional information” that is a physical array. However, linkages with the field of semiotics established a much more compatible approach to biological information [8]. Within this trend “control information” is defined as the capacity to control the acquisition, disposition, and utilization of matter, energy, and information flows functionally.

Functional information I(E_x) represents the probability that an arbitrary configuration of a system (i.e., complex biological systems of many interacting components) will achieve a specific function to a specified degree; therefore, it is as a measure of system complexity. For a given system and function x (e.g., a folded RNA sequence that binds to guanosine triphosphate GTP), and degree of function E_x (e.g., the RNA-GTP binding energy), functional information is I(E_x) = − log₂[F(E_x)], where F(E_x) is the fraction of all possible configurations of the system [14,32–34]. Functional information, for example, analyzes RNA structures that bind target ligands and RNA structures that catalyze specific reactions. Thus, the degree of function E_x of these linear sequences of RNA letters (the four bases adenine, cytosine, guanine, and uracil or thymine A, C, G, and U or T) can be defined quantitatively as the binding energy to a particular molecule or the catalytic activity increase in a specific reaction rate [10,14].

Instead of a single RNA nucleotide, a minimum sequence length (n_min nucleotides) would achieve any significant degree of ribozyme function, E_x > E_min. Increasing the number of nucleotides (n > n_min) will generally lead to more functional sequences. Consequently, for a maximum possible degree of specific function E_max there will be an optimal RNA sequence of length n_opt possessing the maximum functional information of I_max (E_max) = −log₂ [1/(∑_1−nopt(4ⁿ))]. For E_x < E_max, an intermediate functional information shows [I(E_x) < I_max(E_max)] [14].

Each position on the genome is four-base code and the uncertainty at each position is two bits; then the maximum entropy becomes:

$$H_{\text{max}}=-\sum _{i=G,C,A,T}p(i)\hspace{0.17em}{\text{log}}_2\hspace{0.17em}p(i)={\text{log}}_2(4)=2\hspace{0.17em}\text{bits}$$

(26)

since p(i) = ¼. The actual entropy is obtained from the actual probabilities p_j(i) for each position j on the sequence. In N sequences, we have p_j(i) = n_j (i)/N by counting the number of n_j(i) occurrences of nucleotide i at position j (this will be done for all positions j = 1,…,M on the sequence length M). When we ignore correlations between positions j, the information stored in the sequence becomes:

$$I=H_{\text{max}}-H=2M-H\hspace{0.17em}\text{bits}$$

(27)

where $H=-{\displaystyle \sum _{j=1}^M}{\displaystyle \sum _{i=G,C,A,T}}{p}_j(i)\hspace{0.17em}{\text{log}}_2\hspace{0.17em}{p}_j(i)$.

The thermodynamics of protein structures implies that sequence and structure are related. If a structural entropy of proteins H(str) is obtained for a given chain length and environment, the mutual entropy between structure and sequence becomes [8]:

$$I(\text{seq}:\text{str})=H(\text{seq})-H(\text{seq}/\text{str})$$

(28)

where H(seq) is the entropy of sequences of length M and H(seq/str) is the entropy of sequences based given the structure. If the environment requires a certain structure that will be functional in that environment then H (seq/str) ≈ H (seq/env) and I (seq:str) is approximately equal to the physical complexity. Assuming that any given sequence produces an exact structure H(str/seq) = 0, then Equation (28) becomes:

$$I(\text{seq}:\text{env})\approx I(\text{seq}:\text{str})=H(\text{str})$$

(29)

Therefore, thermodynamic entropy of a protein structure is limited by the amount of information about the environment coded by the sequence. This may imply that sequences that encode more information about the environment may be more functional.

One of the consequences of the Human Genome project has proved that biology is an “informational science” [5,9]. The communication in living cells is based on the signals, such as electromagnetic-light, mechanical-touch, and chemicals received. In the signal-transduction pathway, a signal on a cell surface is converted into a specific cellular response in a series of functional steps [7]. This suggests that information is conceived as the communication of a form from object to interpreter through the sign. The evolution of ways of storing, transmitting, and interpreting information can be seen a major step in the increased capacity for collective behavior and robustness in living systems [10,11,29]. Information can be introduced into, or extracted from through the agencies intentionally. A sophisticated information organizing capacity of biological origin emerges from the uncertainty inherent in the complexity of all living systems [12–14].

In semiotic understanding of living systems, interpreters of signs and information will often be an interpreter-dependent objective process. Genes should be regarded as signs in DNA, which can only have any effect on a cell function through a triadic-dependent process that is the communication of a form from the object to the interpretant through the mediation of the sign [8]. The object of sign in DNA is a functional, folded, and chemically configured protein production; when a particular gene product is necessary, a signal from the environment activates the expression of a certain gene. The cell as an interpreter alters its internal states triggered by a collective signal transduction pathway to establish the boundary conditions to processes and perform functional process with the genetic material [8,14].

There are two aspects of molecular and supramolecular information transduction: (i) sphingomyelinase activity leading to a lipid-mediated cross-communication between the sphingomyelinase and phospholipase A₂ pathways, and (ii) the compositionally driven lateral organization of whole glial and neuronal membrane interfaces leading to the differential responses. The interfacial properties may express and regulate the generation, transduction, and amplification of molecular and submolecular properties on a temporal and structural scale mainly at mesoscopic level. However, the transduction of information mainly occurs between the microscopic (<10⁻⁶ s, <1 nm) and the macroscopic (>10⁻¹ s, >500 nm) ranges [42].

A physiological mechanism may exist to ensure fast transmission of information to that hemisphere which is more efficient in its processing [43]. This shows how functional information is transmitted by parts to produce a response that serves a purpose for the whole. This is the result of coordinated awareness between the parts processing particular information and the parts receiving the transmitted information for preparing a functional output (response) for the whole.

Signal transduction includes the interrelation and information exchange between phosphohydrolytic pathways producing lipid mediators of signal transduction whose formation modifies the interfacial composition. At the molecular level, there exists cross-talk between a glial cell and the axonal membrane surface. There also exists the subtle intercommunication and response of functional living cells to information contained in the molecular organization of contacting cellular membranes [42].

3.3. Thermodynamically Coupled Biological Systems and Information

Thermodynamic coupling [16,17,32,44–49] materialized by certain coupling mechanisms (i.e., configurations, ATPase) has the ability to process information and hence the lower the entropy production σ = ∑_iJ_iX_i > 0. This may lead to self-organization because a process with σ₁ < 0 (unnatural) is thermodynamically coupled with another process with σ₂ >> 0 (natural) to achieve a desired output function; therefore, the coupled system satisfies the second law of thermodynamics [(σ₁ < 0) + (σ₂ >> 0)] > 0. Here, the thermodynamic coupling refers that a flux J occurs without its primary thermodynamic driving force X, or opposite to the direction imposed by the driving force. This is consistent with the second law that states that a finite amount of organization may be obtained at the expense of a greater amount of disorganization in a series of coupled spontaneous processes [16,48,49]. The information processed in the thermodynamic coupling is the functional information, which in turn may lower the total entropy [32]. The “complexity” in (biological) systems that display some of the examples of thermodynamic coupling probably refers to the mechanisms enabling the discrete parts of the whole to be aware of each other hence leading to a coherent and collective behavior to achieve the best degree of desired function [1,3,16,44]. This also indicates that entropy productions of various coupled processes are additive.

Biochemical reactions coupled with diffusion of species can lead to molecular pumps and biochemical cycles, such as the adenosine triphosphate (ATP) synthesis coupled to the respiratory electron transport. This shows a functional process leading to organized structures where the ATP synthesis (σ < 0) has been made possible and the coupled processes satisfy ∑ > 0 [17,31–35]. The ATP synthesis, in turn, is matched and synchronized to cellular ATP utilization processes, since energy and matter flows must be directed by the information for them to be functional and serve a purpose. For living systems, the information is the cause and the entropy reduction is the result [34–36,40–47]:

$$\text{Energy}+\text{Matter}+\text{Information}\to \text{Locally\hspace{0.17em}reduced\hspace{0.17em}entropy}\hspace{0.17em}(\text{Increase\hspace{0.17em}of\hspace{0.17em}order})$$

The general approach for incorporating thermodynamics into the information theory has been to derive probability distributions for nonequilibrium steady states by employing the variational principle. However, composing the appropriate constraints to be used in the variational principle is not clear, since there is no definite extremum quantity to characterize the state space of such steady nonequilibrium states. In the vicinity of equilibrium only, the linear phenomenological laws may be useful in that respect [4,16,17]. Therefore, a natural question is that how useful such an approach would be to describe the information processing in functionally coupled and self-organized biochemical cycles of living systems that are mainly far from equilibrium? The probabilistic measure of information derived from Jaynes’ information theory formalism of statistical mechanics is mainly indifferent to meaning [30].

The unified theory of evolution attempts to explain the origin of biological order as a manifestation of the flows of energy, matter, and information on various spatial and temporal scales. Genes originates the information to synthesize the required enzymes, regulatory and structural proteins. The genome is the source of cellular codes; also any cellular structure such as lipids and polysaccharides may store and transmit information. Besides, thermodynamic forces in the form of transmembrane gradients of H⁺, Na⁺, K⁺, Ca²⁺ and consequent electric potential cause significant displacements from equilibrium, and are therefore, potential sources of information. Genome-protein system may be a component of a large ensemble of cellular structures, which store, encode, and transmit the information [5–10].

Le Chatelier’s principle may be applied to analyze how a protein-signaling network at equilibrium returns to its equilibrium state after being slightly perturbed [20]. For a single cell or small cell colony, cell to cell perturbations are small, while the unperturbed state of a single cell may be unstable in the presence of many other cells. Experiments permit observations of the covariance in the fluctuations and evolution of these fluctuations of different proteins when a single cell is perturbed in the presence of other cells. The information theory helps analyze this covariance to understand the network of interacting proteins [36]. Signal transduction inside and across the cells is complex and cooperative, and occurs in a far-from thermodynamic equilibrium. It is also driven by the activation kinetics strongly dependent on the local energetics [7,20,45].

The composite immediate object of a protein coding gene is the sequence of amino acids of a polypeptide, which can be folded in different ways in different cellular contexts and represents dynamical objects. So sign is a sequence of nucleotides in DNA and becomes effective information only when it is used by an active, coordinated, and coupled system of interpretation in the cell. This means that a range of sequence of amino acids required by the environment (cell) has been constructed [8].

The use of maximum entropy formalism in biology is expanding [23–31] toward detecting expression patterns in signal transduction. At the maximum entropy, the probabilities of the different proteins are not equal; each protein will be present in proportion to its partition function, which is the effective thermodynamic weight of a species at thermal equilibrium. MEP may be the unifying optimization for living systems and ecosystem function, in which entropy production might be a general objective function [24–31,46]. It is stated that “the nonequilibrium state of MEP is the most probable as it can be realized microscopically in a greater number of ways than any other nonequilibrium state.” This statement is not universally accepted [46]. In this sense, MEP is a statistical principle, rather than a physical principle open to experimental validation. MEP may predict the behavior of optimal living system from the perspective of natural selection as well as offer a novel statistical reinterpretation of that behavior that is the survival of the likeliest [29].

For a multicomponent fluid system under mechanical equilibrium with n species and N_r number of chemical reactions, the rate of energy dissipation due to local rate of entropy production σ_τ is [16,49,50]:

$$T{\sigma }_{\tau }={\int }_V\left(-{\sum }_i{\mathbf{J}}_i·{(\nabla {\mu }_i)}_{T,P}+{\sum }_{i,j}{\mu }_i{\nu }_{ij}{J}_{rj}\right)dV\ge 0$$

(30)

where J_i is the vector of mass fluxes, μ_i is the chemical potential of species i, and A is the chemical affinity A = −∑ν_iμ_i. The local mass balance of chemical species i is [16,49]:

$$\rho \frac{\partial w_i}{\partial t}=-\nabla ·{\mathbf{j}}_i+{\sum }_{i,r}{\nu }_{ir}{J}_r$$

(31)

For a steady state system, we have ∇ · j_i = ∑_i,jν_ijJ_rj allowing the dissipation to be expressed fully in terms of chemical affinity [29] of many coupled chemical reactions of biochemical cycles.

Considering N number of linear flux-force system J_i = ∑ L_ikX_k [49–54] expressed in matrix form −J = LX in a biochemical cycle, the local rate of entropy production in terms of forces X becomes [16]:

$${\sigma }_{\tau }=\sum _{i,k=1}^N{L}_{ik}{X}_i{X}_k=[X_1\hspace{0.17em}{X}_2\cdots X_N]\hspace{0.17em}\left[\begin{array}{l}{L}_{11}\hspace{0.17em}{L}_{12}\cdots L_{1N}\hfill \\ L_{21}\hspace{0.17em}{L}_{22}\cdots L_{2N}\hfill \\ \dots \dots \dots \dots \dots \dots \mathrm{..}\hfill \\ L_{N1}\hspace{0.17em}{L}_{N2}\cdots L_{NN}\hfill \end{array}\right]\hspace{0.17em}\left[\begin{array}{l}{X}_1\hfill \\ X_2\hfill \\ \cdots \hfill \\ X_N\hfill \end{array}\right]\ge 0$$

(32)

Onsager’s reciprocal relations state that the coefficient matrix L is symmetrical. The L will have N × N elements. The number of elements representing the cross coefficients responsible for the induced effects because of the thermodynamic coupling would be (N²−N)/2. Through the coupling-uncoupling mechanisms, these cross coefficients may be on and off based on the biochemical path and its environment, and leads to (N²−N)/2 bit information. In the absence of pertinent symmetries or invariances, all types of cross-couplings are possible and lead to nonvanishing cross coefficients. If the structure of the system is invariant with respect to some or all of the orthogonal transformations, then the invariance will eliminate certain cross-couplings and their cross-coefficients will vanish.

Thermodynamic coupling may lead to (N²−N)/2 number of possibilities of coupled-uncoupled structures emerging in an N-flux-force system in a biochemical cycle depending on the environmental interpretations. This, in turn, brings out the challenge of implementing the trajectories belonging two or much more coupled processes (correlated with mutual information) with different initial and nonequilibrium states into the fluctuation and information theories.

4. Fluctuation Theorem

Fluctuation theorem (FT) relates the probability p(σ_τ) of observing a phase-space trajectory with entropy production rate of σ_τ over time interval τ, to that of observing a trajectory with entropy production rate of −σ_τ:

$$\frac{p({\sigma }_{\tau })}{p(-{\sigma }_{\tau })}=\text{exp}(\tau {\sigma }_{\tau })$$

(33)

where the Boltzmann constant k_B = 1. The probability distribution of the rate of entropy production becomes p(∑_τ) = p(−∑_τ)exp(τσ_τ). This result describes how the probability of violations of the second law of thermodynamics becomes exponentially small as τ or the system size increases. FT relates the work along nonequilibrium trajectories to the thermodynamic free energy differences, and applicable to single molecule force measurements. FT depends on the following assumptions: the system is finite and in contact with a set of thermal baths; the dynamics are required to be stochastic, Markovian, and microscopically reversible. The probabilities of the time-reversed paths decay faster than the probabilities of the paths themselves and the thermodynamic entropy production arises from the breaking of the time-reversal symmetry of the dynamical randomness. Fluctuations encode fundamental aspects of the physics of a system and are crucial to understand irreversibility and nonequilibrium behavior, which may be unique for that system [3,21–25].

Crook’s FT can be used to determine free energies of folding and unfolding processes occurring in nonequilibrium systems. For that, the unfolding and refolding processes need to be related by time-reversal symmetry, i.e., the optical trap used to manipulate the molecule must be moved at the same speed during unfolding and refolding [21,22].

In processes that are microscopically reversible, Crook’s FT predicts a symmetry relation in the work fluctuations for forward and reverse changes as the system is driven away from thermal equilibrium by the action of an external field or perturbation. A consequence of Crook’s FT is Jarzynski’s equality exp(−ΔG/k_BT) = 〈exp(−W/k_BT). However, for processes that occur far from equilibrium the applicability of Jarzynski’s equality is hampered by large statistical uncertainty arising from the sensitivity of the exponential average to rare events [21].

Sagawa and Ueda [3] generalized the fluctuation theorem and the second law of thermodynamics by taking into account the mutual information between the subsystems (entropy becomes nonadditive), where the entropy production and the mutual information are treated on an equal footing. Their results are valid in Langevin systems and also far from equilibrium situations of classical stochastic dynamics.

In the absence of the initial of final correlations, entropy production satisfies the integral of FT (or Jarzynski’s equality): 〈(−σ)〉 =1 where 〈..〉 is the ensemble average over all microscopic trajectories.

Consider a typical information processing in a composite and stochastic system X, which is in contact with multiple heat baths (k = 1, 2,…) described at inverse temperature β_k (1/k_B) and interacting with system Y, which does not evolve in time during the interactions. The time evolution of X depending on the state of Y from t = 0 to t = τ can be described by classical stochastic dynamics along trajectory a. The system has the x_i and x_f initial and final phase-space points as well as the initial and final probability distributions of p_ia [x_i] and p_fa [x_f]. The conditional probability of path a under the initial condition x_i which depends on y such that the conditional probability becomes p_a [X_a|x,y]. The systems X may be correlated with Y and the joint probability distributions are p_ia [x_i,y] and p_fa [x_f,y], where p_ia [x_i] = ∫ p_ia [x_i,y]dy, p_fa [x_f] = ∫ p_fa [x_f,y]dy, and p_a [y] = ∫ p_ia [x_i,y]dx=∫ p_fa [x_f,y]dx as the y is statistically independent. Then the ensemble averages 〈I_ixiy〉 and 〈I_fx,fy〉 represent the mutual information [3] where the initial and final correlations between X and Y are $I_{i{x}_{i}y}=\text{ln}\frac{p_{ia}[x_i,y]}{p_{ia}[x_i]p_a[y]}$ and $I_{f{x}_{f}y}=\text{ln}\frac{p_{fa}[xf,y]}{p_{fa}[x_f]p_a[y]}$.

In the presence of information I processing with initial and final correlations, the integral FT with energy dissipation and energy cost of information exchange becomes [3]:

$$\langle \text{exp}(-\sigma +\mathrm{\Delta }I)=1\rangle$$

(34)

where ΔI = I_fxfy − I_ixiy is the change in the mutual information, and p_ia [x_i,y] ≠ 0 for any (x,y). Convexity of exp 〈x〉 ≤ 〈exp(x)〉 leads to 〈σ〉 ≥ 〈ΔI〉 [3]. With the correlation remaining after a feedback control (I_rem) by system Y on system X, Equation (34) becomes:

$$\langle \text{exp}(-\sigma -(I-I_{\text{rem}})\rangle =1\hspace{0.17em}\text{so}\hspace{0.17em}\langle \sigma \rangle \ge -\langle I-I_{fb}\rangle$$

(35)

where 〈I − I_rem〉 may be an upper bound of the correlation that can be used.

The detailed FT in the presence of information processing becomes [5]:

$$\frac{p_b[X_b,y]}{p_f[X_f,y]}=\text{exp}(-\sigma +\mathrm{\Delta }I)$$

(36)

with the constraint p [x,y] ≠ 0 (x and y are the initial phase-space points), p_b and p_f are the joint probability distributions of the backward and forward processes, respectively, and (−σ + ΔI) shows the total entropy production of the composite system XY and the baths. Here, system x evolves from x_i to x_f along a path in such a manner that depends on the information about y, which does not evolve in time. To sustain a given fluctuation, a system traverses a precise optimal path in phase space [22,34].

FT allows a general orthogonality property of maximum information entropy MIE to be extended to entropy production EP. Maximum entropy production MEP and the FT are generic properties of MIE probability distributions. Physically, MEP applies to those macroscopic fluxes that are free to vary under the imposed constraints, and corresponds to the selection of the most probable macroscopic flux (flow) configuration [28–31]. The constrained maximization of Shannon information entropy H is an algorithm for constructing probability distributions from partial information. MIE is a universal method for constructing the microscopic probability distributions of equilibrium and nonequilibrium statistical mechanics. The distribution of the microscopic phase space trajectories over a time τ satisfies p_a∝exp(τσ_τ/2). FTs are highly relevant for interpretation of measurements performed on single molecules and can be applied to channel facilitated transport of solutes through a membrane separating two reservoirs. The transport is characterized by the probability p_n(t) that n solute particles have been transported from one reservoir to the other in time t. The fluctuation theorem establishes a relation between p_n(t) and p_-n(t): The ratio p_n(t)/p_-n(t) is independent of time and equal to exp(nA/k_BT), where A/k_BT is the affinity [13].

Equation (36) can be further expanded to include the influence of thermodynamic coupling through the information content of coupling (cross) coefficients (L_ik, i ≠ k) given in Equation (32):

$$\frac{p_b[X_b,y]}{p_f[X_f,y]}=\text{exp}(-\sigma +\mathrm{\Delta }I+I_{\text{TC}})$$

(37)

where I_TC represent the energy cost of information introduced because of thermodynamic coupling. In Equation (37), (−σ+ΔI + I_TC) shows the total entropy production of the composite coupled system XY and the baths. This may empower the FT to play a comprehensive role in estimating the fluctuations in living systems and eventually incorporating the information into physical sciences.

5. Conclusions

Various aspects of the thermodynamic relationships between information theory and living systems are discussed with emphasis on some new efforts toward defining and estimating the information. These efforts may lead to a deep understanding of the role played by the information particularly in living systems. Information theory can be used in investigating protein-protein interactions and the association of enzymes and proteins with their binding sites. If the subsystems are statistically dependent that is correlated by the mutual information, then the entropy becomes nonadditive; however, entropy production of the subsystems are additive. By treating mutual information and entropy production on equal footing a new integrated fluctuation theorem has already been suggested by Sagawa and Ueda [3]. Thermodynamic coupling, similar to the influence of information, can also lead to order. We may treat the information emerged from the thermodynamic coupling and entropy production on equal footing and use it in the generalized fluctuation theorem as Equation (37) suggests. Therefore, the generalized fluctuation theorem may play a comprehensive role in estimating the probability of the fluctuations in systems with information processing.