Information Field Theory and Artificial Intelligence

Abstract

Information field theory (IFT), the information theory for fields, is a mathematical framework for signal reconstruction and non-parametric inverse problems. Artificial intelligence (AI) and machine learning (ML) aim at generating intelligent systems, including such for perception, cognition, and learning. This overlaps with IFT, which is designed to address perception, reasoning, and inference tasks. Here, the relation between concepts and tools in IFT and those in AI and ML research are discussed. In the context of IFT, fields denote physical quantities that change continuously as a function of space (and time) and information theory refers to Bayesian probabilistic logic equipped with the associated entropic information measures. Reconstructing a signal with IFT is a computational problem similar to training a generative neural network (GNN) in ML. In this paper, the process of inference in IFT is reformulated in terms of GNN training. In contrast to classical neural networks, IFT based GNNs can operate without pre-training thanks to incorporating expert knowledge into their architecture. Furthermore, the cross-fertilization of variational inference methods used in IFT and ML are discussed. These discussions suggest that IFT is well suited to address many problems in AI and ML research and application.

1. Motivation

Determining the concrete configuration of a field from measurement data is an ill-posed inverse problem, as physical fields have an infinite number of degrees of freedom (DoF), whereas data sets are always finite in size. Thus, the data provide a finite number of constraints for only a subset of the infinitely many DoF of a field. In order to infer a field, the remaining of its DoF need, therefore, to be constrained via prior information. Fortunately, physics provides such prior information on fields. This information might either be precise, like $\nabla ·B=0$ in electrodynamics, or more phenomenological, in the sense that a field shaped by a certain process can often be characterized by its n-point correlation functions. Having knowledge on such correlations can be sufficient to regularize the otherwise ill-posed field inference problem from finite and noisy data such that meaningful statements about the field can be made.

As a formalism for this, information field theory (IFT) was introduced [1,2] following and extending earlier lines of work [3,4]. IFT is information theory for fields. It investigates the space of possible field configurations and constructs probability densities over those spaces in order to permit Bayesian field inference from data. It has been applied successfully to a number of problems in astrophysics [5,6,7,8,9,10,11,12,13,14,15,16,17,18], particle physics [19,20,21], and elsewhere [22,23,24,25,26]. Here, the relation of IFT with methods and concepts used in artificial intelligence (AI) and machine learning (ML) research are outlined, in particular with generative neural networks (GNNs) and in the usage of variational inference. The presented line of arguments summarizes a number of recent works [27,28,29,30,31,32].

The motivation for this work is twofold. On the one hand, understanding conceptual relations between IFT, ML, and AI techniques allows us to transfer computational methods between these domains and to develop synergistic approaches. This article will discuss such. On the other hand, the current success of deep learning techniques for neural networks has let them appear as a synonym for AI in the public perception. This has consequences for decisions about which kind of technologies get scientific funding. The point this paper is trying to make is that if deep learning qualifies as AI in this respect, then this should also apply to a number of other techniques, including those based on IFT.

The paper is organized as follows. IFT is briefly introduced in Section 2 in its most modern incarnation in terms of standardized, generative models. These are shown to be structurally similar to GNNs in Section 3. The structural similarity of IFT inference and GNN training problems allows for a common set of variational inference methods, as discussed in Section 4. Section 5 concludes on the relation of IFT methods and those used in AI and ML and gives an outlook on future synergies.

2. Information Field Theory

2.1. Basics

IFT allows us to deduce fields from data in a probabilistic way. In order to be able to apply probability theory onto the space of field configurations, a measure in this space is needed. Although no canonical mathematical measure on function spaces exists, for IFT applications, the usage of Gaussian process measures [33], which are mathematically well defined [34,35], is usually fully sufficient. Gaussian processes can also be argued to be a natural starting point for reasoning on fields with known finite first and second order moments, as we will discuss now.

To be specific, let $\phi :\Omega \to \mathbb{R}$ be a scalar field over some domain $\Omega \subset {\mathbb{R}}^u$ and our prior knowledge on $\phi$ be the first and second moments of the field, e.g.,

$$\begin{array}{ccc}\hfill {〈{\phi }^x〉}_{\left(\phi \right)}& =& {\overline{\phi }}^x\mathrm{and}\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill {〈{(\phi -\overline{\phi })}^x{(\phi -\overline{\phi })}^y〉}_{\left(\phi \right)}& =& {\Phi }^{xy}\mathrm{for}\mathrm{all}x,y\in \Omega ,\hfill \end{array}$$

(2)

with ${\phi }^x:=\phi \left(x\right)$ denoting a field value and ${〈f\left(\phi \right)〉}_{\left(\phi \right)}:=\int \mathcal{D}\phi \mathcal{P}\left(\phi \right)f\left(\phi \right)$ a prior expectation value for some function f of the field. If only the first and second field moments are given as prior information, it follows from the maximum entropy principle that the least informative probability distribution function encoding this information is a Gaussian with these moments. Thus, using this Gaussian

$$\begin{array}{ccc}\hfill \mathcal{P}\left(\phi \right|I)& \equiv & \mathcal{G}(\phi -\overline{\phi },\Phi )\hfill \\ & :=& \frac{1}{\sqrt{|2\pi \Phi |}}exp\left(-\frac{1}{2}{(\phi -\overline{\phi })}^{†}{\Phi }^{-1}(\phi -\overline{\phi })\right)\hfill \end{array}$$

(3)

as a prior with background information $I=({〈\phi 〉}_{\left(\phi \right)}=\overline{\phi },{\langle (\phi -\overline{\phi }){(\phi -\overline{\phi })}^{†}\rangle }_{\left(\phi \right)}=\Phi )$ is a conservative choice, as it makes the least additional assumptions about the field except for the moments specified in I.

In many applications, however, the field of interest, the signal s, is not a Gaussian field, but may be related to such via a non-linear transformation. For example, in astronomical applications of IFT, the sky brightness field s is the quantity of interest, which is strictly positive, and therefore cannot be a Gaussian field. However, the logarithm of a brightness can be positive and negative and may therefore be modeled as a Gaussian process. In such a case, one could assign, e.g., $s^x\left(\phi \right)=s_{0}exp\left({\phi }^x\right)$ as a model for a diffuse (spatially correlated) sky emission component, with $s_0$ a reference brightness, chosen such that for example ${〈{\phi }^x〉}_{\left(\phi \right)}=0$ holds.

Having established a field prior, Bayesian reasoning on the field $\phi$, and therefore on the signal of interest $s=s\left(\phi \right)$, based on some data d and its likelihood $\mathcal{P}\left(d\right|\phi ,I)$ is possible. The field posterior

$$\mathcal{P}\left(\phi \right|d,I)=\frac{\mathcal{P}\left(d\right|\phi ,I\left)\mathcal{P}\right(\phi \left|I\right)}{\mathcal{P}\left(d\right|I)}$$

(4)

is defined as well as the prior and permits us to answer questions about the field, like its most probable configuration ${\phi }_{\mathrm{MAP}}={\mathrm{argmax}}_{\phi }\mathcal{P}\left(\phi \right|d,I)$ (MAP = maximum a posteriori), its posterior mean $m={〈\phi 〉}_{\left(\phi \right|d,I)}$, or its posterior uncertainty dispersion $D={\langle (\phi -m){(\phi -m)}^{†}\rangle }_{\left(\phi \right|d,I)}$. IFT exploits the formalism of quantum and statistical field theory to calculate such posterior expectation values [1,28,36,37,38]. These formal calculations, however, should not be the focus here. Instead, it should be the formulation of IFT inference problems in terms of generative models, as these can be interpreted as GNNs.

For this purpose, the likelihood is expressed in terms of a measurement equation

$$\begin{array}{ccc}\hfill d& =& R\left(\phi \right)+n,\mathrm{with}\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill R\left(\phi \right)& :=& {〈d〉}_{\left(d\right|\phi )},\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill n& :=& d-R\left(\phi \right),\mathrm{and}\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill \mathcal{P}\left(d\right|\phi ,I)& \equiv & \mathcal{P}(n=d-R(\phi \left)\right|\phi ),\hfill \end{array}$$

(8)

which is always possible if the data can be embedded into a vector space and the data expectation value ${〈d〉}_{\left(d\right|\phi )}$ exists. Here and in the following, we omit the background information I in probabilities. This rewriting of the likelihood in terms of a mean instrument response $d^{\prime }=R\left(\phi \right)$ to the field $\phi$ and a noise process $\mathcal{P}\left(n\right|\phi )$, which summarizes the fluctuations around that mean $d^{\prime }$, allows us to regard the data as the result of a noisy generative process that maps field values $\phi$ and associated noise realizations n onto data d according to Equation (5).

In case the instrument response and noise processes are provided for the signal s instead of the Gaussian field $\phi$ as $R^{\prime }\left(s\right):={〈d〉}_{\left(d\right|s)}$ and $\mathcal{P}\left(n\right|s)$, their respective pull backs $R\left(\phi \right):={〈d〉}_{\left(d\right|s\left(\phi \right))}=R^{\prime }\left(s\left(\phi \right)\right)$ and $\mathcal{P}\left(n\right|\phi ):=\mathcal{P}(n\left|s\right(\phi \left)\right)$ provide the necessary response and noise statistics w.r.t. the field $\phi$.

All this provides a generative model for the signal s and data d via $\phi \hookleftarrow \mathcal{G}(\phi ,\Phi )$, $s=s\left(\phi \right)$, $n\hookleftarrow \mathcal{P}\left(n\right|s)$, and $d=R^{\prime }\left(s\right)+n$, which should now be standardized. The standardization introduces a generic latent space that permits better comparison to GNNs used in AI and ML and simplifies the usage of variational inference methods discussed later on.

2.2. Prior Standardization

Standardization of a random variable $\phi$ refers to finding a mapping from a standard normal distributed random variable $\xi \hookleftarrow \mathcal{G}(\xi ,𝟙)$ to $\phi$ that reproduces the statistics of $\mathcal{P}\left(\phi \right)$. For a Gaussian field $\phi$, this is just a mapping of the form

$$\phi \left(\xi \right):=\overline{\phi }+{\Phi }^{\frac{1}{2}}\xi ,$$

(9)

where ${\Phi }^{\frac{1}{2}}$ refers to a square root of $\Phi$, which always exists for a covariance matrix that is positive definite. For the large class of band diagonal and therefore translational invariant covariance matrices $\Phi$, which are very relevant for applications as we argue below, the square root of $\Phi$ can be explicitly constructed.

2.3. Power Spectra

In many signal inference problems, no spatial location is singled out a priori, before the measurement. This means that the field covariance between two locations only depends on the distance between these positions, but not on their absolute positions. Thus, ${\Phi }^{xy}=C_{\phi }(x-y)$. As a consequence of the Wiener–Khinchin theorem, such a translational invariant field covariance becomes diagonal in harmonic space,

$${\tilde{\Phi }}^{kq}={\mathcal{F}}_x^k{\Phi }^{xy}{\left({\mathcal{F}}^{†}\right)}_y^q={\left(2\pi \right)}^u\delta (k-q)P_{\phi }\left(k\right)={\widehat{P_{\phi }}}^{kq}.$$

(10)

Here and in the following, $\mathcal{F}$ denotes a harmonic transform (a u-dimensional Fourier transform ${\mathcal{F}}_x^k=exp(ik·x)$ in case of an Euclidean space, as we assume in the following), † the adjoint (complex conjugate and transposed of a matrix or vector), $P_{\phi }\left(k\right):={\mathcal{F}}_{x^{\prime }}^k{C}_{\phi }^{x^{\prime }}$ is the so called power spectrum of $\phi$, the Einstein convention for repeated indices is used, as in ${\tilde{\phi }}^k:={\mathcal{F}}_x^k{\phi }^x\equiv \int d{x}^{u}exp(ik·x)\phi \left(x\right)$, and $\widehat{\varphi }=\mathrm{diag}\left(\varphi \right)$ denotes a diagonal operator in the space of the field $\varphi$ with the values of $\varphi$ on the diagonal.

Thanks to this diagonal representation of the field covariance in harmonic space, an explicit standardization of the field is given via

$$\begin{array}{ccc}\hfill \xi & \hookleftarrow & \mathcal{G}(\xi ,𝟙),\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill \phi & =& \overline{\phi }+{\mathcal{F}}^{-1}{A}_{\phi }\xi ,\mathrm{and}\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill A_{\phi }& =& \widehat{P_{\phi }^{1/2}},\hfill \end{array}$$

(13)

where the latter is an amplitude operator that is diagonal in harmonic space and that imprints the right amplitudes onto the Fourier modes of $\phi$. This can be seen via a direct calculation,

$$\begin{array}{ccc}\hfill {\langle (\phi \left(\xi \right)-\overline{\phi }){(\phi \left(\xi \right)-\overline{\phi })}^{†}\rangle }_{\left(\xi \right)}& =& {\mathcal{F}}^{-1}{A}_{\phi }{〈\xi \xi 〉}_{\left(\xi \right)}{A}_{\phi }^{†}{\mathcal{F}}^{-1†}\hfill \\ \hfill & =& {\mathcal{F}}^{-1}{A}_{\phi }𝟙A_{\phi }^{†}{\mathcal{F}}^{-1†}\hfill \\ \hfill & =& {\mathcal{F}}^{-1}\tilde{\Phi }{\mathcal{F}}^{-1†}=\Phi \hfill \\ \hfill & =& {\langle (\phi -\overline{\phi }){(\phi -\overline{\phi })}^{†}\rangle }_{\left(\phi \right)}.\hfill \end{array}$$

(14)

In case no direction of the space is singled out a priori, the two-point correlation function and the power spectrum of $\phi$ become isotropic, ${\Phi }^{xy}=C_{\phi }\left(\right|x-y\left|\right)$ and ${\tilde{\Phi }}^{kq}={\left(2\pi \right)}^u\delta (k-q)P_{\phi }\left(\right|k\left|\right)$, respectively. In this case, only a one-dimensional power spectrum needs to be known. Such power spectra are often smooth functions on a double logarithmic scale in Fourier space, since any sharp feature in them would correspond to a (quasi-) periodic pattern in position space, which would be very unnatural for most signals. Thus, introducing the logarithmic Fourier space scale variable $\kappa \left(k\right):=lnk/k_0$ w.r.t. some reference scale $k_0$, we expect

$$\psi \left(\kappa \right):=ln\left(P_{\phi }\left(k_0{e}^{\kappa }\right)/P_0\right)$$

(15)

to be a field itself, in the sense that it is sufficiently smooth. Here, $P_0$ is a pivot scale for the power spectrum.

2.4. Amplitude Model

Often, the power spectrum as parameterized through $\psi$ is not known a priori for a field $\phi$, but statistical homogeneity, isotropy, and the absence of long range quasi-periodic signal variations make a Gaussian field prior for $\psi$ plausible, $\mathcal{P}\left(\psi \right)=\mathcal{G}(\psi -\overline{\psi },\Psi )$. This log-log-power spectrum may exhibit fluctuations $\chi :=\psi -\overline{\psi }$ around a non-zero mean $\overline{\psi }\left(\kappa \right)$. The latter might, e.g., encode a preference for falling spectra and therefore for a spatially smooth field $\phi$. In this case, just another layer for $\chi$ of a standardized generative model has to be added,

$$\begin{array}{ccc}\hfill \eta & \hookleftarrow & \mathcal{G}(\eta ,𝟙)\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill \chi \left(\eta \right)& :=& A_{\psi }\eta ,\mathrm{with}\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill A_{\psi }{A}_{\psi }^{†}& :=& \Psi ,\mathrm{and}\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill \psi \left(\eta \right)& :=& \overline{\psi }+\chi \left(\eta \right).\hfill \end{array}$$

(19)

Again, a prior for a field, here the only one dimensional $\chi \left(\kappa \right)$, is needed. A detailed description of how this amplitude model can be provided efficiently is given by [15]. This reference also provides a generative model for the case that the signal domain $\Omega$ is a product of sub-spaces, like position space and an energy spectrum coordinate, each requiring a different correlation structure, and the total correlation being a direct product of those. Assuming a direct product for the correlation structures might be possible for many field inference problems [15,39].

2.5. Dynamical Systems

Let us take a brief detour to fields shaped by dynamical systems. Dynamical systems, typically exhibit correlation structures that are not direct products of the spatial and temporal sub-spaces, as was proposed above. Here, the full spatial and temporal Fourier power spectrum $P_{\phi }(k,\omega )$, with $\omega$ being the temporal frequency, encodes the full dynamics of a linear, homogeneous, and autonomous system. For example, a stochastic wave field $\phi (x,t)$ may follow the dynamical equation

$$\left(\frac{{\partial }^2}{\partial t^2}+\eta \frac{\partial }{\partial t}-c^2\frac{{\partial }^2}{\partial x^2}\right)\phi (x,t)=\xi (x,t),$$

(20)

where c is the wave velocity and $\eta$ a damping constant. The field dynamics are determined by a response operator (or Green’s function) G that is a convolution of the exciting noise field $\xi$ with a kernel g,

$$\phi =G\xi =g\ast \xi ,$$

(21)

where ∗ denotes convolution. In Fourier-space, this kernel can be applied by a direct point wise multiplication, ${\left(\mathcal{F}\phi \right)}^{(k,\omega )}={\left(\mathcal{F}g\right)}^{(k,\omega )}{\left(\mathcal{F}\xi \right)}^{(k,\omega )}$ and is given by

$$\begin{array}{ccc}\hfill {\left(\mathcal{F}g\right)}^{(k,\omega )}& =& {(\omega {}^2-i\eta \omega -c^2{k}^2)}^{-1}=:P_G(k,\omega ).\hfill \end{array}$$

(22)

If the excitation of field fluctuations is caused by a white, stochastic noise field $\xi \hookleftarrow \mathcal{G}(\xi ,𝟙)$, the resulting field has a power spectrum of

$$P_{\phi }(k,\omega )={\left|P_G(k,\omega )\right|}^2{P}_{\xi }(k,\omega )=\frac{1}{{(\omega {}^2-c^2{k}^2)}^2+{\eta }^2{\omega }^2}.$$

(23)

In this case, the spectrum is an analytical function in $\omega$ and k. This results from Equation (20) being a linear, homogeneous, and autonomous partial differential equation.

Linear integro-differential equations, however, can still be solved by convolutions, in which case the kernel might not have an analytically closed form any more, if the equation is still homogeneous and autonomous. For example, in neural field theory [40,41,42,43], a macroscopic description of the brain cortex dynamics, the neural activity $\phi (x,t)$ might be described by

$$\frac{\partial }{\partial t}\phi =-\phi +w\ast (f\circ \phi )+\xi .$$

(24)

Here, w is a spatial–temporal convolution kernel (that usually contains a delta function in time), $f:\mathbb{R}\to \mathbb{R}$ an activation function that is applied point wise to the field, $(f\circ \phi )(x,t)=f\left(\phi \right(x,t\left)\right)$, and we added an input term $\xi$. In case f is linear, the system responds linearly to inputs. Then, the input response is a convolution with a kernel g that has in general a non-analytical spectrum,

$${\left(\mathcal{F}g\right)}^{(k,\omega )}={\left(1+i\omega -{\left(\mathcal{F}w\right)}^{(k,\omega )}{f}^{\prime }\right)}^{-1},$$

(25)

where $f^{\prime }$ is the slope of f and $\mathcal{F}w$ the Fourier transformed kernel of the dynamics.

An inference of such non-analytical and highly structured response spectra from data is possible with IFT and can be used to learn the system dynamics from noisy system measurements [26,44]. It just requires a more complex spectral prior than discussed here. Let us now return to our main line of argumentation.

2.6. Generative Model

To summarize, the field inference problems of IFT can often be stated in terms of a standardized, generative model for the signal and the data. For the illustrative case outlined above, where the probabilistic model is given by

$$\begin{array}{ccc}\hfill \mathcal{P}(d,\phi ,\psi )& =& \mathcal{P}\left(d\right|\phi \left)\mathcal{P}\right(\phi \left|\psi \right)\mathcal{P}\left(\psi \right),\hfill \end{array}$$

(26)

$$\begin{array}{ccc}\hfill \mathcal{P}\left(\psi \right)& =& \mathcal{G}(\psi -\overline{\psi },\Psi ),\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill \mathcal{P}\left(\phi \right|\psi )& =& \mathcal{G}(\phi ,\Phi (\psi \left)\right),\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill \Phi \left(\psi \right)& =& {\mathcal{F}}^{-1}\widehat{P_{\phi }}{\mathcal{F}}^{-1†},\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill P_{\phi }\left(k\right)& =& P_{0}exp\left(\psi \right(ln\left(\right|k|/k_0\left)\right)),\mathrm{and}\hfill \end{array}$$

(30)

$$\begin{array}{ccc}\hfill \mathcal{P}\left(d\right|\phi )& =& \mathcal{P}\left(d\right|s\left(\phi \right)),\hfill \end{array}$$

(31)

the corresponding standardized generative model is

$$\begin{array}{ccc}\hfill \zeta & :=& (\xi ,\eta )\hookleftarrow \mathcal{G}(\zeta ,𝟙),\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill \psi \left(\eta \right)& :=& \overline{\psi }+A_{\psi }\eta ,\hfill \end{array}$$

(33)

$$\begin{array}{ccc}\hfill P_{\phi }\left(k\right)& :=& P_{0}exp\left(\psi \right(ln\left(\right|k|/k_0\left)\right),\mathrm{is}\hfill \end{array}$$

(34)

$$\begin{array}{ccc}\hfill \phi (\xi ,\psi )& :=& {\mathcal{F}}^{-1}\widehat{P_{\phi }^{1/2}}\xi ,\hfill \end{array}$$

(35)

$$\begin{array}{ccc}\hfill s\left(\phi \right)& :=& s_{0}exp\left(\phi \right),\hfill \end{array}$$

(36)

$$\begin{array}{ccc}\hfill n& \hookleftarrow & \mathcal{P}\left(n\right|s),\mathrm{and}\hfill \end{array}$$

(37)

$$\begin{array}{ccc}\hfill d& =& R^{\prime }\left(s\right)+n.\hfill \end{array}$$

(38)

This generative model is illustrated in Figure 1. Variants of it are used in a number of real world data applications [5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21]. Its performance in generative and reconstruction mode is illustrated for synthetic data in Figure 2 and Figure 3.

For the noiseless data $d^{\prime }=R^{\prime }\left(s\right)$ the generative model reads

$$\begin{array}{ccc}\hfill d^{\prime }\left(\zeta \right)& :=& R^{\prime }\left(s\left(\phi (\xi ,\psi \left(\eta \right))\right)\right)\hfill \\ & =& (R^{\prime }\circ s\circ \phi \circ f)\left(\zeta \right),\mathrm{with}\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill f\left(\zeta \right)& :=& (\xi ,\psi (\eta \left)\right).\hfill \end{array}$$

(40)

This way, the full model complexity as given by Equations (26)–(31) is transferred into an effective response function $d^{\prime }=R^{\prime }\circ s\circ \phi \circ f$. For this latent variable vector, the prior is simply $\mathcal{P}\left(\zeta \right)=\mathcal{G}(\zeta ,𝟙)$, whereas the likelihood $\mathcal{P}\left(d\right|\zeta )=\mathcal{P}(n=d-d^{\prime }\left(\zeta \right)|\zeta )$ has absorbed the full model complexity. This so called reparametrization trick [45] was introduced to IFT by [29] to simplify numerical variational inference.

At this point, it is essential to realize that this generative model consists of a latent space white noise process $\mathcal{P}\left(\zeta \right)=\mathcal{G}(\zeta ,𝟙)$ that generates an input vector $\zeta$ and a sequence of non-local linear and local non-linear operations that is applied to it. The Fourier transform ${\mathcal{F}}^{-1}$ and $A_{\psi }$ are examples of non-local linear operations within the model. Among the non-linear operations are the exponential functions and the application of the $\psi$-dependent amplitude operator $A_{\phi }\left(\psi \right)$ to the latent space excitations $\xi$, as there the two components of $\zeta =(\xi ,\eta )$ are multiplied together. Furthermore, the instrument response $R^{\prime }\left(s\right)$ might also be decomposed into sequences of non-local linear and local non-linear operations, as physical processes in measurement devices can often be cast into the propagation of a quantity (an operation that is linear in the quantity) and the local interactions of the quantity (an operation non-linear in it), respectively.

3. Artificial Intelligence

3.1. Neural Networks

AI and ML are vast fields. AI aims at building artificial cognitive systems that perceive their environment, reason about its state and the systems’ best actions, and learn to improve their performance. ML can be regarded as a sub-field of AI, embracing many different methods like self-organized maps, Gaussian mixture models, deep neural networks, and many others. Here, the focus should be on specific neural networks, GNNs, as those have a close relation to the generative IFT models introduced before.

GNNs transform a latent space variable $\xi \hookleftarrow \mathcal{G}(\xi ,𝟙)$ into a signal or data realization, $s=s\left(\xi \right)$ or $d^{\prime }=d^{\prime }\left(\xi \right)$. A neural network is a function $g\left(\xi \right)$ that can be decomposed in terms of n layer processing functions $g_i$ with

$$g=g_n\circ g_{n-1}\circ \dots g_1.$$

(41)

Any of the layer processing functions $g_i:{\xi }_i\mapsto {\xi }_{i+1}$ with ${\xi }_1\equiv \xi$ consists typically of a non-local, affine linear transformation $l_i\left({\xi }_i\right):=L_i{\xi }_i+b_i$ of the input vector ${\xi }_i$ of layer i followed by a local, point wise application of non-linear, so-called activation functions ${\sigma }_i:\mathbb{R}\to \mathbb{R}$. Thus, the output vector ${\xi }_{i+1}$ of layer i is

$$\begin{array}{ccc}\hfill {\xi }_{i+1}& =& \left({\sigma }_i\circ l_i\right)\left({\xi }_i\right),\hfill \end{array}$$

(42)

where ${\sigma }_i$ acts component wise. The set $\eta ={(L_i,b_i)}_{i=1}^n$ of all coefficients of the $l_i$s (the matrix elements of the $L_i$ matrices, and the components of the $b_i$ vectors) determines the function the network represents. Putting the input values and network coefficients into a single vector $\zeta :=(\xi ,\eta )$ a GNN can be regarded as a function of both, latent variables $\xi$ and network parameters $\eta$, $d^{\prime }\left(\zeta \right)=g(\xi ;\eta )$.

3.2. Comparison with IFT Models

From this abstract perspective, a standardized, generative model $d^{\prime }\left(\zeta \right)$ in IFT is structurally a GNN, as both consist of sequences of local non-linear and non-local linear operations on their input vector $\zeta =(\xi ,\eta )$. The concrete architecture of an IFT model and a typical GNN might differ significantly, as GNNs often map a lower dimensional latent space into a higher dimensional data or feature space, whereas the dimension of the IFT model latent space can be very high, as it contains a subset of the virtually infinite many degrees of freedom of a field, see Figure 1.

Additionally, the way IFT-based models and GNNs are usually used differs a bit. Both can be used to generate synthetic samples of outputs by processing random latent space vectors $\xi \hookleftarrow \mathcal{G}(\xi ,𝟙)$. However, typically an IFT model $d^{\prime }\left(\zeta \right)$ is applied to infer all latent space variables in $\zeta$ from data d. From the latent variables, the signal of interest can always be recovered via $s\left(\zeta \right)$.

For this inference the so-called information Hamiltonian, potential, or energy

$$\begin{array}{ccc}\hfill \mathcal{H}(d,\zeta )& =& -ln\mathcal{P}(d,\zeta )=-ln\mathcal{P}\left(d\right|\zeta )-ln\mathcal{P}(\zeta )\hfill \\ \hfill & =& \mathcal{H}(n=d-d^{\prime }\left(\zeta \right)|\zeta )+\frac{1}{2}{\zeta }^{†}\zeta +\mathrm{const}\hfill \end{array}$$

(43)

is investigated with respect to $\zeta$, where $\mathcal{H}\left(a\right|b):=-ln\mathcal{P}(a\left|b\right)$. This quantity is introduced to IFT in analogy to statistical mechanics, it summarizes the full knowledge on the problem (as it is just a logarithmic coordinate transformation in the space of probabilities) and has the nice property, that it allows to speak about information as an additive quantity, as $\mathcal{H}(a,b)=\mathcal{H}\left(a\right|b)+\mathcal{H}(b)$.

Investigating the relevant information Hamiltonian for our IFT problem $\mathcal{H}(d,\zeta )$ can be done, for example, by minimizing it to obtain a MAP estimator for $\zeta$ or—as discussed in the next section—via variational inference (VI). In case of a constant, signal independent Gaussian white noise statistics, the information Hamiltonian becomes

$$\begin{array}{ccc}\hfill \mathcal{H}(d,\zeta )& =& \frac{|d-d^{\prime }{\left(\zeta \right)|}^2}{2{\sigma }_n^2}+\frac{1}{2}{\zeta }^{†}\zeta +\mathrm{const}.\hfill \end{array}$$

(44)

The training of an usual GNN is done with a training data set $\tilde{d}={\left(d_i\right)}_i$ to which a corresponding latent space vector set $\tilde{\zeta }={\left({\zeta }_i\right)}_i$ and common network parameters $\eta$ need to be found. For this a loss function of the form

$$\begin{array}{ccc}\hfill \tilde{\mathcal{H}}(\tilde{d},\tilde{\xi },\eta )& =& \sum _i\tilde{\mathcal{H}}\left(d_i\right|{\xi }_i,\eta )+\tilde{\mathcal{H}}(\tilde{\xi }|\eta )+\tilde{\mathcal{H}}\left(\eta \right),\hfill \end{array}$$

(45)

$$\begin{array}{ccc}\hfill \tilde{\mathcal{H}}\left(\tilde{\xi }\right|\eta )& =& \frac{1}{2}\sum _i{\xi }_i^{†}\xi +\mathrm{const},\mathrm{and}\hfill \end{array}$$

(46)

$$\begin{array}{ccc}\hfill \tilde{\mathcal{H}}\left(d_i\right|{\xi }_i,\eta )& =& \frac{1}{2}\frac{|d_i-d^{\prime }({\xi }_i,\eta ){|}^2}{2{\sigma }_n^2}+\mathrm{const}\hfill \end{array}$$

(47)

might be minimized. Here, a typical GNN data loss function $\tilde{\mathcal{H}}\left({\tilde{d}}_i\right|{\xi }_i,\eta )$ as used for the decoder part of an autoencoder (AE) [49] was assumed. In an generative adversarial network (GAN) [50], however, this data loss function is given in terms of the output of a discriminator network. The network parameter prior term $\tilde{\mathcal{H}}\left(\eta \right)$ might be chosen to be uninformative ($\tilde{\mathcal{H}}\left(\eta \right)=\mathrm{const}$) or informative (e.g., $\tilde{\mathcal{H}}\left(\eta \right)=\frac{1}{2}{\eta }^{†}\eta$ in case of a Gaussian prior on the parameters).

Anyhow, by comparison of Equations (45)–(47) with Equations (43) and (44), it should be apparent that the network loss functions can be structurally similar to the IFT information Hamiltonian. Both consist of a standardized quadratic prior-energy and a likelihood-energy and both can have a probabilistic interpretation in terms of being negative log-probabilities, e.g.,

$$\begin{array}{ccc}\hfill \mathcal{P}(d,\xi ,\eta )& =& e^{-\mathcal{H}(d,\xi ,\eta )}\mathrm{and}\hfill \end{array}$$

(48)

$$\begin{array}{ccc}\hfill \mathcal{P}(\tilde{d},\tilde{\xi },\eta )& =& e^{-\tilde{\mathcal{H}}(\tilde{d},\tilde{\xi },\eta )},\hfill \end{array}$$

(49)

respectively. For this reason, we do not distinguish between an information Hamiltonian $\mathcal{H}$ and a network loss function $\tilde{\mathcal{H}}$ by writing $\mathcal{H}$ for both in the following.

The IFT-GNN can operate with solely a single data vector d due to the domain knowledge coded into their architecture, whereas usual GNNs require sets of data vectors $\tilde{d}={\left(d_i\right)}_i$ to be trained. Recently, more IFT-like architectures for GNNs were proposed as well, which are also able to process data without training [51].

4. Variational Inference

4.1. Basic Idea

So far, it has been assumed here that MAP estimators are used to determine network parameters $\zeta$ for both, IFT-based models as well as traditional GNNs. MAP estimators are known to be prone to over-fitting the data, as they are not probing the adjacent phase-space volumes of their solutions. VI methods perform better in that respect, while still being affordable in terms of computational costs for the high dimensional settings of IFT-based field inference and traditional GNN training. They were used in most recent IFT applications [12,13,14,15,16,17,18,21,52] and are prominently present in the name of variational autoencoders (VAEs) [45] that are built on VI.

In VI, the posterior $\mathcal{P}\left(\zeta \right|d)$ is approximated by a simpler probability distribution $\mathcal{Q}\left(\zeta \right|d^{\prime })$, in many applications by a Gaussian

$$\mathcal{Q}\left(\zeta \right|\theta ,\Theta )=\mathcal{G}(\zeta -\theta ,\Theta ),$$

(50)

where $d^{\prime }=(\theta ,\Theta )$. The Gaussian is chosen to minimize the variational Kullback–Leibler (KL) divergence

$$\begin{array}{ccc}\hfill {\mathrm{KL}}_{\zeta }(d^{\prime },d)& :=& {\mathcal{D}}_{\mathrm{KL}}\left(\mathcal{Q}\right|\left|\mathcal{P}\right)\hfill \\ \hfill & =& \int \mathcal{D}\zeta \mathcal{Q}\left(\zeta \right|d^{\prime })ln\frac{\mathcal{Q}\left(\zeta \right|d^{\prime })}{\mathcal{P}\left(\zeta \right|d)}\hfill \end{array}$$

(51)

with respect to the parameters of $d^{\prime }$, $\theta$ and $\Theta$ in our case.

Ideally, all degrees of freedom (DoF) of $\theta$ and $\Theta$ are optimized. In practice, however, this is often not feasible due to the quadratic scaling of the number of DoF of $\Theta$ with that of $\theta$. Three approximate schemes for handling the high dimensional uncertainty covariance will be discussed in the following, leading to the ADVI, MGVI, and geoVI techniques introduced below, namely

• mean field theory, in which $\Theta$ is assumed to be diagonal, as used by ADVI
• the usage of the Fisher information to approximate $\Theta$ as a function of $\theta$ and thereby effectively removing the DoF of $\Theta$ from the optimization problem as used by MGVI
• a coordinate transformation of the latent space that approximately standardizes the posterior and therefore sets the covariance to the identity matrix in the new coordinates, as performed by geoVI.

Before these are discussed, a note that applies to all of them is in order. Optimizing of the VI KL, Equation (51), is slightly sub-optimal from an information theoretical point of view as this minimizes the amount of information introduced by going from $\mathcal{P}$ to $\mathcal{Q}$. The expectation propagation (EP) KL with reversed arguments ${\mathcal{D}}_{\mathrm{KL}}\left(\mathcal{P}\right|\left|\mathcal{Q}\right)$ would be better, as it minimizes the information loss from approximating $\mathcal{P}$ with $\mathcal{Q}$ [53]. VI is known to underestimate the uncertainties, whereas EP conservatively overestimates them. However, calculating the EP solution for $\theta$ and $\Theta$ would require integrating over the posterior. If this would be feasible, any posterior quantity of interest could be calculated as well and there would be no need to approximate $\mathcal{P}\left(\zeta \right|d)$ in the first place. Estimating and minimizing the VI KL ${\mathcal{D}}_{\mathrm{KL}}\left(\mathcal{Q}\right|\left|\mathcal{P}\right)$ is less demanding, as the integral over the simpler (Gaussian) distribution $\mathcal{Q}$ can very often be performed analytically, or by sample averaging using samples drawn from $\mathcal{Q}$.

4.2. ADVI and Mean Field Approximation

In all here discussed VI techniques, the posterior mean $\theta$ and the posterior uncertainty covariance $\Theta$ become parameters to be determined. The vector $\theta$ has the dimension $N_{\dim }$ of the latent space, whereas the posterior uncertainty covariance $\Theta$ has $N_{\dim }(N_{\dim }-1)/2=\mathcal{O}\left(N_{\dim }^2\right)$ independent DoF. For small problems, these might be solved for, however, for large problems with millions of DoF, these cannot even be stored in a computer memory. To circumvent this, the Automatic Differentiation Variational Inference (ADVI) algorithm [54] often invokes the so called mean field approximation (MFA). This assumes a diagonal covariance ${\Theta }_{\mathrm{MFA}}=\widehat{{\theta }^{\prime }}\equiv \mathrm{diag}\left({\theta }^{\prime }\right)$, with ${\theta }^{\prime }$ being a latent space vector. Cross-correlations between parameters can not be represented by this, which is problematic in particular in combination with the tendency of VI to underestimate uncertainties.

4.3. MGVI and Fisher Information Metric

In order to overcome this limitation of ADVI that limits its usage in IFT contexts with their large number of DoF, the Metric Gaussian Variational Inference (MGVI) [30] algorithm approximates the posterior uncertainty of $\zeta$ with the help of the Fisher information metric

$$\begin{array}{ccc}\hfill M\left(\zeta \right)& :=& {〈\frac{\partial \mathcal{H}\left(d\right|\zeta )}{\partial \zeta }{\frac{\partial \mathcal{H}\left(d\right|\zeta )}{\partial \zeta }}^{†}〉}_{\left(d\right|\zeta )}.\hfill \end{array}$$

(52)

The starting point for obtaining the uncertainty covariance $\Theta$ used in MGVI is the Hessian of the log-posterior

$$\begin{array}{ccc}\hfill \frac{{\partial }^2\mathcal{H}\left(\zeta \right|d)}{\partial \zeta \partial {\zeta }^{†}}& =& \frac{{\partial }^2\mathcal{H}(d,\zeta )}{\partial \zeta \partial {\zeta }^{†}}-\underset{=0}{\underbrace{\frac{{\partial }^2\mathcal{H}\left(d\right)}{\partial \zeta \partial {\zeta }^{†}}}}\hfill \\ & =& \frac{{\partial }^2\mathcal{H}(d,\zeta )}{\partial \zeta \partial {\zeta }^{†}}\hfill \end{array}$$

(53)

as a first guess for the approximate posterior precision matrix ${\Theta }^{-1}$. Using this evaluated at the minimum ${\zeta }_{\mathrm{MAP}}$ of the information Hamiltonian $\mathcal{H}(d,\zeta )$ would correspond to the Laplace approximation, in which the posterior is replaced by a Gaussian obtained from doing a saddle point approximation at its maximum.

However, neither is the MAP solution ideal, as discussed above, nor would this be a good approximation at many locations $\zeta$ that differ from ${\zeta }_{\mathrm{MAP}}$. This is because positive definiteness of the Hessian is not guaranteed there, but it is an essential property of any correlation and precision matrix. For this reason, ${\Theta }^{-1}$ cannot directly be approximated by this Hessian.

It turns out that the likelihood averaged Hessian is strictly positive definite, and is therefore a candidate for an approximate posterior precision matrix for any guessed posterior mean $\theta$. A short calculation shows that the likelihood averaged Hessian is indeed positive definite:

$$\begin{array}{ccc}\hfill {\Theta }^{-1}\left(\theta \right)& \approx & {〈\frac{{\partial }^2\mathcal{H}(d,\zeta )}{\partial \zeta \partial {\zeta }^{†}}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & =& {〈\frac{{\partial }^2\mathcal{H}\left(\zeta \right)}{\partial \zeta \partial {\zeta }^{†}}+\frac{{\partial }^2\mathcal{H}\left(d\right|\zeta )}{\partial \zeta \partial {\zeta }^{†}}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & =& {〈𝟙-\frac{{\partial }^{2}ln\mathcal{P}\left(d\right|\zeta )}{\partial \zeta \partial {\zeta }^{†}}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & =& 𝟙-{〈\frac{1}{\mathcal{P}\left(d\right|\zeta )}\frac{{\partial }^2\mathcal{P}\left(d\right|\zeta )}{\partial \zeta \partial {\zeta }^{†}}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & & +{〈\frac{1}{{\mathcal{P}}^2\left(d\right|\zeta )}\frac{\partial \mathcal{P}\left(d\right|\zeta )}{\partial \zeta }{\frac{\partial \mathcal{P}\left(d\right|\zeta )}{\partial \zeta }}^{†}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & =& 𝟙-\int \mathrm{d}d\frac{\cancel{\mathcal{P}\left(d\right|\zeta )}}{\cancel{\mathcal{P}\left(d\right|\zeta )}}\frac{{\partial }^2\mathcal{P}\left(d\right|\zeta )}{\partial \zeta \partial {\zeta }^{†}}\hfill \\ & & +{〈\frac{\partial \mathcal{H}\left(d\right|\zeta )}{\partial \zeta }{\frac{\partial \mathcal{H}\left(d\right|\zeta )}{\partial \zeta }}^{†}〉}_{\left(d\right|\zeta =\theta )}\hfill \\ & =& 𝟙-\underset{\underset{=0}{︸}}{\frac{{\partial }^2}{\partial \zeta \partial {\zeta }^{†}}\underset{\underset{=1}{︸}}{\int \mathrm{d}d\mathcal{P}\left(d\right|\zeta )}}+M\left(\theta \right)\hfill \\ & =& 𝟙+M\left(\theta \right)>0.\hfill \end{array}$$

(54)

The last step follows because the Fisher metric $M\left(\theta \right)$ is an average over outer products ($v{v}^{†}\ge 0$) of likelihood Hamiltonian gradient vectors $v=\partial \mathcal{H}\left(d\right|\zeta )/\partial \zeta$ and thereby positive semi-definite. Adding $𝟙>0$ to the Fisher metric turns the approximate precision matrix into a positive definite matrix ${\Theta }^{-1}\left(\theta \right)>0$, of which the inverse $\Theta \left(\theta \right)$ exists for all $\theta$, and which is positive definite as well.

4.4. Exact Uncertainty Covariance

Being positive definite is of course not the only property an approximation of the posterior uncertainty covariance has to fulfill. It also has to approximate well. Fortunately, this seems to be the case in many situations. The likelihood averaged Laplace approximation actually becomes the exact posterior uncertainty in case of linear Gaussian measurement problems as is shown in the following. If it is exact in such linear situations, it should be a valid approximation in the vicinity of any linear case.

For linear measurement problems, the measurement equation is of the form $d=R\zeta +n$, the noise statistics $\mathcal{P}\left(n\right|\zeta )=\mathcal{G}(n,N)$, and the standardized prior is $\mathcal{P}\left(\zeta \right)=\mathcal{G}(\zeta ,𝟙)$. The corresponding posterior is known to be a Gaussian

$$\mathcal{P}\left(\zeta \right|d)=\mathcal{G}(\zeta -m,D)$$

(55)

with mean m and covariance D given by the generalized Wiener filter solution $m=D{R}^{†}{N}^{-1}d$ and the Wiener covariance $D={(𝟙+R^{†}{N}^{-1}R)}^{-1}$, respectively (e.g., [1]). In this case, the Fisher information metric $M=R^{†}{N}^{-1}R$ is independent of $\zeta$. The approximate posterior uncertainty covariance as given by Equation (54) equals the exact posterior covariance, $\Theta ={\left(𝟙+M\right)}^{-1}={({𝟙}^{-1}+R^{†}{N}^{-1}R)}^{-1}=D$. Thus indeed, the adopted approximation becomes exact in this situation. This should show why this approximation can hold sensible results in sufficiently well behaved cases, in particular when a linearization of the inference problem around a reference solution (e.g., a MAP estimate) is already a good approximation.

Furthermore, for all signal space directions around this reference point that are unconstrained by the data, this covariance approximation returns the prior uncertainty, as it should. Additional discussion of this approximation can be found in Knollmüller and Enßlin [30], where also its performance with respect to ADVI is numerically investigated.

The important point about this approximate uncertainty covariance $\Theta$ is that it is a function of the latent space mean estimate $\theta ,$ i.e., $\Theta \left(\theta \right)$, and therefore does not need to be inferred as well. For many likelihoods, the Fisher metric is available analytically, alleviating the need to store $\Theta$ in a computer memory as an explicit matrix. It is only necessary that certain operations can be performed with $\Theta$, like applying it to a vector or drawing samples from a Gaussian with this covariance. Relying solely on those memory inexpensive operations, the MGVI algorithm is able to minimize the relevant VI KL, namely ${\mathrm{KL}}_{\zeta }((\theta ,\Theta \left(\theta \right)),d)={\mathcal{D}}_{\mathrm{KL}}(\mathcal{Q},\mathcal{P})$, with respect to the approximate posterior mean $\theta$. The result of MGVI are then the posterior mean $\theta$, the uncertainty covariance $\Theta \left(\theta \right)$, and posterior samples ${\left\{{\zeta }_i\right\}}_i$ drawn according to this mean and covariance. These samples can then be propagated into posterior signal samples $s_i=s\left({\zeta }_i\right)$, from which any desired posterior signal statistics can be calculated.

MGVI has enabled field inference for problems, which are too complex to be solved by MAP, in particular when multiple layers of hyperpriors were involved (e.g., [13,15]).

4.5. Geometric Variational Inference

ADVI’s and MGVI’s weak point, however, can be the Gaussian approximation of the posterior, which might be strongly non-Gaussian in certain applications. In order to overcome this, the geometrical variational inference (geoVI) algorithm [32] was introduced as an extension of MGVI. geoVI puts another coordinate transformation on top of the one used by MGVI, so that $\zeta =g_0\left(y\right)$—with $g_0$ to be performed before any of the other IFT-GNN operations $g_1,\dots g_n$ — approximately standardizes the posterior, $\mathcal{P}\left(y\right|d)\approx \mathcal{G}(y,𝟙)$. Astonishingly, this transformation can be constructed without the (prohibitive) usage of any explicit matrix or higher order tensor in the latent space, thus also allowing us to tackle very high dimensional inference problems, like MGVI. The transformation is basically a normalizing flow (network) [55], just with the difference to their usual usage in ML, that the geoVI flow does not need to be trained, but is derived from the problem statement in form of its information Hamiltonian in an automated fashion. Specifically, the coordinate transformation $g_0$ is defined to solve the constraining equation

$${\left.{\frac{\partial g_0}{\partial y}}^{†}\Theta \left(\zeta \right)\frac{\partial g_0}{\partial y}\right|}_{\zeta =g_0\left(y\right)}\approx 𝟙\forall y,$$

(56)

which fully specifies $g_0$ up to an integration constant $\theta$. This remaining constant is solved for by minimizing the VI KL with respect to $\theta$ to retrieve the optimal geoVI aproximation.

With geoVI, deeper hierarchical models, which more often exhibit non-Gaussian posteriors due to a larger number of degenerate parameters in them, can be approached via VI. The ability of geoVI to provide uncertainty information is illustrated in Figure 2 (bottom middle and right panels) and in Figure 3. Further details on geoVI and detailed comparisons of ADVI, MGVI, geoVI, and Hamiltonian Monte Carlo methods can be found in [32].

5. Conclusions and Outlook

This paper argues that IFT techniques can well be regarded as ML and AI methods by showing their interrelation with GNNs, normalizing flows, and VI techniques. This insight is not necessarily new, as this paper just summarizes a number of recent works [29,30,31,32] that suggested this before.

First, the generative models build and used in IFT are GNNs that can interpret data without initial training, thanks to the domain knowledge coded into their architecture [29]. Related architectures have very recently been proposed as image priors in the context of neural network architectures as well [51]. As IFT models and the newly proposed image priors do not obtain their intelligence from data driven learning, they are strictly not ML techniques, but might be characterized as (expert) knowledge-driven AI systems. From a technical point of view, however, such a distinction could be seen as splitting hairs.

Second, the VI algorithms used in IFT and AI to approximately infer quantities are a natural interface between these areas. Here, the related ADVI [54], MGVI [30], and geoVI [32] algorithms were briefly discussed, which can be used in classical ML and AI as well as in IFT applications.

And third, the common probabilistic formulation of IFT models and GNNs, as well as the common VI infrastructure of the two areas allows for combining pre-trained GNNs and other networks with IFT-style model components. In that respect, the possibility to perform Bayesian reasoning with trained neural networks as described in [31] might give an outlook on the potential to combine IFT with other ML and AI methods.

To summarize, IFT [1,2] addresses perception [5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21], reasoning [22,23,24,25,26,31], and adaptive inference [30,32] tasks. All these are central to the aims of AI and ML to build intelligent systems including such for perception, cognition, and learning.