Sampling Theorems for Stochastic Signals. Appraisal of Paul L. Butzer’s Work

Abstract

The sampling reconstruction theory is one of the great areas of the analysis in which Paul Leo Butzer earned longstanding and excellent theoretical results. Thus, we are forced either by earlier exhaustive presentations of his research activity and/or the highly voluminous material to restrict ourselves to a more narrow and precise sub-area in consideration; we discuss here, giving deeper insight, Paul Butzer’s sampling theoretical work with special attention concerning sampling stochastic signals.

1. Introduction

Let f be expressible in a domain D in the form

$$f(x)=\sum _{n\in \mathbb{Z}}f({\lambda }_n)S_n(x),x\in D,$$

(1)

where $({\lambda }_n)\subseteq D$ and $(S_n(x)\equiv S_n(x,{\lambda }_n))$ is some convenient function class. This type of presentation is the sampling series (of f), since f is restored by its values sampled in the discrete subset $({\lambda }_n)\subset D$ by (in general) an infinite linear combination. In the ’classical case’, when $D\equiv \mathbb{R},{\lambda }_n=n\pi /w,w>0$ and $S_n(x)={(wx-n\pi )}^{-1}sin(wx-n\pi )$ we arrive at the so-called WKS (Whittaker–Kotel’nikov–Shannon) sampling theorem, which reads

$$f(x)=\sum _{n\in \mathbb{Z}}f\left({\displaystyle \frac{\pi }{w}}n\right){\displaystyle \frac{sin(wx-n\pi )}{wx-n\pi }},x\in \mathbb{R},$$

(2)

when the input function f satisfies certain suitable assumptions. The convergence is absolute and uniform with respect to x, when it belongs to any compact from $\mathbb{R}$. In (2) is also usual the notation $\mathrm{sinc}(z):=z^{-1}sin(z),z\ne 0$, while $\mathrm{sinc}(0)=1$, originating back to the sinus cardinalis terminology introduced by Woodward in 1952.

Independently of the mathematicians’ works, the Formula (2) was obtained by the young Soviet electrical engineer Kotel’nikov [1], who pointed out its great importance for communication theory (16 years before) Shannon, who deals with the problems that occur in certain communication system [2,3]. For this reason, the above mentioned Formula (2) in Russian literature is called Kote’lnikov formula and in Western literature Shannon formula. That controversy in nomenclature “Kotel’nikov versus Shannon” was precisely treated by Kolmogorov in 1956 in his MIT talk [4].

At the end of the 1940s the prerequisites for extending the sampling theorem to stochastic weakly stationary processes were developed—Shannon’s paper, the spectral representation formula, Karhunen–Cramér theorem [5] (p. 156). Therefore, except Belyaev’s oversampling paper [6] all other mentioned results follow the described research direction used by the Frenchmen Ville [7,8,9] and Oswald [10], then Americans Lloyd [11] and Parzen [12] and/or the Russian (in that time Soviet) mathematician Belyaev. At this point we have to point out that it is conventionally said that Balakrishnan in 1957 was the inventor of stochastic sampling theorem. It is interesting to quote that both Parzen and Balakrishnan have submitted their manuscripts in 1956; Parzen’s technical report [12] (never published in a journal) was submitted at December 22, while Balakrishnan’s manuscript [13] has been submitted to the IRE Transactions of Information Theory a month earlier, precisely at November 23. Both articles discuss the mean square restoration of bandlimited weak sense stationary stochastic processes. This can confirm Balakrishnan’s priority.

Despite these facts Ville and Oswald gave introductory results to the sampling reconstruction procedure for bandlimited stochastic processes years earlier, see [14]. In his article [6] Belyaev emphasized that “... a rigorous proof of the Formula (2) for the sample functions of random processes with bounded spectra has not been given...” before his work for both the mean-square (and a fortiori almost sure) sampling reconstruction formulae [6] (p. 411). The above referenced authors didn’t cite each other, which we can clearly see by using into account the excellent Lloyd paper [11] obviously unknown by Belyaev. For further information concerning sampling theory development, see among others the cornerstone papers [B], the famous Jerri’s ‘1977–sampling–paper’ [15], the overview of the Soviet sampling research results which are definitely devoted to a greater extent to stochastic signals than the Eastern (mainly USSR) investigations by Khurgin and Yakovlev [16], Higgins ’5 short stories’ [17] and Unser’s [18]. Finally, for the less known almost sure sense convergence approach in stochastic signals reconstruction one can consult the book chapter [19] with the exhaustive references list therein.

2. Paul Butzer’s Stochastic Sampling Theoretical Research

In the following we expose Paul Butzer’s contribution and his excellent results to the topic which gives deep insight into the background and mathematical formulation and generalizations of the Formula (1) when the input signal is a stochastic process.

In his RWTH Aachen, Paul Butzer was a leading person in Analysis area, teaching, conducting research, supervising PhD students in Aachen (according to the Mathematics Genealogy database he has 38 PhD students in 35 years).

Moreover by the MathSciNet, among Approximation theory, Fourier analysis, Harmonic analysis, Functional analysis, Special functions (the famous Butzer-Flocke-Hauss Omega function), Integral transforms (specially Fourier and Mellin transforms), mentioning only a part of his interest areas, Paul Butzer authored and co–authored 47 titles in which “sampling” is mentioned. His co-authors’ list contains the following names: R. L. Stens (22), G. Schmeisser (7), W. Engels (5), J. R. Higgins (5), W. Splettstösser (5), C. Bardaro (4), P. J. S. G. Ferreira (4), G. Vinti (4), M. H. Annaby (3), A. Fischer (3), G. Hinsen (3), I. Mantellini (3), S. Ries (3), G. Schöttler (3), A. I. Zayed (2), M. M. Dodson (1), A. Gessinger (1), M. Hauss (1), S. Jansche (1), O. Lange (1) and U. Scheben (1). This tremendously wide opus cannot be presented in ‘finite/real time’, so we concentrate to few of his papers with results regarding stochastic signals restoration, which are:

• [A] Butzer, P.L. A survey of the Whittaker-Shannon sampling theorem and some of its extensions. J. Math. Res. Exposition 1983, 3, 185–212.
• [B] Butzer, P.L., Splettstösser, W. Stens, R.L. The sampling theorem and linear prediction in signal analysis. Jahresber. Deutsch. Math.-Verein. 1988, 90, 1–70.

Leading the research in the RWTH Aachen, Butzer has also switched to a classical question of probability theory and linear stochastic processes, precisely to the Central Limit Therem (CLT) in which he realized the following titles in cooperation with Ursula Gather:

• [C] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. I. General theorems on rates of convergence. Math. Methods Appl. Sci. 1979, 1, 241–264.
• [D] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. II. Models of the general random noise and pulse train processes. Math. Methods Appl. Sci. 1979, 1, 346–353.

Finally, the prediction of signals from the whole ‘past history’, that is, the known past samples, was treated by Butzer and his most frequent co-author, colleague, PhD student and friend Rudolf (Rolf) Leonard Stens, which is represented out of excellent memoir [B], by

• [E] Butzer, P.L., Stens R.L. Prediction of nonbandlimited signals from past samples in terms of splines of low degree. Math. Nachr. 1987, 132, 115–130.

In [C, D] not necessarily wide sense stationary processes (a connection to the so–called Piranašvili processes, introduced in the Piranašvili’s 1967 cornerstone sampling/interpolation paper [20] could act as a further interesting research goal) are treated from the Central Limit Theorem point of view, exploiting Honda’s concept of linear stochastic processes [21].

The article [E] discusses the signals prediction from past samples, in which the similar approach is exploited as in the sampling reconstruction procedures. It is worth mentioning that the simple implementation of the classical prediction procedure (generalized in [E]) to the weakly stationary stochastic processes was published in the conference talk [22].

3. The Results

My attention to the WKS sampling theorem has been drawn in the mid 1980s by Professor Mátyás Arató during my ten months long research grant spent in the Department of Probability and Statistics of Eötvös Loránd University, Budapest. (Professor Arató was a PhD student of A.N. Kolmogorov. Ergo, he called (2) precisely Kotel’nikov formula). At that time, I knew about this result reading Yaglom’s ‘small book’ [23] (p. 204) but become interested in the sampling theorems for stochastic processes after this suggestion, publishing my first sampling paper [24]. Collecting the research material for this topic, I have found, among others, Paul Butzer’s titles regarding sampling theorems. I have met Paul Butzer in 1987 in Niš, Serbia at the Third Numerical Methods and Approximation Theory Conference—NMAT’87, organized by Gradimir N. Milovanović, where Butzer delivered a lecture composed from results published in [E]. Writing him soon after that conference concerning the possibility for a postdoc position in Aachen, I have been very pleased with his few pages long response letter in which he explained that this would be not possible since his several duties either in teaching, research or other numerous obligations with postdoc fellows and PhD students. However, he kindly sent me a big package of his own papers covering the area including [A, B, E]. With this gift, I began to collect sampling theorems titles and have a strong insight into stochastic sampling results from Balakrishnan’s and Parzen’s papers to new results (for instance, in the so–called exponential sampling) published nowadays.

First, we recall few definitions and results (listed in boldface). Let $f\in \mathrm{C}(\mathbb{R})$. The modulus of continuity with respect to the difference order r is defined by

$${\omega }_r(\delta ;f;\mathrm{C}(\mathbb{R})):=\underset{\left|h\right|<\delta }{sup}{∥\sum _{k=0}^r{(-1)}^k\left(\genfrac{}{}{0pt}{}{r}{k}\right)f(·+kh)∥}_{\mathrm{C}},\delta >0,$$

and the related Lipschitz class of order $\alpha >0$ having Lipschitz constant L by

$${\mathrm{Lip}}_L(\alpha ;\mathrm{C}(\mathbb{R})):=\left\{f\in \mathrm{C}(\mathbb{R}):{\omega }_r(\delta ;f;\mathrm{C}(\mathbb{R}))\le L{\delta }^{\alpha },\delta >0\right\}.$$

The Bernstein function class ${\mathrm{B}}_{\sigma }^p,1\le p\le \infty$ consists of exponentially bounded functions of type $\sigma$, which restriction to $\mathbb{R}$ belongs to ${\mathrm{L}}^p(\mathbb{R})$. The characterization of ${\mathrm{B}}_{\sigma }^p,1\le p\le 2$ by the Paley–Wiener theorem asserts that an $f\in {\mathrm{L}}^p(\mathbb{R})$ has an extension to $\mathbb{C}$ as an element of ${\mathrm{B}}_{\sigma }^p$ if the associated Fourier transform pair $f^{\wedge }(x)$ vanishes a.e. outside of $[-\sigma ,\sigma ]$, that is, it is bandlimited to this interval:

$$f(t)={\displaystyle \frac{1}{\sqrt{2\pi }}}{\int }_{-\sigma }^{\sigma }{\mathrm{e}}^{\mathrm{i}tx}{f}^{\wedge }(x)\mathrm{d}x,t\in \mathbb{R}.$$

In explaining (even in a heuristic manner!) these results, now, in a stochastic framework, we need the following definitions. The stochastic process $\{X(t,\omega )\equiv X(t):t\in \mathbb{R}\}$ defined on a standard probability space $(\Omega ,\mathcal{A},\mathsf{P})$ is an $\mathcal{A}$–measurable function, that is $X(t_0,\omega )$ becomes a random variable for any fixed $t=t_0\in \mathbb{R}$, while $X(t,{\omega }_0),{\omega }_0\in \Omega$ is a deterministic function called trajectory of X. The process $X(t)$ belongs to the class of all finite second moment random variables ${\mathrm{L}}^2(\Omega ):={\mathrm{L}}^2(\Omega ,\mathcal{A},\mathsf{P})$. This class possesses a Hilbert space structure with scalar product defined by the expectation operator $\mathsf{E}$, that is $\langle X,Y\rangle =\mathsf{E}X{Y}^{\ast }$ and with the norm ${∥X∥}_{{\mathrm{L}}^2}=\sqrt{{\mathsf{E}\left|X\right|}^2}$ equipped.

The expectation function $m_X(t)=\mathsf{E}X(t)$, while the correlation function

$$R_X(t,s):=\mathsf{E}(X(t)-m_X(t)){(X(s)-m_X(s))}^{\ast },t,s\in \mathbb{R}.$$

When $m_X(t)$ is a constant and $R_X(t,s)=R_X(t-s)$, the process is wide sense (Hinčin) stationary (WSS) and we have that $R_X(t)=\mathsf{E}X(t)X^{\ast }(0)$; $R_X(0)=\mathsf{D}X(t)$ is the variance of the process, with the property $|R_X(t)|\le R_X(0)$.

The Karhunen–Cramér theorem, which characterization theorem can also be called generalized harmonizability theorem [25,26,27], reads as follows [5] (p. 156). Assume that the correlation function $R_X(t,s)$ of a stochastic process $X(t)$ possesses a double integral representation in the form

$$R_X(t,s)={\int }_{\Lambda }{\int }_{\Lambda }f(t,\lambda )f^{\ast }(s,\mu )F_X(\mathrm{d}\lambda ,\mathrm{d}\mu ),\Lambda \subseteq \mathbb{R};t,s\in \mathbb{R},$$

(3)

where $F_X(\mathrm{d}\lambda ,\mathrm{d}\mu )$ is a positive definite kernel of finite variation and

$${\int }_{\Lambda }{\int }_{\Lambda }f(t,\lambda )f^{\ast }(s,\mu )F_X(\mathrm{d}\lambda ,\mathrm{d}\mu )<\infty .$$

Then $X(t)$ has a spectral representation and

$$X(t)={\int }_{\Lambda }f(t,\lambda )Z_X(\mathrm{d}\lambda ),t\in T,$$

(4)

where $Z_X$ is a stochastic measure satisfying

$$F_X(S_1,S_2)=\mathsf{E}{Z}_X(S_1)Z_X^{\ast }(S_2),S_1,S_2\in \sigma (\Lambda );$$

(5)

here $\sigma (\Lambda )$ stands for the appropriate σ-field.

Conversely, if the stochastic process $X(t)$ is representable in the form (4) in which the spectral measure $Z_X$ satisfies Equation (5) and $F_X$ is of bounded variation, then there holds the spectral representation (3) of the correlation function $R_X$.

In the case when $f(t,\lambda )$ is a Fourier kernel ${\mathrm{e}}^{\mathrm{i}\lambda t}$ and $F_X(u,v)={\delta }_{uv}{F}_X(u)$, i.e., it is concentrated on the diagonal of ${\mathbb{R}}^2$, the process $X(t)$ becomes WSS. Then, the stochastic process X and the related correlation function $R_X$ have the following spectral representations

$$X(t)={\int }_{\Lambda }{\mathrm{e}}^{\mathrm{i}\lambda t}{Z}_X(\mathrm{d}\lambda ),R_X(t)={\int }_{\Lambda }{\mathrm{e}}^{\mathrm{i}\lambda t}{F}_X(\mathrm{d}\lambda );t\in \mathbb{R},$$

respectively. Here, $F_X(S)=\mathsf{E}{\left|Z_X(S)\right|}^2$, for all $S\in {\mathcal{B}}_{\mathbb{R}}$. Moreover, if $\Lambda \equiv \mathbb{R}$ the process is non-bandlimited, while for $\Lambda =[-w,w],w>0$ the stochastic process is bandlimited to the bandwidth w. Obviously, these formulae readily follow from the spectral representations (4) and (3), having in mind Display (5).

Next, the stochastic process $X(t)$ is continuous and respectively differentiable at $t\in \mathbb{R}$ with a first derivative $X^{\prime }(t)$, both in the mean–square sense used, when

$$\underset{h\to 0}{lim}{∥X(t+h)-X(t)∥}_{{\mathrm{L}}^2}^2=0,\underset{h\to 0}{lim}{∥{\displaystyle \frac{X(t+h)-X(t)}{h}}-X^{\prime }(t)∥}_{{\mathrm{L}}^2}^2=0.$$

Higher order mean–square derivatives $X^{(r)}(t),r=2,3,\cdots$, used in the following exposition, can be defined iteratively.

Finally, let $X\in {\mathrm{L}}^2(\Omega )$, that is let X a finite second–order moment stochastic process. The modulus of continuity for stochastic signal X is defined by

$${\omega }_r(\delta ;X;{\mathrm{L}}^2(\Omega )):=\underset{\left|h\right|<\delta }{sup}{∥\sum _{k=0}^r{(-1)}^k\left(\genfrac{}{}{0pt}{}{r}{k}\right)X(·+kh)∥}_{{\mathrm{L}}^2},\delta >0.$$

Obviously, for WSS processes this definition is t-free. The related Lipschitz class is defined analogously.

Also, we introduce the convolution of a process $X\in {\mathrm{L}}^2(\Omega )$ with a (deterministic) function $g\in {\mathrm{L}}^1(\mathbb{R})$ that turns out to be the (stochastic) integral

$$(X\ast g)(t):={\displaystyle \frac{1}{\sqrt{2\pi }}}{\int }_{\mathbb{R}}X(t)g(t-x)\mathrm{d}x,t\in \mathbb{R},$$

where the validity of this representation holds almost surely (with probability 1). For $X\in {\mathrm{L}}^2(\Omega )$ with $X^{(r)}\in \mathrm{Lip}(\alpha ,{\mathrm{L}}^2(\Omega ))$, $\alpha \in (0,1]$, we have the estimate (in the mean–sequre sense)

$$\underset{t\in \mathbb{R}}{sup}{∥X(t)-(X\ast {\theta }_{\rho })(t)∥}_{{\mathrm{L}}^2}=\mathcal{O}\left({\rho }^{-r-\alpha }\right),\rho \to \infty .$$

Here is ${\theta }_{\rho }(t)=\rho \theta (\rho t)$, where $\theta (·)$ denotes the de la Vallée Poussin kernel [B] (p. 8, Eq. (2.3)), [28]

$$\theta (t)={\displaystyle \frac{4}{\sqrt{2\pi }{t}^2}}sin\left({\displaystyle \frac{t}{2}}\right)sin\left({\displaystyle \frac{3t}{2}}\right),t\in \mathbb{R}.$$

Finally, let us recall that the WKS sampling theorem for the bandlimited stochastic process $X(t)$, having a bandwidth $w>0$ reads

$$X(t)=\sum _{n\in \mathbb{Z}}X\left({\displaystyle \frac{\pi }{w}}n\right){\displaystyle \frac{sin(wt-n\pi )}{wt-n\pi }},t\in \mathbb{R},$$

in accordance with the WKS Formula (2) valid for deterministic signals. However, when we consider the convergence of its truncated to $\left|k\right|\le N$ partial–sums–sequence we arrive at the following result.

Theorem 1 ([B] (p. 57, Theorem 6.11.)).

The bandlimited WSS process X has $R_X\in {\mathrm{B}}_{\pi w}^p,p\ge 1$. Then

$$\underset{N\to \infty }{lim}{\parallel X(t)-\sum _{\left|k\right|\le N}X\left({\displaystyle \frac{k}{w}}\right)\mathrm{sinc}(wt-k)\parallel }_{{\mathrm{L}}^2}^2=0.$$

Theorem 2 ([B] (p. 57, Theorem 6.12.)).

Let X be an r time differentiable WSS process with $X^{(r)}\in \mathrm{Lip}(\alpha ,{\mathrm{L}}^2(\Omega ))$, $\alpha \in (0,1]$. If $|R_X(t)|\le M_r{\left|t\right|}^{-\gamma }$, $\gamma \in (0,1]$, $\left|t\right|\ge 1$, and $M_r=M_r(\gamma )>0$, then

$${\parallel X(t)-\sum _{k\in \mathbb{Z}}X\left({\displaystyle \frac{k}{w}}\right)\mathrm{sinc}(wt-k)\parallel }_{{\mathrm{L}}^2}=\mathcal{O}\left(w^{-r-\alpha }logw\right),w\to \infty .$$

Closing the definitions, we introduce generalized sampling expansion [B] (pp. 24–25) when the sinc kernel is replaced with another suitable kernel $\phi$:

$$(S_w^{\phi }f)(t):={\displaystyle \frac{1}{\sqrt{2\pi }}}\sum _{k\in \mathbb{Z}}f\left({\displaystyle \frac{k}{w}}\right)\phi (wt-k).$$

(6)

Consider [B] (p. 25, Theorem 4.1). When $\phi (t-·)\in {\ell }^1(\mathbb{Z})$ for all $t\in \mathbb{R}$, and

$${\displaystyle \frac{1}{\sqrt{2\pi }}}\sum _{k\in \mathbb{Z}}\phi (t-k)=1.$$

(7)

Then

(a) 

there holds true $f(t)={lim}_{w\to \infty }(S_w^{\phi }f)(t)$, for all $f\in \mathrm{C}(\mathbb{R})$, when $t\in \mathbb{R}$;

(b) 

${\left\{S_w^{\phi }\right\}}_{w>0}$ defines a family of bounded linear operators from $\mathrm{C}(\mathbb{R})$ into itself, so that

$${∥S_w^{\phi }f∥}_{[\mathrm{C},\mathrm{C}]}=m_0(\phi ),\mathrm{and}\underset{w\to \infty }{lim}{∥f-S_w^{\phi }f∥}_{\mathrm{C}}=0.$$

The generalized sampling series expansion associated with the kernel $\phi$ we define in a similar way as in (6) for the input function f, applying the operator $S_w^{\phi }$, i.e.,

$$(S_w^{\phi }X)(t):={\displaystyle \frac{1}{\sqrt{2\pi }}}\sum _{k\in \mathbb{Z}}X\left({\displaystyle \frac{k}{w}}\right)\phi (wt-k),t\in \mathbb{R}.$$

Theorem 3 ([B] (p. 58, Theorem 6.14)).

Let $\phi \in \mathrm{C}(\mathbb{R})$ satisfies (6) and (7), $w>0$. Then there holds

$$\underset{w\to \infty }{lim}{∥X(t)-(S_w^{\phi })X(t)∥}_{{\mathrm{L}}^2}=0,$$

uniformly in $t\in \mathbb{R}$.

Let us define the absolute moment of $\phi$ of the order r [B] (p. 24]

$$m_r(\phi ):=\sum _{t\in \mathbb{R}}{\displaystyle \frac{1}{\sqrt{2\pi }}}\sum _{k\in \mathbb{Z}}{|t-k|}^r\left|\phi (t-k)\right|,r\in {\mathbb{N}}_0.$$

Theorem 4 ([B] (p. 59, Theorem 6.16)).

Let $\phi \in \mathrm{C}(\mathbb{R})$, $m_{2r}(\phi )<\infty$ for certain $r\in \mathbb{N}$. If the moments

$$\sum _{k\in \mathbb{Z}}{(t-k)}^j\phi (t-k)=0,j=1,\cdots ,2r-1;t\in \mathbb{R},$$

then

(c) 

for r–fold differentiable WSS process X, we have

$${∥X(t)-(S_w^{\phi }X)(t)∥}_{{\mathrm{L}}^2}^2\le {\displaystyle \frac{(m_0(\phi )+3)m_{2r}(\phi )}{(2r)!w^{2r}}}{R}_X^{(2r)}(0);$$

(8)

(d) 

moreover, there exists an absolute constant $K>0$, such that for WSS process $X\in {\mathrm{L}}^2(\Omega )$ there holds

$${∥X(t)-(S_w^{\phi }X)(t)∥}_{{\mathrm{L}}^2}\le K{\omega }_r(w^{-1};X;{\mathrm{L}}^2(\Omega )),t\in \mathbb{R},w>0.$$

When $\phi$ coincides with the classical sinc–kernel and the input WSS process is non-bandlimited, we have the well–known Brown’s aliasing error upper bound [29]

$${∥X(t)-(S_w^{\mathrm{sinc}}X)(t)∥}_{{\mathrm{L}}^2}^2\le 4{\int }_{\left|\lambda \right|>w}{F}_X(d\lambda ),$$

(9)

giving a result related to (8) for arbitrary finite $w>0$. The similar fashion result for the non-bandlimited differentiable homogeneous random fields has been established by the author, compare [30] (p. 128, Theorem). During the Fourier Analysis and Applications conference, held at York University in 1993 organized by M. Dodson and M. Beaty, Paul Butzer has drawn the author’s attention to the fact that Brown’s results, and, at the same time his generalization of Brown’s upper bound (9) to non–bandlimited random fields, are Jackson–type inequalities.

Ending this exposure, we have to remark that Theorems 3, 4 are more condensed than Theorems 1 and 2 due to the stochastic nature of the input signals, concerning to the spectral representations either of the stochastic input processes or their correlation functions.

Finally, the generalized sampling expansion operator $S_w^{\phi }$ gives a highly elastic tool in generating very different sampling results manipulating with this operator in several directions: derivative sampling, approximative sampling (considering non-bandlimited input signals), multi-channel sampling etc. The modulus of continuity formulation of the earned results, compare (d) in Theorem 4, gives an excellent insight into analysis background of probabilistic setting of the aliasing approximation error’s magnitude in function of the bandwidth $w>0$ of the input stochastic process by its generalized sampling expansion series.

4. Discussion

It is very interesting, therefore it should be emphasized, that Butzer’s stochastic processes sampling results do not take into account the Karhunen–Cramér theorem. However, his research project was realized in the following way: take the generalized sampling expansion of the signal and find its correlation function; (ii) studying the resulting truncated series; now, of the correlation function, the authors describe the final behavior of this double sum. Actually, Butzer and co-authors follow the similar derivation steps as the great probabilists Kari Karhunen and Harald Cramér when they establish the above mentioned, so-called Karhunen–Cramér theorem. Karhunen’s and Cramér’s proving method was based on the isometric isomorphy between the so-called Hilbert space of the stochastic process $X(t)$, $\mathcal{H}(X):={\mathcal{L}}^2\overline{\{X(t):t\in \mathbb{R}\}}$, say, which is the in medio closure of the linear span of the set $\{X(t):t\in \mathbb{R}\}$, and the H–space

$$L^2(F_X;\Lambda )=\left\{\varphi (z):{\int }_{\Lambda }{\int }_{\Lambda }\varphi (z){\varphi }^{\ast }(w)F_X(\mathrm{d}z,\mathrm{d}w)<\infty \right\},$$

being the class of all complex valued functions integrable with respect to the spectral measure $F_X(·,·)$.

On the other side, the excellent real specialist for stochastic processes in the ‘Aachener Schule’ was Wolfgang Splettstösser; it was an interesting coincidence that three great sampling theoretical specialists Butzer, Splettstösser and Stens have been close not only in their research interests, but also geographically, or this wasn’t the case? We could think in an opposite direction: when they were in the same research team, the Aachen Sampling Project began? Anyway, we have the tremendous research record (consult the exhaustive references list in [B] and [31]!) in the topic presented here, and stochastic counterpart results consist due to such beautiful formulae.