Amplitude Constrained MIMO Channels: Properties of Optimal Input Distributions and Bounds on the Capacity ^†

Abstract

In this work, the capacity of multiple-input multiple-output channels that are subject to constraints on the support of the input is studied. The paper consists of two parts. The first part focuses on the general structure of capacity-achieving input distributions. Known results are surveyed and several new results are provided. With regard to the latter, it is shown that the support of a capacity-achieving input distribution is a small set in both a topological and a measure theoretical sense. Moreover, explicit conditions on the channel input space and the channel matrix are found such that the support of a capacity-achieving input distribution is concentrated on the boundary of the input space only. The second part of this paper surveys known bounds on the capacity and provides several novel upper and lower bounds for channels with arbitrary constraints on the support of the channel input symbols. As an immediate practical application, the special case of multiple-input multiple-output channels with amplitude constraints is considered. The bounds are shown to be within a constant gap to the capacity if the channel matrix is invertible and are tight in the high amplitude regime for arbitrary channel matrices. Moreover, in the regime of high amplitudes, it is shown that the capacity scales linearly with the minimum between the number of transmit and receive antennas, similar to the case of average power-constrained inputs.

1. Introduction

While the capacity of a multiple-input multiple-output (MIMO) channel with an average power constraint is well understood [1], there is surprisingly little known about the capacity of the more practically relevant case in which the channel inputs are subject to amplitude constraints. Shannon was the first who considered a channel that is constrained in its amplitude [2]. In that paper, he derived corresponding upper and lower bounds and showed that in the low-amplitude regime, the capacity behaves as that of channel with an average power constraint. The next major contribution to this problem was a seminal paper of Smith [3] published in 1971. Smith showed that, for the single-input single-output (SISO) Gaussian noise channel with an amplitude-constrained input, the capacity-achieving inputs are discrete with finite support. In [4], this result is extended to peak-power-constrained quadrature Gaussian channels. Using the approach of Shamai [4], it is shown in [5] that the input distribution that achieves the capacity of a MIMO channel with an identity channel matrix and a Euclidian norm constraint on the input vector is discrete. Even though the optimal input distribution is known to be discrete, very little is known about the number or the optimal positions of the corresponding constellation points. To the best of our knowledge, the only case for which the input distribution is precisely known is considered in [6], where it is shown for the Gaussian SISO channel with an amplitude constraint that two point masses are optimal if amplitude values are smaller than $1.665$ and three for amplitude values of up to $2.786$. Finally, it has been shown very recently that the number of mass points in the support of the capacity-achieving input distribution of a SISO channel is of the order $O\left(A^2\right)$ with A the amplitude constraint.

Based on a dual capacity expression, in [7], McKellips derived an upper bound on the capacity of a SISO channel that is subject to an amplitude constraint. The bound is asymptotically tight; that is, for amplitude values that tend to infinity. By cleverly choosing an auxiliary channel output distribution in the dual capacity expression, the authors of [8] sharpened McKellips’ upper bound and extended it to parallel MIMO channels with a Euclidian norm constraint on the input. The SISO version of the upper bound in [8] has been further sharpened in [9] by yet another choice of auxiliary output distribution. In [10], asymptotic lower and upper bounds for a $2\times 2$ MIMO channel are presented and the gap between the bounds is specified.

In this work, we make progress on this open problem by deriving several new upper and lower bounds that hold for channels with arbitrary constraints on the support of the channel input distribution and then apply them to the practically relevant special case of MIMO channels that are subject to amplitude-constraints.

1.1. Contributions and Paper Organization

The remainder of the paper is organized as follows. The problem is stated in Section 2. In Section 3, we study properties of input distributions that achieve the capacity of input-constrained MIMO channels. The section reviews known results on the structure of optimal input distributions and presents several new results. In particular, Theorem 3 shows that the support of a capacity-achieving input distribution must necessarily be a small set both topologically and measure theoretically. Moreover, Theorem 8 characterizes conditions on the channel input space as well as on the channel matrix such that the support of the optimal input distribution is concentrated on the boundary of the channel input space.

In Section 4, we derive novel upper and lower bounds on the capacity of a MIMO channel that is subject to an arbitrary constraint on the support of the input. In particular, three families of upper bounds are proposed, which are based on: (i) the maximum entropy principle (see the bound in Theorem 9); (ii) the dual capacity characterization (see the bound in Theorem 10); and (iii) a relationship between mutual information and the minimum mean square error that is known as the I-MMSE relationship (see the bound in Theorem 11). On the other hand, Section 4 provides three different lower bounds. The first one is given in Theorem 12 and is based on the entropy power inequality. The second one (see Theorem 13) is based on a generalization of the celebrated Ozarow–Wyner bound [11] to the MIMO case. The third upper bound (see Theorem 14) is based on Jensen’s inequality and depends on the characteristic function of the channel input distribution.

In Section 5, we evaluate the performance of our bounds by studying MIMO channels with invertible channel matrices. In particular, Theorem 17 states that our upper and lower bounds are within $n{log}_2\left(\rho \right)$ bits, where $\rho$ is the packing efficiency and n the number of transmit and receive antennas. For diagonal channel matrices, it is then shown (see Theorem 18) that the Cartesian product of simple pulse-amplitude modulation (PAM) constellations achieves the capacity to within $1.64n$ bits.

Section 6 is devoted to MIMO channels with arbitrary channel matrices. It is shown that, in the regime of high amplitudes, similar to the case of average power-constrained channel inputs, the capacity scales linearly with the minimum of the number of transmit and receive antennas.

In Section 7, our upper and lower bounds are applied to the SISO case, which are then compared with bounds known from the literature. Finally, Section 8 concludes the paper. Note that parts of the results in this paper were also published in [12].

1.2. Notation

Vectors are denoted as bold lowercase letters, random vectors as bold uppercase letters, and matrices as bold uppercase sans serif letters (e.g., $\mathbf{x}$, $\mathbf{X}$, $\mathsf{X}$). For any deterministic vector $\mathbf{x}\in {\mathbb{R}}^n$, $n\in \mathbb{N}$, we denote the Euclidian norm of $\mathbf{x}$ by $\parallel \mathbf{x}\parallel$. For some random $\mathbf{X}\in \mathsf{supp}\left(\mathbf{X}\right)\subseteq {\mathbb{R}}^n$ and any $p>0$, we define

$${\parallel \mathbf{X}\parallel }_p:={\left(\frac{1}{n}{\mathbb{E}[\parallel \mathbf{X}\parallel }^p]\right)}^{\frac{1}{p}},$$

(1)

where $\mathsf{supp}\left(\mathbf{X}\right)$ denotes the support of $\mathbf{X}$. Note that for $p\ge 1$, the quantity in Equation (1) defines a norm and for $n=1$ we simply have ${\parallel X\parallel }_p^p={\mathbb{E}\left[\right|X|}^p]$.

The norm of a matrix $\mathsf{H}\in {\mathbb{R}}^{n\times n}$ is defined as

$$\parallel \mathsf{H}\parallel :=\underset{\mathbf{x}:\mathbf{x}\ne \mathbf{0}}{sup}\frac{\parallel \mathsf{H}\mathbf{x}\parallel }{\parallel \mathbf{x}\parallel },$$

whereas $\mathrm{Tr}\left(\mathsf{H}\right)$ is denoting its trace. The $n\times n$ identity matrix is represented as ${\mathsf{I}}_n$.

Let $\mathcal{S}$ be a subset of ${\mathbb{R}}^n$. Then,

$$\mathrm{Vol}\left(\mathcal{S}\right):={\int }_{\mathcal{S}}\mathrm{d}\mathbf{x}$$

denotes its volume. Moreover, the boundary of $\mathcal{S}$ is denoted as $\partial \mathcal{S}$.

Let ${\mathbb{R}}_{+}:=\{x\in \mathbb{R}:x\ge 0\}$. We define an n-dimensional ball or radius $r\in {\mathbb{R}}_{+}$ centered at $\mathbf{x}\in {\mathbb{R}}^n$ as the set

$${\mathcal{B}}_{\mathbf{x}}\left(r\right):=\{\mathbf{y}\in {\mathbb{R}}^n:\parallel \mathbf{x}-\mathbf{y}\parallel \le r\}.$$

Recall that, for any $\mathbf{x}\in {\mathbb{R}}^n$ and $r\in {\mathbb{R}}_{+}$,

$$\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{x}}\left(r\right)\right)=\frac{{\pi }^{\frac{n}{2}}}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}{r}^n,$$

where $\mathsf{\Gamma }\left(z\right)$ denotes the gamma function.

For any matrix $\mathsf{H}\in {\mathbb{R}}^{k\times n}$ and some $\mathcal{S}\subset {\mathbb{R}}^n$, we define

$$\mathsf{H}\mathcal{S}:=\{\mathbf{y}\in {\mathbb{R}}^k:\mathbf{y}=\mathsf{H}\mathbf{x},\mathbf{x}\in \mathcal{S}\}.$$

Note that for an invertible $\mathsf{H}\in {\mathbb{R}}^{n\times n}$, we have

$$\mathrm{Vol}\left(\mathsf{H}\mathcal{S}\right)=|det(\mathsf{H}\left)\right|\mathrm{Vol}\left(\mathcal{S}\right)$$

with $det\left(\mathsf{H}\right)$ the determinant of $\mathsf{H}$. We define the maximum and minimum radius of a set $\mathcal{S}\subset {\mathbb{R}}^n$ that contains the origin as

$$\begin{array}{cc}\hfill r_{\mathsf{max}}\left(\mathcal{S}\right)& :=min\{r\in {\mathbb{R}}_{+}:\mathcal{S}\subset {\mathcal{B}}_{\mathbf{0}}\left(r\right)\},\hfill \\ \hfill r_{\mathsf{min}}\left(\mathcal{S}\right)& :=max\{r\in {\mathbb{R}}_{+}:{\mathcal{B}}_{\mathbf{0}}\left(r\right)\subseteq \mathcal{S}\}.\hfill \end{array}$$

For a given vector $\mathbf{a}=(a_1,\cdots ,a_n)\in {\mathbb{R}}_{+}^n$, we define

$$\mathrm{Box}\left(\mathbf{a}\right):=\{\mathbf{x}\in {\mathbb{R}}^n:|x_i|\le a_i,i=1,\cdots ,n\}$$

and the smallest box containing a given set $\mathcal{S}\subset {\mathbb{R}}^n$ as

$$\mathrm{Box}\left(\mathcal{S}\right):=inf\left\{\mathrm{Box}\right(\mathbf{a}):\mathcal{S}\subseteq \mathrm{Box}(\mathbf{a}\left)\right\},$$

respectively.

The entropy of any discrete random object $\mathbf{X}$ is denoted as $H\left(\mathbf{X}\right)$, whereas $h\left(\mathbf{X}\right)$ (i.e., the differential entropy) is used whenever $\mathbf{X}$ is continuous. The mutual information between two random objects $\mathbf{X}$ and $\mathbf{Y}$ is denoted as $I(\mathbf{X};\mathbf{Y})$ and $\mathcal{N}(\mathbf{m},\mathsf{C})$ denotes the multivariate normal distribution with mean vector $\mathbf{m}$ and covariance matrix $\mathsf{C}$. Finally, ${log}_a^{+}\left(x\right):=max\{{log}_a\left(x\right),0\}$, for any base $a>0$, $Q\left(x\right)$, $x\in \mathbb{R}$, denotes the Q-function, and ${\delta }_{\mathbf{x}}\left(\mathbf{y}\right)$ the Kronecker delta, which is one for $\mathbf{x}=\mathbf{y}$ and zero otherwise.

2. Problem Statement

Consider a MIMO system with $n_t\in \mathbb{N}$ transmit and $n_r\in \mathbb{N}$ receive antennas. The corresponding $n_r$-dimensional channel output for a single channel use is of the form

$$\mathbf{Y}=\mathsf{H}\mathbf{X}+\mathbf{Z},$$

for some fixed channel matrix $\mathsf{H}\in {\mathbb{R}}^{n_r\times n_t}$ (considering a real-valued channel model is without loss of generality). Here and hereafter, we assume $\mathbf{Z}\sim \mathcal{N}(\mathbf{0},{\mathsf{I}}_{n_r})$ is independent of the channel input $\mathbf{X}\in {\mathbb{R}}^{n_t}$ and $\mathsf{H}$ is known to both the transmitter and the receiver.

Now, let $\mathcal{X}\subset {\mathbb{R}}^{n_t}$ be a convex and compact channel input space that contains the origin (i.e., the length-$n_t$ zero vector) and let $F_{\mathbf{X}}$ denote the cumulative distribution function of $\mathbf{X}$. As of the writing of this paper, the capacity

$$C(\mathcal{X},\mathsf{H}):=\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathbf{X};\mathbf{Y})=\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z}),$$

(2)

of this channel is unknown and we are interested in finding novel lower and upper bounds. Even though most of the results in this paper hold for arbitrary convex and compact $\mathcal{X}$, we are mainly interested in the two important special cases:

(i)

per-antenna amplitude constraints, i.e., $\mathcal{X}=\mathrm{Box}\left(\mathbf{a}\right)$ for some given $\mathbf{a}=(A_1,\cdots ,A_{n_t})\in {\mathbb{R}}_{+}^{n_t}$; and

(ii)

$n_t$-dimensional amplitude constraint, i.e., $\mathcal{X}={\mathcal{B}}_{\mathbf{0}}\left(A\right)$ for some given $A\in {\mathbb{R}}_{+}$.

Remark 1.

Note that determining the capacity of a MIMO channel with average per-antenna power constraints is also still an open problem and has been solved for some special cases only [13,14,15,16,17].

3. Properties of an Optimal Input Distribution

Unlike the special cases of real and complex-valued SISO channels (i.e., $n_t=n_r=1$), the structure of the capacity-achieving input distribution, denoted as $F_{\mathbf{X}}^{\star }$, is in general not known. To motivate why in this paper we are seeking for novel upper and lower bounds on the capacity (Equation (2)) instead of trying to solve the optimization problem directly, in this section we first summarize properties optimal input distributions must posses, which demonstrate how complicated the optimization problem actually is. Note that, whereas an optimal input distribution always exists, it does not necessarily need to be unique.

3.1. Necessary and Sufficient Conditions for Optimality

To study properties of an optimal input distribution, we need the notion of a point of increase of probability distribution.

Definition 1.

(Points of Increase of a Distribution) A point $\mathbf{x}\in {\mathbb{R}}^n$, $n\in \mathbb{N}$, is said to be a point of increase of a given probability distribution $F_{\mathbf{X}}$ if for any open set $\mathcal{A}\subset {\mathbb{R}}^n$ containing $\mathbf{x}$, $F_{\mathbf{X}}\left(\mathcal{A}\right)>0$.

The following result provides necessary and sufficient conditions for the optimality of a channel input distribution.

Theorem 1.

Let $F_{\mathbf{X}}$ be some given channel input distribution and let $\mathcal{E}\left(F_{\mathbf{X}}\right)\subset \mathcal{X}$ denote the set of points of increase of $F_{\mathbf{X}}$. Then, the following holds:

• $F_{\mathbf{X}}$ is capacity-achieving if and only if the Karush–Kuhn–Tucker (KKT) conditions

  $$\begin{array}{cc}\hfill h(\mathbf{x};F_{\mathbf{X}})& \le h(\mathsf{H}\mathbf{X}+\mathbf{Z}),\mathbf{x}\in \mathcal{X},\hfill \end{array}$$

  (3a)

  $$\begin{array}{cc}\hfill h(\mathbf{x};F_{\mathbf{X}})& =h(\mathsf{H}\mathbf{X}+\mathbf{Z}),\mathbf{x}\in \mathcal{E}\left(F_{\mathbf{X}}\right)\subset \mathcal{X},\hfill \end{array}$$

  (3b)

  are satisfied [3,18], where

  $$\begin{array}{c}\hfill h(\mathbf{x};F_{\mathbf{X}}):=-{\int }_{{\mathbb{R}}^{n_r}}\frac{1}{{\left(2\pi \right)}^{\frac{n_r}{2}}}{\mathrm{e}}^{-\frac{{\parallel \mathbf{y}-\mathsf{H}\mathbf{x}\parallel }^2}{2}}{log}_2\left(f_{\mathbf{Y}}\left(\mathbf{y}\right)\right)\mathrm{d}\mathbf{y}\end{array}$$

  with $f_{\mathbf{Y}}$ being the probability density of the channel output induced by the channel input $\mathbf{X}\sim F_{\mathbf{X}}$.
• $F_{\mathbf{X}}$ is unique and symmetric if $\mathsf{H}$ is left invertible [18].
• $F_{\mathbf{Y}}$ (i.e., the channel output distribution) is unique [18,19].

3.2. General Structure of Capacity-Achieving Input Distributions

Theorem 1 can be used to find general properties of the support of a capacity-achieving input distribution, which we will do in this subsection.

Remark 2.

Fully characterizing an input distribution that achieves the capacity of a general MIMO channel with per-antenna or an $n_t$-dimensional amplitude constraint is still an open problem. To the best of our knowledge, the only general available for showing that discrete channel inputs are optimal was developed by Smith in [3] for the amplitude and variance-constrained Gaussian SISO channel. Since then, it has been useful to also characterize the optimal input distribution of several other SISO channels (see, for instance, [4,20,21,22,23,24]). The method relies on the following series of steps:

• Towards a contradiction, it is assumed that the set of points of increase $\mathcal{E}\left(F_{\mathbf{X}}\right)$ is infinite.
• The assumption in Step 1 is then used to establish a certain property of the function $h(\mathbf{x};F_{\mathbf{X}})$ on the input space $\mathcal{X}$. For example, by showing that $h(\mathbf{x};F_{\mathbf{X}})$ has an analytic continuation to $\mathbb{C}$. Then, by means of the Identity Theorem of complex analysis and the Bolzano–Weierstrass Theorem [25], Smith was able to show that $h(\mathbf{x};F_{\mathbf{X}})$ must be constant.
• By using either the Fourier or Laplace transform of $h(\mathbf{x};F_{\mathbf{X}})$ together with the property of $h(\mathbf{x};F_{\mathbf{X}})$ established in Step 2, a new a property of the channel output distribution $F_{\mathbf{Y}}$ is established. For example, Smith was able to show that $F_{\mathbf{Y}}$ must be constant.
• A conclusion out of Step 3 is used to reach a contradiction. The contradiction implies that $\mathcal{E}\left(F_{\mathbf{X}}\right)$ must be finite. For example, to reach a contradiction, Smith was using the fact that the channel output distribution $F_{\mathbf{Y}}$ results from a convolution with a Gaussian probability density, which cannot be constant.

Remark 3.

Under the restriction that the output space, $\mathcal{Y}$, of a Gaussian SISO channel is finite and the channel input space, $\mathcal{X}$, is subject to an amplitude constraint, Witsenhausen has shown in [26] that the capacity-achieving input distribution is discrete with the number of mass points bounded as $\left|\mathcal{X}\right|\le \left|\mathcal{Y}\right|$. The approach of Witsenhausen, however, does not use the variational technique of Smith and relies on arguments from convex analysis instead.

According to Remark 2, assuming in the MIMO case that $\mathcal{E}\left(F_{\mathbf{X}}\right)$ is of infinite cardinality does not help (or at least it is not clear how this assumption should be used) in showing that the capacity-achieving input distribution is discrete and finite. However, by using the weaker assumption that $\mathcal{E}\left(F_{\mathbf{X}}\right)$ contains a non-empty open subset in conjunction with the following version of the Identity Theorem, we can show that the support of the optimal input distribution is a small set in a certain topological sense.

Theorem 2.

(Identity Theorem for Real-Analytic Functions [27]) For some $n\in \mathbb{N}$ let $\mathcal{U}$ be a subset of ${\mathbb{R}}^n$ and $f,g:\mathcal{U}\to \mathbb{R}$ be two real-analytic functions that agree on a set $\mathcal{A}\subseteq \mathcal{U}$. Then, f and g agree on ${\mathbb{R}}^n$ if one of the following two conditions is satisfied:

(i)

$\mathcal{A}$ is an open set.

(ii)

$\mathcal{A}$ is a set of positive Lebesgue measure.

Furthermore, for $n=1$, it suffices for $\mathcal{A}$ to be an arbitrary set with an accumulation point.

We also need the definitions of a dense and a nowhere dense set.

Definition 2.

(Dense and Nowhere Dense Sets) A subset $\mathcal{A}\subset \mathcal{X}$ is said to be dense in the set $\mathcal{X}$ if every element $\mathbf{x}\in \mathcal{X}$ either belongs to $\mathcal{A}$ or is an accumulation point of $\mathcal{A}$. A subset $\mathcal{A}\subset \mathcal{X}$ is said to be nowhere dense if for every nonempty open subset $\mathcal{U}\subset \mathcal{X}$, the intersection $\mathcal{A}\cap \mathcal{U}$ is not dense in $\mathcal{U}$.

With Theorem 2 at our disposal, we are now able to prove the following result on the structure of the support of the optimal input distribution.

Theorem 3.

The set of points of increase $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ of an optimal input distribution $F_{\mathbf{X}}^{\star }$ is a nowhere dense subset of $\mathcal{X}$ that is of Lebesgue measure zero.

Proof. 

It is not difficult to show that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ is a real-analytic function on ${\mathbb{R}}^{n_t}$ ([18] Proposition 5). Now, in order to prove the result, we follow a series of steps similar to those outlined in Remark 2. Towards a contradiction, assume that the set of points of increase $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ of $F_{\mathbf{X}}^{\star }$ is not a nowhere dense subset of $\mathcal{X}$. Then, according to Definition 2, there exists an open set $\mathcal{U}\subset \mathcal{X}$ such that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\cap \mathcal{U}$ is dense in $\mathcal{U}$.

By using the KKT condition in Equation (3b), we have that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ is constant on the intersection $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\cap \mathcal{U}$. Thus, as $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\cap \mathcal{U}$ is dense in $\mathcal{U}$, it follows by the properties of continuous functions (real-analytic functions are continuous) that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ is also constant on $\mathcal{U}$. Moreover, as $\mathcal{U}$ is an open set, Theorem 2 implies that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ must also be constant on ${\mathbb{R}}^{n_t}$. This, however, leads to a contradiction as $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ cannot be constant on all of ${\mathbb{R}}^{n_t}$, which can be shown by taking the Fourier transform of $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ and solving for the probability density $f_{\mathbf{Y}}\left(\mathbf{y}\right)$ of the channel output (the reader is referred to [3] for details). Therefore, we conclude that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ is a nowhere dense subset of $\mathcal{X}$.

Showing that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ has Lebesgue measure zero follows along similar lines by assuming that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ is a set of positive measure. Then, Property (ii) of Theorem 2 can be used to conclude that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ must be zero on all of $\mathcal{U}$. This again leads to a contradiction, which implies that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ must be of Lebesgue measure zero. □

Remark 4.

Note that if $\mathcal{X}={\mathcal{B}}_{\mathbf{0}}\left(A\right)$ for some $A\in {\mathbb{R}}_{+}$ and $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ is orthogonally equivariant (i.e., it only depends on $\parallel \mathbf{x}\parallel$), then $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ can be written as a union of concentric spheres. That is,

$$\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)=\bigcup _j\mathcal{C}\left(A_j\right)$$

(4)

with $\mathcal{C}\left(A_j\right):=\{\mathbf{x}\in {\mathbb{R}}^{n_t}:\parallel \mathbf{x}\parallel =A_j\}$ for some $A_j\in {\mathbb{R}}_{+}$. To see this, let

$$g\left(\mathbf{x}\right):=h(\mathbf{x};F_{\mathbf{X}}^{\star })-h(\mathsf{H}\mathbf{X}+\mathbf{Z})$$

and observe that if $\mathbf{x}\in \mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$, then $g\left(\mathbf{x}\right)=0$. Combining this with the symmetry of the function $\parallel \mathbf{x}\parallel \mapsto g(\parallel \mathbf{x}\parallel )$, we have that (We know that it is abuse of notation to use the same letter for the functions $\mathbf{x}\mapsto g\left(\mathbf{x}\right)$ and $\parallel \mathbf{x}\parallel \mapsto g(\parallel \mathbf{x}\parallel )$ even if they are different. It is an attempt to say in a compact way that g is orthogonally equivariant.)

$$\forall \parallel \mathbf{x}\parallel =\parallel \mathbf{y}\parallel :\mathbf{x}\in \mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\Rightarrow \mathbf{y}\in \mathcal{E}\left(F_{\mathbf{X}}^{\star }\right).$$

Moreover, this implies that

$$\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)=\bigcup _{j\in \mathcal{I}}\mathcal{C}\left(A_j\right),$$

where $\mathcal{I}$ is possibly of infinite cardinality. In fact, $\mathcal{I}$ has finite cardinality. To see this, note that, if $g\left(\mathbf{x}\right)$ is real-analytic, then so is $g(\parallel \mathbf{x}\parallel )$. However, as $\parallel \mathbf{x}\parallel \mapsto g(\parallel \mathbf{x}\parallel )$ is a non-zero real-analytic function on $\mathbb{R}$, it can have at most finitely many zeros on an interval.

As an example consider the special case $n_r=n_t=n$ with $\mathsf{H}={\mathsf{I}}_n$. Then, the union in Equation (4) implies that the cardinality of $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ is uncountable and that discrete inputs are in general not optimal. Therefore, Theorem 3 can generally not be improved in the sense that for $n>1$, statements about the cardinality of $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$ cannot be made. Note, however, that the magnitude of $\mathbf{X}$ is discrete. An example of the corresponding optimal input distribution for the case of $n=2$ is given in Figure 1.

Even though Theorem 3 does not allow us to conclude that the optimal input distribution of an arbitrary MIMO channel is discrete and finite, for the special case of a SISO channel we have the following partial result.

Theorem 4.

(Optimal Input Distribution of a SISO Channel [3,6,28]) For some fixed $h\in \mathbb{R}$ and $A\in {\mathbb{R}}_{+}$, consider the SISO channel $Y=hX+Z$ with input space $\mathcal{X}=[-A,A]$. Let $F_X^{\star }$ be an input distribution that achieves the capacity, $C(\mathcal{X},h)$, of that channel. Then, $F_X^{\star }$ satisfies the following properties:

• $F_X^{\star }$ is unique.
• $F_X^{\star }$ is symmetric.
• $F_X^{\star }$ is discrete with the number of mass points being of the order $O\left(A^2\right)$.
• $F_X^{\star }$ contains probability mass points at $\{-A,A\}$.

Moreover, binary communication with mass points at $\{-A,A\}$ is optimal if and only if $A\le \overline{A}$, where $\overline{A}\approx 1.665$.

Theorem 4 can now be used to also address the special cases of multiple-input single output (MISO) and single-input multiple output (SIMO) channels.

Theorem 5.

Let $Y={\mathsf{h}}^T{\mathbf{X}}^{\star }+Z$ be a MISO channel with channel matrix ${\mathsf{h}}^T\in {\mathbb{R}}^{n_t}$ and some optimal input ${\mathbf{X}}^{\star }\in \mathcal{X}\subset {\mathbb{R}}^{n_t}$. Then, the distribution of ${\mathsf{h}}^T{\mathbf{X}}^{*}$ is discrete with finitely many mass points. On the other hand, let $\mathbf{Y}=\mathsf{h}{X}^{\star }+\mathbf{Z}$ be a SIMO channel with channel matrix $\mathsf{h}\in {\mathbb{R}}^{n_r}$. Then, the optimal input $X^{\star }\in \mathcal{X}\subset \mathbb{R}$ has a discrete distribution with finitely many mass points.

Proof. 

For the MISO case, the capacity can expressed as

$$\begin{array}{cc}\hfill C(\mathcal{X},{\mathsf{h}}^T)& =\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathbf{X};{\mathsf{h}}^T\mathbf{X}+Z)\hfill \\ & =\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I({\mathsf{h}}^T\mathbf{X};{\mathsf{h}}^T\mathbf{X}+Z)\hfill \\ & =\underset{F_S:S\in {\mathsf{h}}^T\mathcal{X}}{max}I(S;S+Z)\hfill \\ & =\underset{F_S:\left|S\right|\le r_{\mathsf{max}}\left({\mathsf{h}}^T\mathcal{X}\right)}{max}I(S;S+Z).\hfill \end{array}$$

(5)

Using Theorem 5 we have that the maximizing distribution in Equation (5) $F_S^{\star }$, where $S:={\mathsf{h}}^T\mathbf{X}$, is discrete with finitely many mass points.

For the SIMO case, the channel input distribution is discrete as a SIMO channel can be transformed into a SISO channel. Thus, let $A\in {\mathbb{R}}_{+}$ be finite. Then, the capacity of the SIMO channel can be expressed as

$$\begin{array}{cc}\hfill C(\mathcal{X},\mathsf{h})& =\underset{F_X:\left|X\right|\le A}{max}I(X;\mathsf{h}X+\mathbf{Z})\hfill \\ & =\underset{F_X:\left|X\right|\le A}{max}I(X;\parallel \mathsf{h}\parallel X+Z).\hfill \end{array}$$

(6)

Again, it follows from Theorem 5 that the mutual information in Equation (6) is maximized by a channel input distribution, $F_X^{\star }$, that is discrete with finitely many mass points. This concludes the proof. □

Remark 5.

Note that in the MISO case, we do not claim $F_{\mathbf{X}}^{\star }$ to be discrete with finitely many points. To illustrate the difficulty, let ${\mathsf{h}}^T=[1,-1]$ so that

$${\mathsf{h}}^T{\mathbf{X}}^{\star }={\mathsf{h}}^T\left(\begin{array}{c}{X}_1^{\star }\\ X_2^{\star }\end{array}\right)=X_1^{\star }-X_2^{\star }.$$

As $X_1^{\star }$ and $X_2^{\star }$ can be arbitrarily correlated, we cannot rule out cases in which $X_1^{\star }=X-D$ and $X_2^{\star }=X-2D$, with D a discrete random variable and X of arbitrary distribution. Clearly the distribution of ${\mathbf{X}}^{\star }$ is not discrete.

Note that in general it can be shown that the capacity-achieving input distribution is discrete if the optimization problem in Equation (2) can be reformulated as an optimization over one dimensional distributions. This, for example, has been done in [5] for parallel channels with a total amplitude constraint.

3.3. Properties of Capacity-Achieving Input Distributions in the Small (But Not Vanishing) Amplitude Regime

In this subsection, we study properties of capacity-achieving input distribution in the regime of small amplitudes. To that end, we will need the notion of a subharmonic function.

Definition 3.

(Subharmonic Function) Let f be a real-valued function that is twice continuously differentiable on an open set $\mathcal{G}\subset {\mathbb{R}}^n$. Then, f is subharmonic if ${\nabla }^{2}f\ge 0$ on $\mathcal{G}$, where ${\nabla }^2$ denotes the Laplacian (if f is twice differentiable, its Laplacian is given by ${\nabla }^{2}f(x_1,\cdots ,x_n)={\sum }_{i=1}^n\frac{{\partial }^{2}f}{\partial x_i^2}$).

We use the following Theorem, which states that a subharmonic function always attains its maximum on the boundary of its domain.

Theorem 6.

(Maximum Principle of Subharmonic Functions [29]) Let $\mathcal{G}\subset {\mathbb{R}}^n$ be a connected open set. If $f:\mathcal{G}\to \mathbb{R}$ is subharmonic and attains a global maximum in the interior of $\mathcal{G}$, then f is constant on $\mathcal{G}$.

In addition to Theorem 6, we need the following result that has been proven in [30].

Lemma 1.

Let the likelihood function of the output of a MIMO channel be defined as

$$\ell :{\mathbb{R}}^{n_r}\to \mathbb{R},\ell \left(\mathbf{y}\right)=\frac{f_{\mathbf{Y}}\left(\mathbf{y}\right)}{\frac{1}{{\left(2\pi \right)}^{\frac{n_r}{2}}}{\mathrm{e}}^{-\frac{{\parallel \mathbf{y}\parallel }^2}{2}}}$$

and let ${\mathsf{A}}_{\mathbf{y}}$ denote the Hessian matrix of ${log}_{\mathrm{e}}\left(\ell \left(\mathbf{y}\right)\right)$. Then, the Laplacian (or the trace of ${\mathsf{A}}_{\mathbf{y}}$) is given by

$${\nabla }^2{log}_{\mathrm{e}}\left(\ell \left(\mathbf{y}\right)\right)=\mathrm{Tr}\left({\mathsf{A}}_{\mathbf{y}}\right)=\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2|\mathbf{Y}=\mathbf{y}\right]-{∥\mathbb{E}\left[\mathsf{H}\mathbf{X}\right|\mathbf{Y}=\mathbf{y}]∥}^2.$$

(7)

Theorem 7.

Suppose that $r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le {log}_2\left(\mathrm{e}\right)$. Then, $\mathbf{x}\mapsto h(\mathbf{x};F_{\mathbf{X}})$ is a subharmonic function for every $F_{\mathbf{X}}$.

Proof. 

Let $F_{\mathbf{X}}$ be arbitrary and observe that

$$\begin{array}{cc}\hfill h(\mathbf{x};F_{\mathbf{X}})& =-\mathbb{E}\left[{log}_2\left(f_{\mathbf{Y}}(\mathsf{H}\mathbf{x}+\mathbf{Z})\right)\right]\hfill \\ & =-\mathbb{E}\left[\ell (\mathsf{H}\mathbf{x}+\mathbf{Z})\right]-\mathbb{E}\left[{log}_2\left(\frac{1}{{\left(2\pi \right)}^{\frac{n_r}{2}}}{\mathrm{e}}^{-\frac{{\parallel \mathsf{H}\mathbf{x}+\mathbf{Z}\parallel }^2}{2}}\right)\right]\hfill \\ & =-\mathbb{E}\left[\ell (\mathsf{H}\mathbf{x}+\mathbf{Z})\right]+\mathbb{E}\left[\frac{{\parallel \mathsf{H}\mathbf{x}+\mathbf{Z}\parallel }^2}{2}\right]{log}_2\left(\mathrm{e}\right)+{log}_2\left({\left(2\pi \right)}^{\frac{n_r}{2}}\right)\hfill \\ & =-\mathbb{E}\left[\ell (\mathsf{H}\mathbf{x}+\mathbf{Z})\right]+\frac{{\parallel \mathsf{H}\mathbf{x}\parallel }^2+n_r}{2}{log}_2\left(\mathrm{e}\right)+{log}_2\left({\left(2\pi \right)}^{\frac{n_r}{2}}\right).\hfill \end{array}$$

With this expression in hand, the Laplacian of $h(\mathbf{x};F_{\mathbf{X}})$ with respect to $\mathbf{x}$ can be bounded from below as follows:

$$\begin{array}{cc}\hfill {\nabla }^{2}h(\mathbf{x};F_{\mathbf{X}})& ={\nabla }^2\left(-\mathbb{E}\left[\ell (\mathsf{H}\mathbf{x}+\mathbf{Z})\right]+\frac{{\parallel \mathsf{H}\mathbf{x}\parallel }^2+n_r}{2}{log}_2\left(\mathrm{e}\right)+{log}_2\left({\left(2\pi \right)}^{\frac{n_r}{2}}\right)\right)\hfill \\ & =-\mathbb{E}\left[{\nabla }^2\ell (\mathsf{H}\mathbf{x}+\mathbf{Z})\right]+{\nabla }^2\frac{{\parallel \mathsf{H}\mathbf{x}\parallel }^2}{2}{log}_2\left(\mathrm{e}\right)\hfill \\ & \stackrel{\left(a\right)}{=}-\mathbb{E}\left[\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T{\mathsf{A}}_{\mathsf{H}\mathbf{x}+\mathbf{Z}}\right)\right]+{\nabla }^2\frac{{\parallel \mathsf{H}\mathbf{x}\parallel }^2}{2}{log}_2\left(\mathrm{e}\right)\hfill \\ & =-\mathbb{E}\left[\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T{\mathsf{A}}_{\mathsf{H}\mathbf{x}+\mathbf{Z}}\right)\right]+\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T\right){log}_2\left(\mathrm{e}\right)\hfill \\ & \stackrel{\left(b\right)}{\ge }-\mathbb{E}\left[\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T\right)\mathrm{Tr}\left({\mathsf{A}}_{\mathsf{H}\mathbf{x}+\mathbf{Z}}\right)\right]+\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T\right){log}_2\left(\mathrm{e}\right)\hfill \\ & =\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T\right)\left(-\mathbb{E}\left[\mathrm{Tr}\left({\mathsf{A}}_{\mathsf{H}\mathbf{x}+\mathbf{Z}}\right)\right]+log\left(\mathrm{e}\right)\right)\hfill \\ & \stackrel{\left(c\right)}{\ge }\mathrm{Tr}\left(\mathsf{H}{\mathsf{H}}^T\right)\left(-r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)+log\left(\mathrm{e}\right)\right),\hfill \end{array}$$

(8)

where $\left(a\right)$ follows from Equation (7) and the chain rule for the Hessian; $\left(b\right)$ from using the well-known inequality

$$\mathrm{Tr}{\left(\mathsf{C}\mathsf{D}\right)}^2\le \mathrm{Tr}{\left(\mathsf{C}\right)}^2\mathrm{Tr}{\left(\mathsf{D}\right)}^2$$

that holds for $\mathsf{C}$ and $\mathsf{D}$ positive semi-definite; and $\left(c\right)$ from using the inequality

$$\begin{array}{cc}\hfill \mathrm{Tr}\left({\mathsf{A}}_{\mathbf{y}}\right)& =\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2|\mathbf{Y}=\mathbf{y}\right]-{∥\mathbb{E}\left[\mathsf{H}\mathbf{X}\right|\mathbf{Y}=\mathbf{y}]∥}^2\hfill \\ & \le \mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2|\mathbf{Y}=\mathbf{y}\right]\hfill \\ & \le r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right).\hfill \end{array}$$

Thus, according to the assumption that $r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le {log}_2\left(\mathrm{e}\right)$, the right-hand side of Equation (8) is nonnegative, which proves the result. □

Now, knowing that $h(\mathbf{x};F_{\mathbf{X}})$ is a subharmonic function allows us to characterize the support of an optimal input distribution of a MIMO channel provided that the radius of the channel input space, $\mathcal{X}$, is sufficiently small.

Theorem 8.

Let $F_{\mathbf{X}}^{\star }$ be a capacity-achieving input distribution and $r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le {log}_2\left(\mathrm{e}\right)$. Then, $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\subseteq \partial \mathcal{X}$.

Proof. 

From the KKT conditions in Equation (3), we know that, if $\mathbf{x}\in \mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)$, then $\mathbf{x}$ is a maximizer of $h(\mathbf{x};F_{\mathbf{X}}^{\star })$. According to Theorem 7, we also know that $h(\mathbf{x};F_{\mathbf{X}}^{\star })$ is subharmonic. Hence, from the Maximum Principle of Subharmonic Functions (i.e., Theorem 6), it follows $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\subseteq \partial \mathcal{X}$. □

Combining Theorem 8 with the observations made in Remark 4 leads to the following corollary.

Corollary 1.

Let $\mathcal{X}={\mathcal{B}}_{\mathbf{0}}\left(A\right)$ and $A\le \frac{1}{\parallel \mathsf{H}\parallel }{log}_2\left(\mathrm{e}\right)$. Then,

$$\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)\subseteq \mathcal{C}\left(A\right),$$

where $\mathcal{C}\left(A\right):=\{\mathbf{x}\in {\mathbb{R}}^n:\parallel \mathbf{x}\parallel =A\}$ denotes a sphere of radius A.

We conclude this section by noting that for the special case $n_t=n_r=n$ with $\mathsf{H}={\mathsf{I}}_n$, the exact value of A such that $\mathcal{E}\left(F_{\mathbf{X}}^{\star }\right)=\mathcal{C}\left(A\right)$ has been characterized in terms of an integral equation in [31], which is approximately equal to $1.5\sqrt{n}$.

4. Upper and Lower Bounds on the Capacity

The considerations in the previous section have shown that characterizing the structure of an optimal channel-input distribution is a challenging question in itself that we could only partially answer. A full characterization, however, is a necessary prerequisite to narrow down the search space in Equation (2) to one that is tractable. Except for some special cases (i.e., special choices of $\mathcal{X}$), optimizing over the most general space of input distributions that consists of all continuous $n_t$-dimensional probability distributions $F_{\mathbf{X}}$ with $\mathbf{X}\in \mathcal{X}$, is prohibitive (Note that Dytso et al. [32] summarized methods of how to optimize functionals over the space of probability distributions that are constrained in there support). Thus, up to the writing of this paper, there is little hope in being able to solve the problem in Equation (2) in full generality so that in this section we are proposing novel lower and upper bounds on the capacity $C(\mathcal{X},\mathsf{H})$. Nevertheless, these bounds will allow us to better understand how the capacity of such MIMO channels behaves.

Towards this end, in Section 4.1, we provide four upper bounds. The first is based on an upper bound on the differential entropy of a random vector that is constraint in its pth moment, the second and third bounds are based on duality arguments, and the fourth on the relationship between mutual information and the minimum mean square error (MMSE), I-MMSE relationship for short, known from [33]. The three lower bounds proposed in Section 4.2, on the other hand, are based on the celebrated entropy power inequality, a generalization of the Ozarow–Wyner capacity bound taken from [11], and on Jensen’s inequality.

4.1. Upper Bounds

To establish our first upper bound on Equation (2), we need the following result ([11] Th. 1).

Lemma 2.

(Maximum Entropy Under pth Moment Constraint) Let $n\in \mathbb{N}$ and $p\in (0,\infty )$ be arbitrary. Then, for any $\mathbf{U}\in {\mathbb{R}}^n$ such that $h\left(\mathbf{U}\right)<\infty$ and ${\parallel \mathbf{U}\parallel }_p<\infty$, we have

$$h\left(\mathbf{U}\right)\le n{log}_2\left(k_{n,p}{n}^{\frac{1}{p}}{\parallel \mathbf{U}\parallel }_p\right),$$

where

$$k_{n,p}:=\frac{\sqrt{\pi }{\mathrm{e}}^{\frac{1}{p}}{\left(\frac{p}{n}\right)}^{\frac{1}{p}}\mathsf{\Gamma }{\left(\frac{n}{p}+1\right)}^{\frac{1}{n}}}{\mathsf{\Gamma }{\left(\frac{n}{2}+1\right)}^{\frac{1}{n}}}.$$

Theorem 9.

(Moment Upper Bound) For any channel input space $\mathcal{X}$ and any fixed channel matrix $\mathsf{H}$, we have

$$\begin{array}{c}\hfill C(\mathcal{X},\mathsf{H})\le {\overline{C}}_{\mathsf{M}}(\mathcal{X},\mathsf{H}):=\underset{p>0}{inf}{n}_r{log}_2\left(\frac{k_{n_r,p}}{{\left(2\pi \mathrm{e}\right)}^{\frac{1}{2}}}{n}_r^{\frac{1}{p}}{\parallel \tilde{\mathbf{x}}+\mathbf{Z}\parallel }_p\right),\end{array}$$

where $\tilde{\mathbf{x}}\in \mathsf{H}\mathcal{X}$ is chosen such that $\parallel \tilde{\mathbf{x}}\parallel =r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right)$.

Proof. 

Expressing Equation (2) in terms of differential entropies results in

$$\begin{array}{cc}\hfill C(\mathcal{X},\mathsf{H})& =\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}h(\mathsf{H}\mathbf{X}+\mathbf{Z})-h\left(\mathbf{Z}\right)\hfill \\ & \stackrel{\left(a\right)}{\le }\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}{n}_r{log}_2\left(\frac{k_{n_r,p}}{{\left(2\pi \mathrm{e}\right)}^{\frac{1}{2}}}{n}_r^{\frac{1}{p}}{\parallel \mathsf{H}\mathbf{X}+\mathbf{Z}\parallel }_p\right)\hfill \\ & \stackrel{\left(b\right)}{=}{n}_r{log}_2\left(\frac{k_{n_r,p}}{{\left(2\pi \mathrm{e}\right)}^{\frac{1}{2}}}{n}_r^{\frac{1}{p}}\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}{\parallel \mathsf{H}\mathbf{X}+\mathbf{Z}\parallel }_p\right),\hfill \end{array}$$

(9)

where $\left(a\right)$ follows from Lemma 2 with the fact that $h\left(\mathbf{Z}\right)=\frac{n_r}{2}{log}_2\left(2\pi \mathrm{e}\right)$; and $\left(b\right)$ from the monotonicity of the logarithm.

Now, notice that ${\parallel \mathsf{H}\mathbf{X}+\mathbf{Z}\parallel }_p$ is linear in $F_{\mathbf{X}}$ and therefore it attains its maximum at an extreme point of the set ${\mathcal{F}}_{\mathbf{X}}:=\{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}\}$ (i.e., the set of all cumulative distribution functions of $\mathbf{X}$). As a matter of fact [26], the extreme points of ${\mathcal{F}}_{\mathbf{X}}$ are given by the set of degenerate distributions on $\mathcal{X}$; that is, ${\{F_{\mathbf{X}}\left(\mathbf{y}\right)={\delta }_{\mathbf{x}}\left(\mathbf{y}\right),\mathbf{y}\in \mathcal{X}\}}_{\mathbf{x}\in \mathcal{X}}$. This allows us to conclude

$$\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}{\parallel \mathsf{H}\mathbf{X}+\mathbf{Z}\parallel }_p=\underset{\mathbf{x}\in \mathcal{X}}{max}{\parallel \mathsf{H}\mathbf{x}+\mathbf{Z}\parallel }_p.$$

Observe that the Euclidian norm is a convex function, which is therefore maximized at the boundary of the set $\mathsf{H}\mathcal{X}$. Combining this with Equation (9) and taking the infimum over $p>0$ completes the proof. □

The following Theorem provides two alternative upper bounds that are based on duality arguments.

Theorem 10.

(Duality Upper Bounds) For any channel input space $\mathcal{X}$ and any fixed channel matrix $\mathsf{H}$

$$C(\mathcal{X},\mathsf{H})\le {\overline{C}}_{\mathsf{Dual},1}(\mathcal{X},\mathsf{H}):={log}_2\left(c_{n_r}\left(d\right)+\frac{\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(d\right)\right)}{{\left(2\pi \mathrm{e}\right)}^{\frac{n_r}{2}}}\right),$$

(10)

where

$$d:=r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right),c_{n_r}\left(d\right):=\sum _{i=1}^{n_r-1}\left(\genfrac{}{}{0pt}{}{n_r-1}{i}\right)\frac{\mathsf{\Gamma }\left(\frac{n_r-1}{2}\right)}{2^{\frac{n_r}{2}}\mathsf{\Gamma }\left(\frac{n_r}{2}\right)}{d}^i,$$

and

$$C(\mathcal{X},\mathsf{H})\le {\overline{C}}_{\mathsf{Dual},2}(\mathcal{X},\mathsf{H}):=\sum _{i=1}^{n_r}{log}_2\left(1+\frac{2A_i}{\sqrt{2\pi \mathrm{e}}}\right),$$

(11)

where $\mathbf{a}=(A_1,\cdots ,A_{n_r})$ such that $\mathrm{Box}\left(\mathbf{a}\right)=\mathrm{Box}\left(\mathsf{H}\mathcal{X}\right)$.

Proof. 

Using duality bounds, it has been shown in [8] that for any centered n-dimensional ball of radius $r\in {\mathbb{R}}_{+}$

$$\underset{F_{\mathbf{X}}:\mathbf{X}\in {\mathcal{B}}_{\mathbf{0}}\left(r\right)}{max}I(\mathbf{X};\mathbf{X}+\mathbf{Z})\le {log}_2\left(c_n\left(r\right)+\frac{\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(r\right)\right)}{{\left(2\pi \mathrm{e}\right)}^{\frac{n}{2}}}\right),$$

(12)

where $c_n\left(r\right):={\sum }_{i=1}^{n-1}\left(\genfrac{}{}{0pt}{}{n-1}{i}\right)\frac{\mathsf{\Gamma }\left(\frac{n-1}{2}\right)}{2^{\frac{n}{2}}\mathsf{\Gamma }\left(\frac{n}{2}\right)}{r}^i$.

Now, observe that

$$\begin{array}{cc}\hfill C(\mathcal{X},\mathsf{H})& =\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}h(\mathsf{H}\mathbf{X}+\mathbf{Z})-h(\mathsf{H}\mathbf{X}+\mathbf{Z}|\mathsf{H}\mathbf{X})\hfill \\ & =\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathsf{H}\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})\hfill \\ & =\underset{F_{\tilde{\mathbf{X}}}:\tilde{\mathbf{X}}\in \mathsf{H}\mathcal{X}}{max}I(\tilde{\mathbf{X}};\tilde{\mathbf{X}}+\mathbf{Z})\hfill \\ & \stackrel{\left(a\right)}{\le }\underset{F_{\tilde{\mathbf{X}}}:\tilde{\mathbf{X}}\in {\mathcal{B}}_{\mathbf{0}}\left(d\right),d:=r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right)}{max}I(\tilde{\mathbf{X}};\tilde{\mathbf{X}}+\mathbf{Z})\hfill \\ & \stackrel{\left(b\right)}{\le }{log}_2\left(c_{n_r}\left(d\right)+\frac{\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(d\right)\right)}{{\left(2\pi \mathrm{e}\right)}^{\frac{n_r}{2}}}\right).\hfill \end{array}$$

(13)

where $\left(a\right)$ follows from enlarging the optimization domain; and $\left(b\right)$ from using the upper bound in Equation (12). This proves Equation (10).

To show the upper bound in Equation (11), we proceed with an alternative upper bound to Equation (13):

$$\begin{array}{cc}\hfill C(\mathcal{X},\mathsf{H})& =\underset{F_{\tilde{\mathbf{X}}}:\tilde{\mathbf{X}}\in \mathsf{H}\mathcal{X}}{max}I(\tilde{\mathbf{X}};\tilde{\mathbf{X}}+\mathbf{Z})\hfill \\ & \stackrel{\left(a\right)}{\le }\underset{F_{\tilde{\mathbf{X}}}:\tilde{\mathbf{X}}\in \mathrm{Box}\left(\mathsf{H}\mathcal{X}\right)}{max}I(\tilde{\mathbf{X}};\tilde{\mathbf{X}}+\mathbf{Z})\hfill \\ & \stackrel{\left(b\right)}{\le }\underset{F_{\tilde{\mathbf{X}}}:\tilde{\mathbf{X}}\in \mathrm{Box}\left(\mathsf{H}\mathcal{X}\right)}{max}\sum _{i=1}^{n_r}I({\tilde{X}}_i;{\tilde{X}}_i+Z_i)\hfill \\ & \stackrel{\left(c\right)}{=}\sum _{i=1}^{n_r}\underset{F_{{\tilde{X}}_i}:\left|{\tilde{X}}_i\right|\le A_i}{max}I({\tilde{X}}_i;{\tilde{X}}_i+Z_i)\hfill \\ & \stackrel{\left(d\right)}{\le }\sum _{i=1}^{n_r}{log}_2\left(1+\frac{2A_i}{\sqrt{2\pi \mathrm{e}}}\right),\hfill \end{array}$$

where the (in)equalities follow from: $\left(a\right)$ enlarging the optimization domain; $\left(b\right)$ single-letterizing the mutual information; $\left(c\right)$ choosing individual amplitude constraints $(A_1,\cdots ,A_{n_r})\mathbf{a}\in {\mathbb{R}}_{+}^{n_r}$ such that $\mathrm{Box}\left(\mathbf{a}\right)=\mathrm{Box}\left(\mathsf{H}\mathcal{X}\right)$; and $\left(d\right)$ using the upper bound in Equation (12) for $n=1$. This concludes the proof. □

As mentioned at the beginning of the section, another simple technique for deriving upper bounds on the capacity is to use the I-MMSE relationship [33]

$$I(\mathbf{X};\mathbf{X}+\mathbf{Z})=\frac{{log}_2\left(\mathrm{e}\right)}{2}{\int }_0^1\mathbb{E}\left[{∥\mathbf{X}-\mathbb{E}\left[\mathbf{X}\right|\sqrt{\gamma }\mathbf{X}+\mathbf{Z}]∥}^2\right]\mathrm{d}\gamma .$$

(14)

For any $\gamma \ge 0$, the quantity $\mathbb{E}\left[\parallel \mathbf{X}-\mathbb{E}\left[\mathbf{X}\right|\sqrt{\gamma }\mathbf{X}+\mathbf{Z}]{\parallel }^2\right]$ is known as the MMSE of estimating $\mathbf{X}$ from the noisy observation $\sqrt{\gamma }\mathbf{X}+\mathbf{Z}$. An important fact that will be useful is that the conditional expected value $\mathbb{E}\left[\mathbf{X}\right|\sqrt{\gamma }\mathbf{X}+\mathbf{Z}]$ is the best estimator in the sense that it minimizes the mean square error over all measurable functions $f:{\mathbb{R}}^{n_r}\to {\mathbb{R}}^{n_t}$; that is, for any $\mathbf{Y}\in {\mathbb{R}}^{n_r}$ and $\mathbf{X}\in {\mathbb{R}}^{n_t}$

$$\mathbb{E}\left[{\parallel \mathbf{X}-\mathbb{E}\left[\mathbf{X}\right|\mathbf{Y}]\parallel }^2\right]=\underset{f\mathrm{is}\mathrm{measurable}}{inf}\mathbb{E}\left[{\parallel \mathbf{X}-f\left(\mathbf{Y}\right)\parallel }^2\right].$$

(15)

Theorem 11.

(I-MMSE Upper Bound) For any channel input space $\mathcal{X}$ and any fixed channel matrix $\mathsf{H}$

$$C(\mathcal{X},\mathsf{H})\le {\overline{C}}_{\mathsf{I}-\mathsf{MMSE}}(\mathcal{X},\mathsf{H})={log}_2\left(\mathrm{e}\right)\left\{\begin{array}{cc}\frac{n_r}{2}+\frac{n_r}{2}{log}_2\left(\frac{r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)}{n_r}\right),\hfill & r_{\mathsf{max}}^2(\mathsf{H}\mathcal{X}\ge n_r\hfill \\ \frac{r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)}{2},\hfill & r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le n_r\hfill \end{array}\right..$$

Proof. 

Fix some $ϵ\in [0,1]$ and observe that

$$\begin{array}{cc}\hfill \frac{2}{{log}_2\left(\mathrm{e}\right)}I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})& \stackrel{\left(a\right)}{=}\frac{2}{{log}_2\left(\mathrm{e}\right)}I(\mathsf{H}\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})\hfill \\ & \stackrel{\left(b\right)}{=}{\int }_0^1\mathbb{E}\left[\parallel \mathsf{H}\mathbf{X}-\mathbb{E}\left[\mathsf{H}\mathbf{X}\right|\sqrt{\gamma }\mathsf{H}\mathbf{X}+\mathbf{Z}]{\parallel }^2\right]\mathrm{d}\gamma \hfill \\ & ={\int }_0^{ϵ}\mathbb{E}\left[\parallel \mathsf{H}\mathbf{X}-\mathbb{E}\left[\mathsf{H}\mathbf{X}\right|\sqrt{\gamma }\mathsf{H}\mathbf{X}+\mathbf{Z}]{\parallel }^2\right]\mathrm{d}\gamma \hfill \\ & +{\int }_{ϵ}^1\mathbb{E}\left[\parallel \mathsf{H}\mathbf{X}-\mathbb{E}\left[\mathsf{H}\mathbf{X}\right|\sqrt{\gamma }\mathsf{H}\mathbf{X}+\mathbf{Z}]{\parallel }^2\right]\mathrm{d}\gamma \hfill \\ & \stackrel{\left(c\right)}{\le }{\int }_0^{ϵ}\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}-\mathbf{0}\parallel }^2\right]\mathrm{d}\gamma \hfill \\ & +{\int }_{ϵ}^1\mathbb{E}\left[{∥\mathsf{H}\mathbf{X}-\frac{1}{\sqrt{\gamma }}(\sqrt{\gamma }\mathsf{H}\mathbf{X}+\mathbf{Z})∥}^2\right]\mathrm{d}\gamma \hfill \\ & =ϵ\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2\right]+{\int }_{ϵ}^1\frac{1}{\gamma }\mathbb{E}\left[{\parallel \mathbf{Z}\parallel }^2\right]\mathrm{d}\gamma \hfill \\ & =ϵ\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2\right]+\mathbb{E}\left[{\parallel \mathbf{Z}\parallel }^2\right]{log}_{\mathrm{e}}\left(\frac{1}{ϵ}\right)\hfill \\ & =ϵ\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2\right]+n_r{log}_{\mathrm{e}}\left(\frac{1}{ϵ}\right),\hfill \end{array}$$

where the (in)equalities follow from: $\left(a\right)$ using that $I(\mathsf{H}\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})=I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})$ for any fixed $\mathsf{H}$; $\left(b\right)$ using the I-MMSE relationship in Equation (14); and $\left(c\right)$ using the property that conditional expectation minimizes mean square error (i.e., (15)).

Now, notice that

$$\begin{array}{cc}\hfill \underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})& \le \underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}\frac{{log}_2\left(\mathrm{e}\right)}{2}\left(ϵ\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2\right]+n_r{log}_{\mathrm{e}}\left(\frac{1}{ϵ}\right)\right)\hfill \\ & \stackrel{\left(a\right)}{=}\frac{1}{2}\left(ϵ\underset{\tilde{\mathbf{x}}\in \mathcal{X}}{max}{\parallel \mathsf{H}\tilde{\mathbf{x}}\parallel }^2+n_r{log}_{\mathrm{e}}\left(\frac{1}{ϵ}\right)\right)\hfill \\ & \stackrel{\left(b\right)}{=}\frac{1}{2}\left(ϵr_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)+n_r{log}_{\mathrm{e}}\left(\frac{1}{ϵ}\right)\right),\hfill \end{array}$$

(16)

where $\left(a\right)$ follows from ${max}_{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}\mathbb{E}\left[{\parallel \mathsf{H}\mathbf{X}\parallel }^2\right]={max}_{\tilde{\mathbf{x}}\in \mathcal{X}}{\parallel \mathsf{H}\tilde{\mathbf{x}}\parallel }^2$ (the same argument was used in the proof of Theorem 9); and $\left(b\right)$ from the definition of $r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)$.

Since $ϵ$ is arbitrary, we can choose it to minimize the upper bound in Equation (16). Towards this end, we need the following optimization result

$$\underset{ϵ\in [0,1]}{min}\left(ϵa+blog\left(\frac{1}{ϵ}\right)\right)=\left\{\begin{array}{cc}b+blog\left(\frac{a}{b}\right),\hfill & a\ge b\hfill \\ a,\hfill & a\le b\hfill \end{array}\right.,$$

(17)

which is easy to show. Combining Equation (16) with Equation (17), we obtain the following upper bound on the capacity

$$\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})\le \left\{\begin{array}{cc}\frac{n_r}{2}+\frac{n_r}{2}log\left(\frac{r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)}{n_r}\right),\hfill & r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\ge n_r\hfill \\ r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right),\hfill & r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le n_r\hfill \end{array}\right..$$

This concludes the proof. □

Corollary 2.

For any channel input space $\mathcal{X}$ and any fixed channel matrix $\mathsf{H}$

$${\overline{C}}_{\mathsf{I}-\mathsf{MMSE}}(\mathcal{X},\mathsf{H})\le {log}_2\left(\mathrm{e}\right)\left\{\begin{array}{cc}\frac{n_r}{2}+\frac{n_r}{2}{log}_{\mathrm{e}}\left(\frac{{\parallel \mathsf{H}\parallel }^2{r}_{\mathsf{max}}^2\left(\mathcal{X}\right)}{n_r}\right),\hfill & {\parallel \mathsf{H}\parallel }^2{r}_{\mathsf{max}}^2\left(\mathcal{X}\right)\ge n_r\hfill \\ \frac{{\parallel \mathsf{H}\parallel }^2{r}_{\mathsf{max}}^2\left(\mathcal{X}\right)}{2},\hfill & {\parallel \mathsf{H}\parallel }^2{r}_{\mathsf{max}}^2\left(\mathcal{X}\right)\le n_r\hfill \end{array}\right..$$

Proof. 

The corollary follows by upper bounding Equation (16) using the fact that $r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)\le \parallel \mathsf{H}\parallel r_{\mathsf{max}}^2\left(\mathcal{X}\right)$. □

Remark 6.

In the proof of Theorem 11, instead of using sub-optimal estimators $f\left(\mathbf{Y}\right)=0$ and $f\left(\mathbf{Y}\right)=\frac{1}{\gamma }\mathbf{Y}$, we could have used an optimal linear estimator of the form $f\left(\mathbf{Y}\right)={\mathsf{K}}_{\mathbf{X}\mathbf{Y}}{\mathsf{K}}_{\mathbf{Y}}^{-1}\mathbf{Y}$, where ${\mathsf{K}}_{\mathbf{X}\mathbf{Y}}$ denotes the cross-covariance matrix between $\mathbf{X}$ and $\mathbf{Y}$ and ${\mathsf{K}}_{\mathbf{Y}}$ the covariance matrix of $\mathbf{Y}$. This choice would result in the capacity upper bound

$$C(\mathcal{X},\mathsf{H})\le \underset{{\mathsf{K}}_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}\frac{1}{2}{log}_2\left(det\left({\mathsf{I}}_{n_r}+\mathsf{H}{\mathsf{K}}_{\mathbf{X}}{\mathsf{H}}^T\right)\right)$$

(18)

with ${\mathsf{K}}_{\mathbf{X}}$ the covariance matrix of the channel input. While Equation (18) is a valid upper bound, as of the writing of this paper, it is not clear how to perform an optimization over covariance matrices of random variables with bounded support. One possibility to avoid this is to use the inequality between arithmetic and geometric mean and bound the determinant by the trace:

$$det\left({\mathsf{I}}_{n_r}+\mathsf{H}{\mathsf{K}}_{\mathbf{X}}{\mathsf{H}}^T\right)\le {\left(\frac{\mathrm{Tr}\left({\mathsf{I}}_{n_r}+\mathsf{H}{\mathsf{K}}_{\mathbf{X}}{\mathsf{H}}^T\right)}{n_r}\right)}^{n_r}={\left(\frac{{\parallel \mathbf{Z}+\mathsf{H}\mathbf{X}\parallel }_2^2}{n_r}\right)}^{n_r}.$$

(19)

However, combining Equation (19) with Equation (18) is merely a special case of the moment upper bound of Theorem 9 for $p=2$. Therefore, the estimators in Theorem 11 are chosen to obtain a non-trivial upper bound avoiding the optimization over covariance matrices.

In Section 5, we present a comparison of the upper bounds of Theorems 9–11 by means of a simple example.

4.2. Lower Bounds

A classical approach to bound a mutual information from below is to use the entropy power inequality (EPI).

Theorem 12.

(EPI Lower Bounds) For any fixed channel matrix $\mathsf{H}$ and any channel input space $\mathcal{X}$ with $\mathbf{X}$ absolutely continuous, we have

$$C(\mathcal{X},\mathsf{H})\ge {\underline{C}}_{\mathsf{EPI}}(\mathcal{X},\mathsf{H}):=\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}\frac{n_r}{2}{log}_2\left(1+\frac{2^{\frac{2}{n_r}h\left(\mathsf{H}\mathbf{X}\right)}}{2\pi \mathrm{e}}\right).$$

(20)

Moreover, if $n_t=n_r=n$, $\mathsf{H}\in {\mathbb{R}}^{n\times n}$ invertible, and $\mathbf{X}$ uniformly distributed over $\mathcal{X}$, then

$$C(\mathcal{X},\mathsf{H})\ge {\underline{C}}_{\mathsf{EPI}}(\mathcal{X},\mathsf{H}):=\frac{n}{2}{log}_2\left(1+\frac{{\left|\det \left(\mathsf{H}\right)\right|}^{\frac{2}{n}}\mathrm{Vol}{\left(\mathcal{X}\right)}^{\frac{2}{n}}}{2\pi \mathrm{e}}\right).$$

(21)

Proof. 

By means of the EPI

$$2^{\frac{2}{n_r}h(\mathsf{H}\mathbf{X}+\mathbf{Z})}\ge 2^{\frac{2}{n_r}h\left(\mathsf{H}\mathbf{X}\right)}+2^{\frac{2}{n_r}h\left(\mathbf{Z}\right)},$$

we conclude

$$2^{\frac{2}{n_r}C(\mathcal{X},\mathsf{H})}\ge 1+{\left(2\pi \mathrm{e}\right)}^{-1}{2}^{{\displaystyle \frac{2}{n_r}\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}h\left(\mathsf{H}\mathbf{X}\right)}},$$

which finalizes the proof of the lower bound in Equation (20).

To show the lower bound in Equation (21), all we need is to recall that

$$h\left(\mathsf{H}\mathbf{X}\right)=h\left(\mathbf{X}\right)+{log}_2\left|\det \left(\mathsf{H}\right)\right|,$$

which is maximized for $\mathbf{X}$ uniformly distributed over $\mathcal{X}$. However, if $\mathbf{X}$ is uniformly drawn from $\mathcal{X}$, we have

$$2^{\frac{2}{n}h\left(\mathsf{H}\mathbf{X}\right)}=\mathrm{Vol}{\left(\mathsf{H}\mathcal{X}\right)}^{\frac{2}{n}}={\left|\det \left(\mathsf{H}\right)\right|}^{\frac{2}{n}}\mathrm{Vol}{\left(\mathcal{X}\right)}^{\frac{2}{n}},$$

which completes the proof. □

The considerations in Section 3 suggest that a channel input distribution that maximizes Equation (2) might be discrete. Therefore, there is a need for lower bounds that unlike the bounds in Theorem 12 rely on discrete inputs.

Theorem 13.

(Ozarow–Wyner Type Lower Bound) Let ${\mathbf{X}}_D\in \mathsf{supp}\left({\mathbf{X}}_D\right)\subset {\mathbb{R}}^{n_t}$ be a discrete random vector of finite entropy, $g:{\mathbb{R}}^{n_r}\to {\mathbb{R}}^{n_t}$ a measurable function, and $p>0$. Furthermore, let ${\mathcal{K}}_p$ be a set of continuous random vectors, independent of ${\mathbf{X}}_D$, such that for every $\mathbf{U}\in {\mathcal{K}}_p$ we have $h\left(\mathbf{U}\right)<\infty$, ${\parallel \mathbf{U}\parallel }_p<\infty$, and

$$\mathsf{supp}(\mathbf{U}+{\mathbf{x}}_i)\cap \mathsf{supp}(\mathbf{U}+{\mathbf{x}}_j)=⌀$$

(22)

for all ${\mathbf{x}}_i,{\mathbf{x}}_j\in \mathsf{supp}\left({\mathbf{X}}_D\right)$, $i\ne j$. Then,

$$C(\mathcal{X},\mathsf{H})\ge {\underline{C}}_{\mathsf{OW}}(\mathcal{X},\mathsf{H}):={[H\left({\mathbf{X}}_D\right)-\mathsf{gap}]}^{+},$$

where

$$\mathsf{gap}:=\underset{\begin{array}{c}\mathbf{U}\in {\mathcal{K}}_p\\ gmeasurable\\ p>0\end{array}}{inf}\left(G_{1,p}(\mathbf{U},{\mathbf{X}}_D,g)+G_{2,p}\left(\mathbf{U}\right)\right)$$

with

$$\begin{array}{cc}\hfill G_{1,p}(\mathbf{U},{\mathbf{X}}_D,g)& :=n_t{log}_2\left(\frac{\parallel \mathbf{U}+{\mathbf{X}}_D{-g\left(\mathbf{Y}\right)\parallel }_p}{{\parallel \mathbf{U}\parallel }_p}\right),\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill G_{2,p}\left(\mathbf{U}\right)& :=n_t{log}_2\left(\frac{k_{n_t,p}{n}_t^{\frac{1}{p}}{\parallel \mathbf{U}\parallel }_p}{2^{\frac{1}{n_t}h\left(\mathbf{U}\right)}}\right),\hfill \end{array}$$

(24)

and $k_{n_t,p}$ as defined in Lemma 2, respectively.

Proof. 

The proof is identical to ([11] Theorem 2). To make the manuscript more self-contained, we repeat it here.

Let $\mathbf{U}$ and ${\mathbf{X}}_D$ be statistically independent. Then, the mutual information $I({\mathbf{X}}_D;\mathbf{Y})$ can be lower bounded as

$$\begin{array}{cc}\hfill I({\mathbf{X}}_D;\mathbf{Y})& \stackrel{\left(a\right)}{\ge }I({\mathbf{X}}_D+\mathbf{U};\mathbf{Y})\hfill \\ & =h({\mathbf{X}}_D+\mathbf{U})-h({\mathbf{X}}_D+\mathbf{U}|\mathbf{Y})\hfill \\ & \stackrel{\left(b\right)}{=}H\left({\mathbf{X}}_D\right)+h\left(\mathbf{U}\right)-h({\mathbf{X}}_D+\mathbf{U}|\mathbf{Y}).\hfill \end{array}$$

(25)

Here, $\left(a\right)$ follows from the data processing inequality as ${\mathbf{X}}_D+\mathbf{U}\to {\mathbf{X}}_D\to \mathbf{Y}$ forms a Markov chain in that order; and $\left(b\right)$ from the assumption in Equation (22). By using Lemma 2, we have that the last term in Equation (25) can be bounded from above as

$$h({\mathbf{X}}_D+\mathbf{U}|\mathbf{Y})\le n_t{log}_2\left(k_{n_t,p}{n}_t^{\frac{1}{p}}{\parallel {\mathbf{X}}_D+\mathbf{U}-g\left(\mathbf{Y}\right)\parallel }_p\right).$$

Combining this expression with Equation (25) results in

$$I({\mathbf{X}}_D;\mathbf{Y})\ge H\left({\mathbf{X}}_D\right)-\left(G_{1,p}(\mathbf{U},{\mathbf{X}}_D,g)+G_{2,p}\left(\mathbf{U}\right)\right),$$

with $G_{1,p}$ and $G_{2,p}$ as defined in Equations (23) and (24), respectively. Maximizing the right-hand side over all $\mathbf{U}\in {\mathcal{K}}_p$, measurable functions $g:{\mathbb{R}}^{n_r}\to {\mathbb{R}}^{n_t}$, and $p>0$ provides the bound. □

Interestingly, the bound of Theorem 13 holds for arbitrary channels and is therefore not restricted to MIMO channels. The interested reader is referred to [11] for details.

We conclude the section by providing a lower bound that is based on Jensen’s inequality and holds for arbitrary inputs.

Theorem 14.

(Jensen’s Inequality Lower Bound) For any channel input space $\mathcal{X}$ and any fixed channel matrix $\mathsf{H}$, we have

$$C(\mathcal{X},\mathsf{H})\ge {\underline{C}}_{\mathsf{Jensen}}(\mathcal{X},\mathsf{H}):=\underset{F_{\mathbf{X}}:\mathbf{X}\in \mathcal{X}}{max}{log}_2^{+}\left({\left(\frac{2}{\mathrm{e}}\right)}^{\frac{n_r}{2}}{\psi }^{-1}(\mathbf{X},\mathsf{H})\right)$$

(26)

with

$$\psi (\mathbf{X},\mathsf{H}):=\mathbb{E}[exp\left(-\frac{\parallel \mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}\right)]=\mathbb{E}\left[{\left|{\varphi }_{\mathbf{X}}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right)\right|}^2\right],$$

where ${\mathbf{X}}^{\prime }$ is an independent copy of $\mathbf{X}$ and ${\varphi }_{\mathbf{X}}$ denotes the characteristic function of $\mathbf{X}$.

Proof. 

To show the lower bound, we follow an approach of Dytso et al. [34]. Note that by Jensen’s inequality

$$h\left(\mathbf{Y}\right)=-\mathbb{E}\left[{log}_2{f}_{\mathbf{Y}}\left(\mathbf{Y}\right)\right]\ge -{log}_2\mathbb{E}\left[f_{\mathbf{Y}}\left(\mathbf{Y}\right)\right]=-{log}_2{\int }_{{\mathbb{R}}^{n_r}}{f}_{\mathbf{Y}}\left(\mathbf{y}\right)f_{\mathbf{Y}}\left(\mathbf{y}\right)\mathrm{d}\mathbf{y}.$$

(27)

Now, evaluating the integral in Equation (27) results in

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^{n_r}}{f}_{\mathbf{Y}}\left(\mathbf{y}\right)f_{\mathbf{Y}}\left(\mathbf{y}\right)\mathrm{d}\mathbf{y}& =\frac{1}{{\left(2\pi \right)}^{n_r}}{\int }_{{\mathbb{R}}^{n_r}}\mathbb{E}\left[{\mathrm{e}}^{-\frac{{\parallel \mathbf{y}-\mathsf{H}\mathbf{X}\parallel }^2}{2}}\right]\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathbf{y}-\mathsf{H}{\mathbf{X}}^{\prime }{\parallel }^2}{2}}\right]\mathrm{d}\mathbf{y}\hfill \\ & \stackrel{\left(a\right)}{=}\frac{1}{{\left(2\pi \right)}^{n_r}}\mathbb{E}\left[{\int }_{{\mathbb{R}}^{n_r}}{\mathrm{e}}^{-\frac{{\parallel \mathbf{y}-\mathsf{H}\mathbf{X}\parallel }^2+{\parallel \mathbf{y}-\mathsf{H}{\mathbf{X}}^{\prime }\parallel }^2}{2}}\mathrm{d}\mathbf{y}\right]\hfill \\ & \stackrel{\left(b\right)}{=}\frac{1}{{\left(2\pi \right)}^{n_r}}\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{H}\mathbf{X}-\mathsf{H}{\mathbf{X}}^{\prime }{\parallel }^2}{4}}{\int }_{{\mathbb{R}}^{n_r}}{\mathrm{e}}^{-\parallel \mathbf{y}-\frac{\mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime })}{2}{\parallel }^2}\mathrm{d}\mathbf{y}\right]\hfill \\ & \stackrel{\left(c\right)}{=}\frac{1}{2^{n_r}{\pi }^{\frac{n_r}{2}}}\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right],\hfill \end{array}$$

(28)

where $\left(a\right)$ follows from the independence of $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ and Tonelli’s Theorem ([35] Chapter 5.9); $\left(b\right)$ from completing a square; and $\left(c\right)$ from the fact that ${\int }_{{\mathbb{R}}^{n_r}}{\mathrm{e}}^{-\parallel \mathbf{y}-\frac{\mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime })}{2}{\parallel }^2}\mathrm{d}\mathbf{y}={\int }_{{\mathbb{R}}^{n_r}}{\mathrm{e}}^{-{\parallel \mathbf{y}\parallel }^2}\mathrm{d}\mathbf{y}={\pi }^{\frac{n_r}{2}}$. Combining Equation (27) with Equation (28) and subtracting $h\left(\mathbf{Z}\right)=\frac{n_r}{2}{log}_2\left(2\pi \mathrm{e}\right)$ completes the proof of the first version of the bound.

To show the second version, observe that

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right]& \stackrel{\left(d\right)}{=}\mathbb{E}\left[{\varphi }_{\frac{\mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime })}{\sqrt{2}}}\left(\mathbf{Z}\right)\right]\hfill \\ & \stackrel{\left(e\right)}{=}\mathbb{E}\left[{\varphi }_{\frac{\mathsf{H}\mathbf{X}}{\sqrt{2}}}\left(\mathbf{Z}\right){\varphi }_{-\frac{\mathsf{H}{\mathbf{X}}^{\prime }}{\sqrt{2}}}\left(\mathbf{Z}\right)\right]\hfill \\ & \stackrel{\left(f\right)}{=}\mathbb{E}\left[{\varphi }_{\mathbf{X}}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right){\varphi }_{{\mathbf{X}}^{\prime }}\left(-\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right)\right]\hfill \\ & \stackrel{\left(g\right)}{=}\mathbb{E}\left[{\varphi }_{\mathbf{X}}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right){\varphi }_{\mathbf{X}}\left(-\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right)\right]\hfill \\ & \stackrel{\left(h\right)}{=}\mathbb{E}\left[{\varphi }_{\mathbf{X}}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right){\varphi }_{\mathbf{X}}^{*}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right)\right]\hfill \\ & =\mathbb{E}\left[{\left|{\varphi }_{\mathbf{X}}\left(\frac{{\mathsf{H}}^T\mathbf{Z}}{\sqrt{2}}\right)\right|}^2\right],\hfill \end{array}$$

where $\left(d\right)$ follows from Parseval’s identity ([35] Chapter 9.5) by noting that $\exp (-\parallel ·{\parallel }^2/2)$ is a characteristic function of $\mathbf{Z}$ and ${\varphi }_{\frac{\mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime })}{\sqrt{2}}}(·)$ is a characteristic function of $\frac{\mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime })}{\sqrt{2}}$; $\left(e\right)$ from using the property that the characteristic function of a sum of random vectors is equal to the product of its characteristic functions; $\left(f\right)$ from using the fact that a characteristic function is a linear transformation; $\left(g\right)$ from using that $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ have the same characteristic function; and $\left(h\right)$ from the fact that the characteristic function is Hermitian. This completes the proof. □

Remark 7.

As is evident from our examples in the following sections, in many cases, the Jensen’s inequality lower bound of Theorem 14 performs remarkably well. The bound, however, is also useful for MIMO channels that are subject to an average power constraint. For example, evaluating Equation (26) with $\mathbf{X}\sim \mathcal{N}(\mathbf{0},{\mathsf{K}}_{\mathbf{X}})$ results in

$$I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})\ge \frac{1}{2}{log}_2^{+}\left({\left(\frac{2}{\mathrm{e}}\right)}^{min(n_r,n_t)}det\left({\mathsf{I}}_{n_r}+\mathsf{H}{\mathsf{K}}_{\mathbf{X}}{\mathsf{H}}^T\right)\right).$$

Note that this bound is within $\frac{min(n_r,n_t)}{2}{log}_2\left(\frac{2}{\mathrm{e}}\right)$ bits of the capacity of the power-constrained channel.

In Section 3, we discuss that the distributions that maximize mutual information in $n_t$-dimensions are typically singular, which means that they are concentrated on a set of Lebesgue measure zero. Singular distributions generally do not have a probability density, whereas the characteristic function always exists. This is why the version of Jensen’s inequality lower bound in Theorem 14 that is based on the characteristic function of the channel input is especially useful for amplitude-constrained MIMO channels.

5. Invertible Channel Matrices

Consider the symmetric case of $n_t=n_r=n$ antennas with $\mathsf{H}\in {\mathbb{R}}^{n\times n}$ being invertible. In this section, we evaluate some of the lower and upper bounds proposed in the previous section for the special case of $\mathsf{H}$ being also diagonal and then characterize the gap to the capacity for arbitrary invertible channel matrices.

5.1. Diagonal Channel Matrices

Suppose the channel inputs are subject to per-antenna or an n-dimensional amplitude constraint. Then, the duality upper bound ${\overline{C}}_{\mathsf{Dual},2}(\mathcal{X},\mathsf{H})$ of Theorem 10 takes on the following form.

Theorem 15.

(Upper Bounds) Let $\mathsf{H}=\mathrm{diag}(h_{11},\cdots ,h_{nn})\in {\mathbb{R}}^{n\times n}$ be fixed. If $\mathcal{X}=\mathrm{Box}\left(\mathbf{a}\right)$ for some $\mathbf{a}=(A_1,\cdots ,A_n)\in {\mathbb{R}}_{+}^n$, then

$${\overline{C}}_{\mathsf{Dual},2}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})=\sum _{i=1}^n{log}_2\left(1+\frac{2|h_{ii}|A_i}{\sqrt{2\pi \mathrm{e}}}\right).$$

(29)

Moreover, if $\mathcal{X}={\mathcal{B}}_{\mathbf{0}}\left(A\right)$ for some $A\in {\mathbb{R}}_{+}$, then

$${\overline{C}}_{\mathsf{Dual},2}({\mathcal{B}}_{\mathbf{0}}\left(A\right),\mathsf{H})=\sum _{i=1}^n{log}_2\left(1+\frac{2|h_{ii}|A}{\sqrt{n}\sqrt{2\pi \mathrm{e}}}\right).$$

(30)

Proof. 

The bound in Equation (29) immediately follows from Theorem 10 by observing that $\mathrm{Box}\left(\mathsf{H}\mathrm{Box}\right(\mathbf{a}\left)\right)=\mathrm{Box}\left(\mathsf{H}\mathbf{a}\right)$. The bound in Equation (30) follows from Theorem 10 by the fact that

$$\mathrm{Box}\left(\mathsf{H}{\mathcal{B}}_{\mathbf{0}}\left(A\right)\right)\subset \mathrm{Box}\left(\mathsf{H}\mathrm{Box}\left({\mathcal{B}}_{\mathbf{0}}\left(A\right)\right)\right)=\mathrm{Box}\left(\mathbf{h}\right),$$

where $\mathbf{h}:=\frac{A}{\sqrt{n}}\left(\right|h_{11}|,\cdots ,|h_{nn}\left|\right)$. This concludes the proof. □

For an arbitrary channel input space $\mathcal{X}$, the EPI lower bound of Theorem 12 and Jensen’s inequality lower bound of Theorem 14 take on the following form.

Theorem 16.

(Lower Bounds) Let $\mathsf{H}=\mathrm{diag}(h_{11},\cdots ,h_{nn})\in {\mathbb{R}}^{n\times n}$ be fixed and $\mathcal{X}$ arbitrary. Then,

$${\underline{C}}_{\mathsf{Jensen}}(\mathcal{X},\mathsf{H})={log}_2^{+}\left({\left(\frac{2}{\mathrm{e}}\right)}^{\frac{n}{2}}\frac{1}{\psi (\mathsf{H},{\mathbf{b}}^{\star })}\right)$$

(31)

with

$$\psi (\mathsf{H},{\mathbf{b}}^{\star }):=\underset{\mathbf{b}\in \mathcal{X}}{min}\prod _{i=1}^n\phi \left(\right|h_{ii}\left|B_i\right),$$

where $\mathbf{b}:=(B_1,\cdots ,B_n)$ and $\phi :{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$,

$$\phi \left(x\right):=\frac{1}{x^2}\left({\mathrm{e}}^{-x^2}-1+\sqrt{\pi }x\left(1-2Q\left(\sqrt{2}x\right)\right)\right).$$

(32)

Moreover,

$${\underline{C}}_{\mathsf{EPI}}(\mathcal{X},\mathsf{H})=\frac{n}{2}{log}_2\left(1+\mathrm{Vol}{\left(\mathcal{X}\right)}^{\frac{2}{n}}\frac{{\left|{\prod }_{i=1}^n{h}_{ii}\right|}^{\frac{2}{n}}}{2\pi \mathrm{e}}\right).$$

(33)

Proof. 

For some given values $B_i\in {\mathbb{R}}_{+}$, $i=1,\cdots ,n$, let the ith component of $\mathbf{X}=(X_1,\cdots ,X_n)$ be independent and uniformly distributed over the interval $[-B_i,B_i]$. Thus, the expected value appearing in the bound of Theorem 14 can be written as

$$\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right]=\mathbb{E}\left[{\mathrm{e}}^{-\frac{{\sum }_{i=1}^n{h}_{ii}^2{(X_i-X_i^{\prime })}^2}{4}}\right]=\mathbb{E}\left[\prod _{i=1}^n{\mathrm{e}}^{-\frac{h_{ii}^2{(X_i-X_i^{\prime })}^2}{4}}\right]=\prod _{i=1}^n\mathbb{E}\left[{\mathrm{e}}^{-\frac{h_{ii}^2{(X_i-X_i^{\prime })}^2}{4}}\right].$$

(34)

Now, if ${\mathbf{X}}^{\prime }$ is an independent copy of $\mathbf{X}$, it can be shown that the expected value at the right-hand side of Equation (34) is of the explicit form

$$\mathbb{E}\left[{\mathrm{e}}^{-\frac{h_{ii}^2{(x_i-x_i^{\prime })}^2}{4}}\right]=\phi \left(\right|h_{ii}\left|B_i\right)$$

with $\phi$ as defined in Equation (32). Finally, optimizing over all $\mathbf{b}=(B_1,\cdots ,B_n)\in \mathcal{X}$ results in the bound (31). The bound in Equation (33) follows by inserting $\left|\det \left(\mathsf{H}\right)\right|=\left|{\prod }_{i=1}^n{h}_{ii}\right|$ into Equation (21), which concludes the proof. □

In Figure 2, the upper bounds of Theorems 9 and 15 and the lower bounds of Theorem 16 are depicted for a diagonal $2\times 2$ MIMO channel with per-antenna amplitude constraints. It turns out that the moment upper bound and the EPI lower bound perform well in the small amplitude regime while the duality upper bound and Jensen’s inequality lower bound perform well in the high amplitude regime. Interestingly, for this specific example, the duality upper bound and Jensen’s lower bound are asymptotically tight.

5.2. Gap to the Capacity

Our first result provides and upper bound to the gap between the capacity in Equation (2) and the lower bound in Equation (21).

Theorem 17.

Let $\mathsf{H}\in {\mathbb{R}}^{n\times n}$ be of full rank and

$$\rho (\mathcal{X},\mathsf{H}):=\frac{\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right)\right)\right)}{\mathrm{Vol}\left(\mathsf{H}\mathcal{X}\right)}.$$

Then,

$$C(\mathcal{X},\mathsf{H})-{\underline{C}}_{\mathsf{EPI}}(\mathcal{X},\mathsf{H})\le \frac{n}{2}{log}_2\left({\left(\pi n\right)}^{\frac{1}{n}}\rho {(\mathcal{X},\mathsf{H})}^{\frac{2}{n}}\right).$$

Proof. 

For notational convenience, let the volume of an n-dimensional ball of radius $r>0$ be denoted as

$$V_n\left(r\right):=\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(r\right)\right)=V_n\left(1\right)r^n=\frac{{\pi }^{\frac{n}{2}}{r}^n}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}.$$

Now, observe that, by choosing $p=2$, the upper bound of Theorem 9 can further be upper bounded as

$$\begin{array}{cc}\hfill {\overline{C}}_{\mathsf{M}}(\mathcal{X},\mathsf{H})& \le n{log}_2\left(\frac{k_{n,2}}{{\left(2\pi \mathrm{e}\right)}^{\frac{1}{2}}}{n}^{\frac{1}{2}}{\parallel \tilde{\mathbf{x}}+\mathbf{Z}\parallel }_2\right)\hfill \\ & \stackrel{\left(a\right)}{=}\frac{n}{2}{log}_2\left(\frac{1}{n}\mathbb{E}\left[\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }^2\right]\right)\hfill \\ & \stackrel{\left(b\right)}{=}\frac{n}{2}{log}_2\left(1+\frac{1}{n}{\parallel \tilde{\mathbf{x}}\parallel }^2\right),\hfill \end{array}$$

where $\left(a\right)$ follows since $k_{n,2}=\sqrt{\frac{2\pi \mathrm{e}}{n}}$; and $\left(b\right)$ since $\mathbb{E}[\parallel \mathbf{Z}{\parallel }^2]=n$. Therefore, the gap between Equation (21) and the moment upper bound of Theorem 9 can be upper bounded as follows:

$$\begin{array}{cc}\hfill {\overline{C}}_{\mathsf{M}}(\mathcal{X},\mathsf{H})-{\underline{C}}_{\mathsf{EPI}}(\mathcal{X},\mathsf{H})& =\frac{n}{2}{log}_2\left(\frac{1+\frac{1}{n}{\parallel \tilde{\mathbf{x}}\parallel }^2}{1+\frac{\mathrm{Vol}{\left(\mathsf{H}\mathcal{X}\right)}^{\frac{2}{n}}}{2\pi \mathrm{e}}}\right)\hfill \\ & \stackrel{a)}{=}\frac{n}{2}{log}_2\left(\frac{1+\frac{1}{n}{\left(\frac{V_n(\parallel \tilde{\mathbf{x}}\parallel )}{V_n\left(1\right)}\right)}^{\frac{2}{n}}}{1+\frac{\mathrm{Vol}{\left(\mathsf{H}\mathcal{X}\right)}^{\frac{2}{n}}}{2\pi \mathrm{e}}}\right)\hfill \\ & =\frac{n}{2}{log}_2\left(\frac{1+\frac{1}{n}{\left(\frac{\rho (\mathcal{X},\mathsf{H})\mathrm{Vol}\left(\mathsf{H}\mathcal{X}\right)}{V_n\left(1\right)}\right)}^{\frac{2}{n}}}{1+\frac{\mathrm{Vol}{\left(\mathsf{H}\mathcal{X}\right)}^{\frac{2}{n}}}{2\pi \mathrm{e}}}\right)\hfill \\ & \stackrel{b)}{\le }\frac{n}{2}{log}_2\left(\frac{1}{n}2\pi \mathrm{e}{\left(\frac{\rho (\mathcal{X},\mathsf{H})}{V_n\left(1\right)}\right)}^{\frac{2}{n}}\right)\hfill \\ & \stackrel{c)}{\le }\frac{n}{2}{log}_2\left({\left(\pi n\right)}^{\frac{1}{n}}\rho {(\mathcal{X},\mathsf{H})}^{\frac{2}{n}}\right).\hfill \end{array}$$

where $\left(a\right)$ is due to the fact that $\parallel \tilde{\mathbf{x}}\parallel$ is the radius of an n-dimensional ball; $\left(b\right)$ follows from the inequality $\frac{1+cx}{1+x}\le c$ for $c\ge 1$ and $x\in {\mathbb{R}}_{+}$; and $\left(c\right)$ follows from using Stirling’s approximation to obtain ${\left(\frac{1}{V_n\left(1\right)}\right)}^{\frac{2}{n}}\le \frac{1}{2\mathrm{e}{\pi }^{1-\frac{1}{n}}}{n}^{1+\frac{1}{n}}$. □

The term $\rho (\mathcal{X},\mathsf{H})$ is referred to as the packing efficiency of the set $\mathsf{H}\mathcal{X}$. In the following proposition, we present the packing efficiencies for important special cases.

Proposition 1.

(Packing Efficiencies) Let $\mathsf{H}\in {\mathbb{R}}^{n\times n}$ be of full rank, $A\in {\mathbb{R}}_{+}$, and $\mathbf{a}:=(A_1,\cdots ,A_n)\in {\mathbb{R}}_{+}^n$. Then,

$$\begin{array}{cc}\hfill \rho \left({\mathcal{B}}_{\mathbf{0}}\left(A\right),{\mathsf{I}}_n\right)& =1,\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill \rho \left({\mathcal{B}}_{\mathbf{0}}\left(A\right),\mathsf{H}\right)& =\frac{{\parallel \mathsf{H}\parallel }^n}{\left|\det \right(\mathsf{H}\left)\right|},\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill \rho \left(\mathrm{Box}\left(\mathbf{a}\right),{\mathsf{I}}_n\right)& =\frac{{\pi }^{\frac{n}{2}}}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}\frac{{\parallel \mathbf{a}\parallel }^n}{{\prod }_{i=1}^n{A}_i},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill \rho \left(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H}\right)& \le \frac{{\pi }^{\frac{n}{2}}}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}\frac{{\parallel \mathsf{H}\parallel }^n{\parallel \mathbf{a}\parallel }^n}{\left|\det \left(\mathsf{H}\right)\right|{\prod }_{i=1}^n{A}_i}.\hfill \end{array}$$

(38)

Proof. 

The packing efficiency in Equation (35) follows immediately. Note that

$$r_{\mathsf{max}}\left(\mathsf{H}{\mathcal{B}}_{\mathbf{0}}\left(A\right)\right)=\underset{\mathbf{x}\in {\mathcal{B}}_{\mathbf{0}}\left(A\right)}{max}\parallel \mathsf{H}\mathbf{x}\parallel =\parallel \mathsf{H}\parallel A.$$

Thus, as $\mathsf{H}$ is assumed to be invertible, we have $\mathrm{Vol}\left(\mathsf{H}{\mathcal{B}}_{\mathbf{0}}\left(A\right)\right)=\left|\det \left(\mathsf{H}\right)\right|\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(A\right)\right)$, which results in Equation (36). To show Equation (37), observe that

$$\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left({\mathsf{I}}_n\mathrm{Box}\left(\mathbf{a}\right)\right)\right)\right)=\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}(\parallel \mathbf{a}\parallel )\right)=\frac{{\pi }^{\frac{n}{2}}}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}{\parallel \mathbf{a}\parallel }^n.$$

The proof of Equation (37) is concluded by observing that $\mathrm{Vol}\left({\mathsf{I}}_n\mathrm{Box}\left(\mathbf{a}\right)\right)={\prod }_{i=1}^n{A}_i$. Finally, observe that $\mathrm{Box}\left(\mathbf{a}\right)\subset {\mathcal{B}}_{\mathbf{0}}(\parallel \mathbf{a}\parallel )$ implies $r_{\mathsf{max}}\left(\mathsf{H}\mathrm{Box}\left(\mathbf{a}\right)\right)\le r_{\mathsf{max}}(\mathsf{H}{\mathcal{B}}_{\mathbf{0}}(\parallel \mathbf{a}\parallel ))$ so that

$$\rho \left(\mathsf{H},\mathrm{Box}\left(\mathbf{a}\right)\right)\le \frac{\mathrm{Vol}\left({\mathcal{B}}_{\mathbf{0}}(\parallel \mathsf{H}\parallel \parallel \mathbf{a}\parallel )\right)}{\mathrm{Vol}\left(\mathsf{H}\mathrm{Box}\left(\mathbf{a}\right)\right)}=\frac{{\pi }^{\frac{n}{2}}}{\mathsf{\Gamma }\left(\frac{n}{2}+1\right)}\frac{{\parallel \mathsf{H}\parallel }^n{\parallel \mathbf{a}\parallel }^n}{\left|\det \left(\mathsf{H}\right)\right|{\prod }_{i=1}^n{A}_i},$$

which is the bound in Equation (38). □

We conclude this section by characterizing the gap to the capacity when $\mathsf{H}$ is diagonal and the channel input space is the Cartesian product of n PAM constellations. In this context, $\mathsf{PAM}(N,A)$ refers to the set of $N\in \mathbb{N}$ equidistant PAM-constellation points with amplitude constraint $A\in {\mathbb{R}}_{+}$ (see Figure 3 for an illustration), whereas $X\sim \mathsf{PAM}(N,A)$ means that X is uniformly distributed over $\mathsf{PAM}(N,A)$ [11].

Theorem 18.

Let $\mathsf{H}=\mathrm{diag}(h_{11},\cdots ,h_{nn})\in {\mathbb{R}}^{n\times n}$ be fixed and $\mathbf{X}=(X_1,\cdots ,X_n)$. Then, if $X_i\sim \mathsf{PAM}(N_i,A_i)$, $i=1,\cdots ,n$, for some given $\mathbf{a}=(A_1,\cdots ,A_n)\in {\mathbb{R}}_{+}^n$, it holds that

$${\overline{C}}_{\mathsf{Dual},2}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})-{\underline{C}}_{\mathsf{OW}}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})\le c·nbits,$$

(39)

where $N_i:=⌊1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}⌋$ and

$$c:=1+\frac{1}{2}{log}_2\left(\frac{\pi \mathrm{e}}{6}\right)+\frac{1}{2}{log}_2\left(1+\frac{6}{\pi \mathrm{e}}\right)\approx 1.64.$$

Moreover, if $X_i\sim \mathsf{PAM}(N_i,A)$, $i=1,\cdots ,n$, for some given $A\in {\mathbb{R}}_{+}$, it holds that

$${\overline{C}}_{\mathsf{Dual},2}({\mathcal{B}}_{\mathbf{0}}\left(A\right),\mathsf{H})-{\underline{C}}_{\mathsf{OW}}({\mathcal{B}}_{\mathbf{0}}\left(A\right),\mathsf{H})\le c·nbits,$$

(40)

where $N_i:=⌊1+\frac{2A|h_{ii}|}{\sqrt{n}\sqrt{2\pi \mathrm{e}}}⌋$.

Proof. 

Since the channel matrix is diagonal, letting the channel input $\mathbf{X}$ be such that its elements $X_i$, $i=1,\cdots ,n$, are independent, we have that

$$I(\mathbf{X};\mathsf{H}\mathbf{X}+\mathbf{Z})=\sum _{i=1}^{n}I(X_i;h_{ii}{X}_i+Z_i).$$

Let $X_i\sim \mathsf{PAM}(N_i,A_i)$ with $N_i:=⌊1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}⌋$ and observe that half the Euclidean distance between any pair of adjacent points in $\mathsf{PAM}(N_i,A_i)$ is equal to ${\mathsf{\Delta }}_i:=A_i/(N_i-1)$ (see Figure 3), $i=1,\cdots ,n$. To lower bound the mutual information $I(X_i;h_{ii}{X}_i+Z_i)$, we use the bound of Theorem 13 for $p=2$ and $n_t=1$. Thus, for some continuous random variable U that is uniformly distributed over the interval $[-{\mathsf{\Delta }}_i,{\mathsf{\Delta }}_i)$ and independent of $X_i$, we have that

$$I(X_i;h_{ii}{X}_i+Z_i)\ge H\left(X_i\right)-\frac{1}{2}{log}_2\left(\frac{\pi \mathrm{e}}{6}\right)-\frac{1}{2}{log}_2\left(\frac{\mathbb{E}\left[{(U+X_i-g\left(Y_i\right))}^2\right]}{\mathbb{E}\left[U^2\right]}\right).$$

(41)

Now, note that the entropy term in Equation (41) can be lower bounded as

$$H\left(X_i\right)={log}_2\left(⌊1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}⌋\right)\ge {log}_2\left(1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}\right)+{log}_2\left(2\right),$$

(42)

where we have used that $\lfloor x\rfloor \ge \frac{x}{2}$ for every $x\ge 1$. On the other hand, the last term in Equation (41) can be upper bounded by upper bounding its argument as follows:

$$\begin{array}{cc}\hfill \frac{\mathbb{E}\left[{(U+X_i-g\left(Y_i\right))}^2\right]}{\mathbb{E}\left[U^2\right]}& \stackrel{a)}{=}1+\frac{3\mathbb{E}\left[{(X_i-g\left(Y_i\right))}^2\right]}{{\mathsf{\Delta }}_i^2}\hfill \\ & \stackrel{b)}{\le }1+\frac{3\mathbb{E}\left[Z_i^2\right]{(N_i-1)}^2}{A_i^2{\left|h_{ii}\right|}^2}\hfill \\ & =1+\frac{3{(N_i-1)}^2}{A_i^2{\left|h_{ii}\right|}^2}\hfill \\ & \stackrel{c)}{\le }1+\frac{3{\left(\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}\right)}^2}{A_i^2{\left|h_{ii}\right|}^2}\hfill \\ & =1+\frac{6}{\pi \mathrm{e}}.\hfill \end{array}$$

(43)

where $\left(a\right)$ follows from using that $X_i$ and U are independent and $\mathbb{E}\left[U^2\right]=\frac{{\mathsf{\Delta }}_i^2}{3}$; $\left(b\right)$ from using the estimator $g\left(Y_i\right)=\frac{1}{h_{ii}}{Y}_i$; and $\left(c\right)$ from $N_i=⌊1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}⌋\le 1+\frac{2A_i\left|h_{ii}\right|}{\sqrt{2\pi \mathrm{e}}}$. Combining Equations (41), (42), and (43) results in the gap in (39).

The proof of the capacity gap in Equation (40) follows along similar lines, which concludes the proof. □

We are also able to determine the gap to the capacity for a general invertible channel matrix.

Theorem 19.

For any $\mathcal{X}$ and any invertible $\mathsf{H}$

$$C(\mathcal{X},\mathsf{H})-{\underline{C}}_{\mathsf{OW}}(\mathcal{X},\mathsf{H})\le {log}_2\left(\pi n\right)+\frac{n}{2}{log}_2\left(1+4n(4+4n)\left(\frac{\parallel {\mathsf{H}}^{-1}{\parallel }^2{r}_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}+\frac{n\parallel {\mathsf{H}}^{-1}{\parallel }^2}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}\right)\right).$$

Proof. 

Let $\mathbf{X}$ be uniformly distributed over a set constructed from an n-dimensional cubic lattice with the number of points equal to $N=⌊\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2^n⌋$, where $\tilde{\mathbf{x}}\in \mathsf{H}\mathcal{X}$ is chosen such that $\parallel \tilde{\mathbf{x}}\parallel =r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right)$, and scaled such that it is contained in the input space $\mathcal{X}$. Note that the minimum distance between point in $\mathbf{X}$ are given by

$$d_{\mathsf{min}}\left(\mathsf{supp}\left(\mathbf{X}\right)\right):=\frac{r_{\mathsf{min}}\left(\mathcal{X}\right)}{N^{\frac{1}{n}}}.$$

Now, we compute the difference between the moment upper bound of Theorem 9 and the Ozarow–Wyner lower bound of Theorem 13:

$$\begin{array}{cc}\hfill {\overline{C}}_{\mathsf{M}}(\mathcal{X},\mathsf{H})-{\underline{C}}_{\mathsf{OW}}(\mathcal{X},\mathsf{H})& \stackrel{\left(a\right)}{\le }{log}_2\left(\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2\right)-H\left(\mathbf{X}\right)+\mathsf{gap}\hfill \\ & =n{log}_2\left(\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2\right)-{log}_2\left(\lfloor \parallel \tilde{\mathbf{x}}+\mathbf{Z}{\parallel }_2^n\rfloor \right)+\mathsf{gap}\hfill \\ & \stackrel{\left(b\right)}{\le }{log}_2\left(2\right)+\mathsf{gap},\hfill \end{array}$$

(44)

where $\left(a\right)$ follows from Theorem 9 by choosing $p=2$; and $\left(b\right)$ by using the bound $\lfloor x\rfloor \ge \frac{x}{2}$ for $x>1$. The next step in the proof consists in bounding the $\mathsf{gap}$ term, which requires to upper bound the terms in Equations (23) and (24) individually. Towards this end, choose $p=2$ and let $\mathbf{U}$ be a random vector that is uniformly distributed over a ball of radius $d_{\mathsf{min}}\left(\mathbf{X}\right)$. Thus, for (23) it follows

$$\begin{array}{cc}\hfill G_{1,2}(\mathbf{U},\mathbf{X},g)& =n{log}_2\left(\frac{{\parallel \mathbf{U}+\mathbf{X}-g\left(\mathbf{Y}\right)\parallel }_2}{{\parallel \mathbf{U}\parallel }_2}\right)\hfill \\ & \stackrel{\left(a\right)}{=}n{log}_2\left(\frac{\parallel \mathbf{U}-{\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2}{{\parallel \mathbf{U}\parallel }_2}\right)\hfill \\ & =\frac{n}{2}{log}_2\left(1+\frac{\parallel {\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2^2}{{\parallel \mathbf{U}\parallel }_2^2}\right)\hfill \\ & \stackrel{\left(b\right)}{=}\frac{n}{2}{log}_2\left(1+\frac{4(4+4n)\parallel {\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2^2}{d_{\mathsf{min}}^2\left(\mathsf{supp}\left(\mathbf{X}\right)\right)}\right)\hfill \\ & \stackrel{\left(c\right)}{\le }\frac{n}{2}{log}_2\left(1+\frac{(4+4n)\parallel {\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2^2{\parallel \tilde{\mathbf{x}}+\mathbf{Z}\parallel }_2^2}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}\right)\hfill \\ & \stackrel{\left(d\right)}{=}\frac{n}{2}{log}_2\left(1+\frac{(4+4n)\parallel {\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2^2\left(r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)+n\right)}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}\right)\hfill \\ & \stackrel{\left(e\right)}{\le }\frac{n}{2}{log}_2\left(1+\frac{4(4+4n)\parallel {\mathsf{H}}^{-1}{\parallel }^2{\parallel \mathbf{Z}\parallel }_2^2\left(r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)+n\right)}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}\right)\hfill \\ & =\frac{n}{2}{log}_2\left(1+\frac{4(4+4n)\parallel {\mathsf{H}}^{-1}{\parallel }^{2}n\left(r_{\mathsf{max}}^2\left(\mathsf{H}\mathcal{X}\right)+n\right)}{r_{\mathsf{min}}^2\left(\mathcal{X}\right)}\right).\hfill \end{array}$$

where $\left(a\right)$ follows by choosing $g\left(\mathbf{Y}\right)={\mathsf{H}}^{-1}\mathbf{Y}$: $\left(b\right)$ by using ${\parallel \mathbf{U}\parallel }_2^2=\frac{r^2}{4+2n}$, where $r=\frac{d_{\mathsf{min}}\left(\mathsf{supp}\left(\mathbf{X}\right)\right)}{2}$ is the radius of an n-dimensional ball; $\left(c\right)$ from dropping the floor function in the expression for the minimum distance, i.e.,

$$d_{\mathsf{min}}^{-1}\left(\mathsf{supp}\left(\mathbf{X}\right)\right)=\frac{N^{\frac{1}{n}}}{r_{\mathsf{min}}\left(\mathcal{X}\right)}=\frac{{⌊\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2^n⌋}^{\frac{1}{n}}}{r_{\mathsf{min}}\left(\mathcal{X}\right)}\le \frac{\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2}{r_{\mathsf{min}}\left(\mathcal{X}\right)};$$

$\left(d\right)$ follows by expanding $\parallel \tilde{\mathbf{x}}{+\mathbf{Z}\parallel }_2^2$ using that $\parallel \tilde{\mathbf{x}}\parallel =r_{\mathsf{max}}\left(\mathsf{H}\mathcal{X}\right)$; and $\left(e\right)$ from using the bound $\parallel {\mathsf{H}}^{-1}{\mathbf{Z}\parallel }_2\le \parallel {\mathsf{H}}^{-1}{\parallel \parallel \mathbf{Z}\parallel }_2$.

On the other hand, the term $G_{2,p}\left(\mathbf{U}\right)$ can be bounded from above as follows ([36] Appendix L):

$$G_{2,p}\left(\mathbf{U}\right)=n{log}_2\left(\frac{k_{n,p}{n}^{\frac{1}{p}}{\parallel \mathbf{U}\parallel }_p}{2^{\frac{1}{n}h\left(\mathbf{U}\right)}}\right)\le n{log}_2\left({\left(\pi n\right)}^{\frac{1}{n}}\right).$$

Combining these two bounds with the one in (44) provides the result. □

6. Arbitrary Channel Matrices

For an arbitrary MIMO channel with an average power constraint, it is well known that the capacity is achieved by a singular value decomposition (SVD) of the channel matrix (i.e., $\mathsf{H}=\mathsf{U}\mathsf{\Lambda }{\mathsf{V}}^T$) along with considering the equivalent channel model

$$\tilde{\mathbf{Y}}=\mathsf{\Lambda }\tilde{\mathbf{X}}+\tilde{\mathbf{Z}},$$

where $\tilde{\mathbf{Y}}:={\mathsf{U}}^T\mathbf{Y}$, $\tilde{\mathbf{X}}:={\mathsf{V}}^T\mathbf{X}$, and $\tilde{\mathbf{Z}}:={\mathsf{U}}^T\mathbf{Z}$, respectively.

To provide lower bounds for channels with amplitude constraints and SVD precoding, we need the following lemma.

Lemma 3.

For any given orthogonal matrix $\mathsf{V}\in {\mathbb{R}}^{n_t\times n_t}$ and constraint vector $\mathbf{a}=(A_1,\cdots ,A_{n_t})\in {\mathbb{R}}_{+}^{n_t}$, there exists a distribution $F_{\mathbf{X}}$ of $\mathbf{X}$ such that $\tilde{\mathbf{X}}={\mathsf{V}}^T\mathbf{X}$ is uniformly distributed over $\mathrm{Box}\left(\mathbf{a}\right)$. Moreover, the components ${\tilde{X}}_1,\cdots ,{\tilde{X}}_{n_t}$ of $\tilde{\mathbf{X}}$ are mutually independent with ${\tilde{X}}_i$ uniformly distributed over $[-A_i,A_i]$, $i=1,\cdots ,n_t$.

Proof. 

Suppose that $\tilde{\mathbf{X}}$ is uniformly distributed over $\mathrm{Box}\left(\mathbf{a}\right)$; that is, the density of $\tilde{\mathbf{X}}$ is of the form

$$f_{\tilde{\mathbf{X}}}\left(\tilde{\mathbf{x}}\right)=\frac{1}{\mathrm{Vol}\left(\mathrm{Box}\left(\mathbf{a}\right)\right)},\tilde{\mathbf{x}}\in \mathrm{Box}\left(\mathbf{a}\right).$$

Since $\mathsf{V}$ is orthogonal, we have $\mathsf{V}\tilde{\mathbf{X}}=\mathbf{X}$ and by the change of variable Theorem for $\mathbf{x}\in \mathsf{V}\mathrm{Box}\left(\mathbf{a}\right)$

$$\begin{array}{c}\hfill f_{\mathbf{X}}\left(\mathbf{x}\right)=\frac{1}{\left|\det \right(\mathsf{V}\left)\right|}{f}_{\tilde{\mathbf{X}}}\left({\mathsf{V}}^T\mathbf{x}\right)=\frac{1}{\left|\det \left(\mathsf{V}\right)\right|\mathrm{Vol}\left(\mathrm{Box}\left(\mathbf{a}\right)\right)}=\frac{1}{\mathrm{Vol}\left(\mathrm{Box}\left(\mathbf{a}\right)\right)}.\end{array}$$

Therefore, such a distribution of $\mathbf{X}$ exists. □

Theorem 20.

(Lower Bounds with SVD Precoding) Let $\mathsf{H}\in {\mathbb{R}}^{n_r\times n_t}$ be fixed, $n_{\mathsf{min}}:=min(n_r,n_t)$, and $\mathcal{X}=\mathrm{Box}\left(\mathbf{a}\right)$ for some $\mathbf{a}=(A_1,\cdots ,A_{n_t})\in {\mathbb{R}}_{+}^{n_t}$. Furthermore, let ${\sigma }_i$, $i=1,\cdots ,n_{\mathsf{min}}$, be the ith singular value of $\mathsf{H}$. Then,

$${\underline{C}}_{\mathsf{Jensen}}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})={log}_2^{+}\left({\left(\frac{2}{\mathrm{e}}\right)}^{\frac{n_{\mathsf{min}}}{2}}\frac{1}{\psi (\mathsf{H},{\mathbf{b}}^{\star })}\right)$$

(45)

and

$${\underline{C}}_{\mathsf{EPI}}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})=\frac{n_{\mathsf{min}}}{2}{log}_2\left(1+\frac{{\left|{\prod }_{i=1}^{n_{\mathsf{min}}}{A}_i{\sigma }_i\right|}^{\frac{2}{n_{\mathsf{min}}}}}{2\pi \mathrm{e}}\right),$$

(46)

where

$$\psi (\mathsf{H},{\mathbf{b}}^{\star }):=\underset{\mathbf{b}\in \mathrm{Box}\left(\mathbf{a}\right)}{min}\prod _{i=1}^{n_{\mathsf{min}}}\phi \left({\sigma }_i{B}_i\right)$$

with $\mathbf{b}:=(B_1,\cdots ,B_{n_t})$ and φ as defined in Equation (32).

Proof. 

Performing the SVD, the expected value in Theorem 14 can be written as

$$\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{H}(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right]=\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{U}\mathsf{\Lambda }{\mathsf{V}}^T(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right]=\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{\Lambda }{\mathsf{V}}^T(\mathbf{X}-{\mathbf{X}}^{\prime }){\parallel }^2}{4}}\right]=\mathbb{E}\left[{\mathrm{e}}^{-\frac{\parallel \mathsf{\Lambda }(\tilde{\mathbf{X}}-{\tilde{\mathbf{X}}}^{\prime }){\parallel }^2}{4}}\right].$$

By Lemma 3, there exists a distribution $F_{\mathbf{X}}$ such that the components of $\tilde{\mathbf{X}}$ are independent and uniformly distributed. Since $\mathsf{\Lambda }$ is a diagonal matrix, we can use Theorem 16 to arrive at Equation (45).

Note that by Lemma 3 there exists a distribution on $\mathbf{X}$ such that $\tilde{\mathbf{X}}$ is uniform over $\mathrm{Box}\left(\mathbf{a}\right)\subset {\mathbb{R}}^{n_t}$ and $\mathsf{\Lambda }\tilde{\mathbf{X}}$ is uniform over $\mathsf{\Lambda }\mathrm{Box}\left(\mathbf{a}\right)\subset {\mathbb{R}}^{n_{\mathsf{min}}}$, respectively. Therefore, by the EPI lower bound given in Equation (20), we obtain

$$\begin{array}{cc}\hfill {\underline{C}}_{\mathsf{EPI}}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})& =\frac{n_{\mathsf{min}}}{2}{log}_2\left(1+\frac{2^{\frac{2}{n_{\mathsf{min}}}h\left(\mathsf{\Lambda }\tilde{\mathbf{X}}\right)}}{2\pi \mathrm{e}}\right)\hfill \\ & =\frac{n_{\mathsf{min}}}{2}{log}_2\left(1+\frac{\mathrm{Vol}{\left(\mathsf{\Lambda }\mathrm{Box}\left(\mathbf{a}\right)\right)}^{\frac{2}{n_{\mathsf{min}}}}}{2\pi \mathrm{e}}\right)\hfill \\ & =\frac{n_{\mathsf{min}}}{2}{log}_2\left(1+\frac{{\left({\prod }_{i=1}^{n_{\mathsf{min}}}{A}_i\right)}^{\frac{2}{n_{\mathsf{min}}}}{\left|{\prod }_{i=1}^{n_{\mathsf{min}}}{\sigma }_i\right|}^{\frac{2}{n_{\mathsf{min}}}}}{2\pi \mathrm{e}}\right),\hfill \end{array}$$

which is exactly the expression in Equation (46). This concludes the proof. □

Remark 8.

Notice that choosing the optimal $\mathbf{b}$ for the lower bound in Equation (45) is an amplitude allocation problem, which is reminiscent of waterfilling in the average power constraint case. It would be interesting to study whether the bound in Equation (45) is connected to what is called mercury waterfilling in [37,38].

In Figure 4, the lower bounds of Theorem 20 are compared to the moment upper bound of Theorem 2 for the special case of a $3\times 1$ MIMO channel. Similar to the example presented in Figure 2, the EPI lower bound performs well in the low amplitude regime, while Jensen’s inequality lower bound performs well in the high amplitude regime.

We conclude this section by showing that for an arbitrary channel input space $\mathcal{X}$, in the large amplitude regime the capacity pre-log is given by $min(n_r,n_t)$.

Theorem 21.

Let $\mathcal{X}$ be arbitrary and $\mathsf{H}\in {\mathbb{R}}^{n_r\times n_t}$ fixed. Then,

$$\underset{r_{\mathsf{min}}\left(\mathcal{X}\right)\to \infty }{lim}\frac{C(\mathcal{X},\mathsf{H})}{{log}_2\left(1+\frac{2r_{\mathsf{min}}\left(\mathcal{X}\right)}{\sqrt{2\pi \mathrm{e}}}\right)}=min(n_r,n_t).$$

Proof. 

Notice that there always exists $\mathbf{a}\in {\mathbb{R}}_{+}^{n_t}$ and $c\in {\mathbb{R}}_{+}$ such that $\mathrm{Box}\left(\mathbf{a}\right)\subseteq \mathcal{X}\subset c\mathrm{Box}\left(\mathbf{a}\right)$. Thus, without loss of generality, we can consider $\mathcal{X}=\mathrm{Box}\left(\mathbf{a}\right)$, $\mathbf{a}=(A,\cdots ,A)$, for sufficiently large $A\in {\mathbb{R}}_{+}$. To prove the result, we therefore start with enlarging the constraint set of the bound in Equation (11):

$$\begin{array}{cc}\hfill \mathrm{Box}\left(\mathsf{H}\mathrm{Box}\left(\mathbf{a}\right)\right)& \subseteq {\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{H}\mathrm{Box}\left(\mathbf{a}\right)\right)\right)\hfill \\ & \subseteq {\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{H}{\mathcal{B}}_{\mathbf{0}}\left(\sqrt{n_t}A\right)\right)\right)\hfill \\ & ={\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{U}\mathsf{\Lambda }{\mathsf{V}}^T{\mathcal{B}}_{\mathbf{0}}\left(\sqrt{n_t}A\right)\right)\right)\hfill \\ & ={\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{U}\mathsf{\Lambda }{\mathcal{B}}_{\mathbf{0}}\left(\sqrt{n_t}A\right)\right)\right)\hfill \\ & ={\mathcal{B}}_{\mathbf{0}}\left(r_{\mathsf{max}}\left(\mathsf{\Lambda }{\mathcal{B}}_{\mathbf{0}}\left(\sqrt{n_t}A\right)\right)\right)\hfill \\ & \subseteq {\mathcal{B}}_{\mathbf{0}}\left(r\right)\hfill \\ & \subseteq \mathrm{Box}\left({\mathbf{a}}^{\prime }\right),\hfill \end{array}$$

where $r:=\sqrt{n_t}A\sqrt{{\sum }_{i=1}^{n_{\mathsf{min}}}{\sigma }_i^2}$ and ${\mathbf{a}}^{\prime }:=\left(\frac{r}{\sqrt{n_{\mathsf{min}}}},\cdots ,\frac{r}{\sqrt{n_{\mathsf{min}}}}\right)\in {\mathbb{R}}_{+}^{n_{\mathsf{min}}}$. Therefore, by using the upper bound in Equation (11), it follows that

$$C(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{H})\le \sum _{i=1}^{n_r}{log}_2\left(1+\frac{2A_i}{\sqrt{2\pi \mathrm{e}}}\right)\le n_{\mathsf{min}}{log}_2\left(1+\frac{2}{\sqrt{2\pi \mathrm{e}}}\frac{\sqrt{n_t}A\sqrt{{\sum }_{i=1}^{n_{\mathsf{min}}}{\sigma }_i^2}}{\sqrt{n_{\mathsf{min}}}}\right).$$

Moreover,

$$\underset{A\to \infty }{lim}\frac{C\left(\mathrm{Box}\right(\mathbf{a}),\mathsf{H})}{{log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)}\le n_{\mathsf{min}}\underset{A\to \infty }{lim}\frac{{log}_2\left(1+\frac{2}{\sqrt{2\pi \mathrm{e}}}\frac{\sqrt{n_t}A\sqrt{{\sum }_{i=1}^{n_{\mathsf{min}}}{\sigma }_i^2}}{\sqrt{n_{\mathsf{min}}}}\right)}{{log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)}=n_{\mathsf{min}}.$$

Next, using the EPI lower bound in Equation (46), we have that

$$\underset{A\to \infty }{lim}\frac{{\underline{C}}_{\mathsf{EPI}}(\mathrm{Box}\left(\mathbf{a}\right),\mathsf{\Lambda })}{{log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)}=n_{\mathsf{min}}\underset{A\to \infty }{lim}\frac{\frac{1}{2}{log}_2\left(1+\frac{A{\left|{\prod }_{i=1}^{n_{\mathsf{min}}}{\sigma }_i\right|}^{\frac{2}{n_{\mathsf{min}}}}}{2\pi \mathrm{e}}\right)}{{log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)}=n_{\mathsf{min}}.$$

This concludes the proof. □

7. The SISO Case

In this section, we apply the upper and lower bounds presented in the previous sections to the special case of a SISO channel that is subject to an amplitude constraint (i.e., $\mathcal{X}=[-A,A]$ for some $A\in {\mathbb{R}}_{+}$) and compare them with the state-of-the art. More precisely, we are interested in upper and lower bounds to the capacity

$$C([-A,A],h):=\underset{F_X:X\in [-A,A]}{max}I(X;hX+Y).$$

(47)

Without loss of generality, we assume $h=1$ in all that follows.

7.1. Upper and Lower Bounds

As a starting point for our comparisons, the following Theorem summarizes bounds on the capacity (47) that are known from the literature. The bounds are all based on the duality approach that we generalize in Section 4 to the MIMO case.

Theorem 22.

(Known Duality Upper Bounds) Let $A>0$ be arbitrary. Then, the following are valid upper bounds to the capacity of the amplitude-constrained SISO channel defined in Equation (47).

• McKellips upper bound [7]:

  $$C([-A,A],1)\le {\overline{C}}_{\mathsf{McK}}([-A,A],1):={log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right).$$

  (48)
• Thangaraj–Kramer–Böcherer upper bound ([8] Theorem 1):

  $$\begin{array}{c}\hfill C([-A,A],1)\le {\overline{C}}_{\mathsf{TKB}}([-A,A],1):=\left\{\begin{array}{cc}\beta \left(A\right){log}_{\mathrm{e}}\left(\sqrt{\frac{2}{\pi \mathrm{e}}}A\right)+H_{\mathsf{b}}\left(\beta \left(A\right)\right),\hfill & A^2\le 6.304\mathrm{dB}\hfill \\ {\overline{C}}_{\mathsf{McK}}([-A,A],1),\hfill & \mathrm{else},\hfill \end{array}\right.\end{array}$$

  (49)

  where $\beta \left(A\right):=\frac{1}{2}-Q\left(2A\right)$ and $H_{\mathsf{b}}$ denotes the binary entropy function.
• Rassouli–Clerckx upper bound [9]:

  $$C([-A,A],1)\le {\overline{C}}_{\mathsf{RC}}([-A,A],1):={\overline{C}}_{\mathsf{TKB}}([-A,A],1)+W\left(A\right),$$

  (50)

  where

  $$W\left(A\right):=\frac{1}{2}\left({log}_{\mathrm{e}}\left({\sigma }^2\left(A\right)\right)+\frac{1}{{\sigma }^2\left(A\right)}-1\right)\left(\frac{1}{2}+Q\left(2A\right)\right)+\frac{g\left(2A\right)}{2{\sigma }^2\left(A\right)},$$

  $${\sigma }^2\left(A\right):=1+\frac{2g\left(2A\right)}{1+2Q\left(2A\right)},$$

  and

  $$g\left(x\right):=x^{2}Q\left(x\right)-\frac{x}{\sqrt{2\pi }}{\mathrm{e}}^{-\frac{x^2}{2}}.$$

Now, we apply the moment upper bound of Theorem 9 to the SISO case.

Theorem 23.

(Moment Upper Bound) Let $A>0$ be arbitrary. Then,

$$C([-A,A],1)\le {\overline{C}}_{\mathsf{M}}([-A,A],1)=\underset{p>0}{inf}{log}_2\left(\frac{k_{1,p}}{{\left(2\pi \mathrm{e}\right)}^{\frac{1}{2}}}\mathbb{E}{\left[{|A+Z|}^p\right]}^{\frac{1}{p}}\right),$$

(51)

where the expected value is of the explicit form

$$\mathbb{E}\left[{|A+Z|}^p\right]=\frac{2^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p+1}{2}\right)}{\sqrt{\pi }}{}_1{F}_1\left(-\frac{p}{2};\frac{1}{2};-\frac{A^2}{2}\right)$$

with ${}_1{F}_1(a;b;z)$ being the confluent hypergeometric function of the first kind ([39] Chapter 13).

Proof. 

First, note that $r_{\mathsf{max}}\left([-A,A]\right)=A$. Then, by using the expression for the raw absolute moment of a Gaussian distribution given in [40], we have that

$$\underset{a\in [0,A]}{max}\mathbb{E}\left[{|a+Z|}^p\right]=\underset{a\in [0,A]}{max}\frac{2^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p+1}{2}\right)}{\sqrt{\pi }}{}_1{F}_1\left(-\frac{p}{2};\frac{1}{2};-\frac{a^2}{2}\right).$$

The proof is concluded by observing that $f\left(a\right):={}_1{F}_1\left(-\frac{p}{2};\frac{1}{2};-\frac{a^2}{2}\right)$ is an increasing function in a. □

The following theorem establishes the EPI and the Jensen lower bound of Section 4.2 assuming the channel input symbols are uniformly distributed.

Theorem 24.

(Lower Bounds with Uniform Inputs) Let $A>0$ be arbitrary and the channel input X be uniformly distributed over $[-A,A]$. Then,

$$C([-A,A],1)\ge {\underline{C}}_{\mathsf{EPI}}([-A,A],1)=\frac{1}{2}{log}_2\left(1+\frac{2A^2}{\pi \mathrm{e}}\right)$$

(52)

and

$$C([-A,A],1)\ge {\underline{C}}_{\mathsf{Jensen}}([-A,A],1)={log}_2\left(\frac{\sqrt{2}{A}^2}{{\mathrm{e}}^{\frac{1}{2}}\left({\mathrm{e}}^{-A^2}-1+\sqrt{\pi }A\left(1-2Q\left(\sqrt{2}A\right)\right)\right)}\right).$$

(53)

Proof. 

The lower bound in Equation (52) follows from Theorem 12 by observing that $\mathrm{Vol}\left(\mathcal{X}\right)=2A$. To show the lower bound in Equation (53), consider Theorem 14 and let X and $X^{\prime }$ be independent and uniformly distributed over $[-A,A]$. Then, we have

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\mathrm{e}}^{-\frac{|X-X^{\prime }{|}^2}{4}}\right]& =\frac{1}{4A^2}{\int }_{-A}^A{\int }_{-A}^A{\mathrm{e}}^{-\frac{{(x-x^{\prime })}^2}{4}}\mathrm{d}x\mathrm{d}{x}^{\prime }\hfill \\ & =\frac{1}{A^2}\left({\mathrm{e}}^{-A^2}-1+\sqrt{\pi }A\left(1-2Q\left(\sqrt{2}A\right)\right)\right),\hfill \end{array}$$

which concludes the proof. □

Restricting the channel inputs to be discrete allows for another set of lower bounds on Equation (47).

Theorem 25.

(Lower Bounds with Discrete Inputs) Let $A>1$ be arbitrary, $X_B\in \{-A,A\}$ equally likely, and $X_D\sim \mathsf{PAM}\left(N\right)$ with $N=⌈1+\frac{A}{\sqrt{2\pi \mathrm{e}}}⌉$. Then,

$$\begin{array}{cc}\hfill C([-A,A],1)\ge {\underline{C}}_{\mathsf{Binary}}([-A,A],1)& :=I(X_B;X_B+Z)\hfill \end{array}$$

(54)

$$\begin{array}{c}=\frac{1}{{log}_{\mathrm{e}}\left(2\right)}\left(A^2-{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}{log}_{\mathrm{e}}\left(cosh(A^2-Ay)\right)\mathrm{d}y\right),\hfill \end{array}$$

(55)

$$C([-A,A],1)\ge {\underline{C}}_{\mathsf{Jensen}}([-A,A],1)=-{log}_2\left(\sqrt{\frac{\mathrm{e}}{2}}\frac{1}{N^2}\sum _{(x_{D_i},x_{D_j})\in \mathsf{PAM}{\left(N\right)}^2}{\mathrm{e}}^{-\frac{{(x_{D_i}-x_{D_j})}^2}{4}}\right),$$

(56)

and

$$C([-A,A],1)\ge {\underline{C}}_{\mathsf{OW}}([-A,A],1)={\overline{C}}_{\mathsf{McK}}([-A,A],1)-2.$$

(57)

Proof. 

The expression of the mutual information in Equation (54) for a uniform binary input $X_B\in \{-A,A\}$ is found in [41] by using the I-MMSE relationship. The bound in Equation (56) follows from using Theorem 14 and the bound in Equation (57) from Theorem 13, respectively. This concludes the proof. □

Figure 5 compares the upper and lower bounds presented in this section in dependency of the amplitude constraint A. Observe that for values of A smaller than ≈1.665 (i.e., to the left of the gray vertical line), the lower bound (55) is in fact equal to the capacity. Up to constraints of $A\approx 1$, the moment upper bound in Equation (51) is the best after which the bound in Equation (50) becomes the tightest. The best lower bound for constraint values smaller than $A\approx 10$ is the bound in Equation (56) after which the lower bound in Equation (53) becomes the tightest. Note that all lower and upper bounds are asymptotically tight (i.e., for $A\to \infty$).

7.2. High and Low Amplitude Asymptotics

In this subsection, we study how the capacity in Equation (47) behaves in the high and low amplitude regimes. To this end, we need the following expression

$${\overline{C}}_{\mathsf{AWGN}}([-A,A],1):=\frac{1}{2}{log}_2\left(1+A^2\right),$$

which is either the capacity of a SISO channel with an average power constraint $A^2$ or the moment bound in Equation (51) evaluated for $p=2$.

Theorem 26.

(SISO High and Low Amplitude Asymptotics) It holds

$$\underset{A\to 0}{lim}\frac{C\left(\right[-A,A],1)}{{\overline{C}}_{\mathsf{AWGN}}([-A,A],1)}=1,$$

(58)

$$\underset{A\to \infty }{lim}\frac{C\left(\right[-A,A],1)}{{\overline{C}}_{\mathsf{McK}}([-A,A],1)}=1,$$

(59)

and

$$\underset{A\to \infty }{lim}C([-A,A],1)-{\overline{C}}_{\mathsf{AWGN}}([-A,A],1)=\frac{1}{2}{log}_2\left(\frac{\pi \mathrm{e}}{2}\right)\approx 1.044.$$

(60)

Proof. 

The capacity of an amplitude-constrained SISO channel in the regime of low amplitudes (i.e., for amplitudes smaller than $A\approx 1.655$) was given by Guo et al. [41]

$$C([-A,A],1)=\frac{1}{{log}_{\mathrm{e}}\left(2\right)}\left(A^2-{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}{log}_{\mathrm{e}}\left(cosh(A^2-Ay)\right)\mathrm{d}y\right).$$

Now, observe that

$$\begin{array}{c}\underset{A\to 0}{lim}\frac{1}{\frac{1}{2}{log}_2(1+A^2)}{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}{log}_{\mathrm{e}}\left(cosh(A^2-Ay)\right)\mathrm{d}y\hfill \\ =\frac{2}{{log}_2\left(\mathrm{e}\right)}{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}\frac{{log}_{\mathrm{e}}\left(cosh(A^2-Ay)\right)}{{log}_{\mathrm{e}}(1+A^2)}\mathrm{d}y\hfill \\ =\frac{2}{{log}_2\left(\mathrm{e}\right)}{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}\underset{A\to 0}{lim}\frac{{log}_{\mathrm{e}}\left(cosh(A^2-Ay)\right)}{{log}_{\mathrm{e}}(1+A^2)}\mathrm{d}y\hfill \\ =\frac{2}{{log}_2\left(\mathrm{e}\right)}{\int }_{-\infty }^{\infty }\frac{{\mathrm{e}}^{-\frac{y^2}{2}}}{\sqrt{2\pi }}\frac{y^2}{2}\mathrm{d}y\hfill \\ =\frac{1}{{log}_2\left(\mathrm{e}\right)}.\hfill \end{array}$$

Therefore, the limit in Equation (58) is given by

$$\begin{array}{cc}\hfill \underset{A\to 0}{lim}\frac{C\left(\right[-A,A],1)}{{\overline{C}}_{\mathsf{AWGN}}([-A,A],1)}& =\underset{A\to 0}{lim}\frac{A^2}{{log}_{\mathrm{e}}\left(2\right)\frac{1}{2}{log}_2(1+A^2)}-\frac{1}{{log}_{\mathrm{e}}\left(2\right){log}_2\left(\mathrm{e}\right)}\hfill \\ & =\frac{2{log}_{\mathrm{e}}\left(2\right)}{{log}_{\mathrm{e}}\left(2\right)}-\frac{1}{{log}_{\mathrm{e}}\left(2\right){log}_2\left(\mathrm{e}\right)}\hfill \\ & =2-1=1.\hfill \end{array}$$

The limit in Equation (59) follows from comparing the EPI lower bound ${\underline{C}}_{\mathsf{EPI}}([-A,A],1)=\frac{1}{2}{log}_2\left(1+\frac{2A^2}{\pi \mathrm{e}}\right)$ in (52) with the McKellips upper bound ${\overline{C}}_{\mathsf{McK}}([-A,A],1)={log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)$ given in Equation (48).

Finally, to show Equation (60), observe that

$$\begin{array}{cc}\hfill \underset{A\to \infty }{lim}C([-A,A],1)-{\overline{C}}_{\mathsf{AWGN}}([-A,A],1)& =\underset{A\to \infty }{lim}{\overline{C}}_{\mathsf{McK}}([-A,A],1)-{\overline{C}}_{\mathsf{AWGN}}([-A,A],1)\hfill \\ & =\underset{A\to \infty }{lim}{log}_2\left(1+\frac{2A}{\sqrt{2\pi \mathrm{e}}}\right)-\frac{1}{2}{log}_2(1+A^2)\hfill \\ & =\frac{1}{2}{log}_2\left(\frac{\pi \mathrm{e}}{2}\right).\hfill \end{array}$$

This concludes the proof. □

8. Conclusions

In this work, we studied the capacity of MIMO channels with bounded input spaces. Several new properties of input distributions that achieve the capacity of such channels have been provided. In particular, it is shown that the support of a capacity-achieving channel input distribution is a set that is small in a topological and measure theoretical sense. In addition to that, it is shown that, if the radius of the underlying channel input space, $\mathcal{X}$, is small enough, then the support of a corresponding capacity-achieving input distribution must necessarily be a subset of the boundary of $\mathcal{X}$. As the considerations on the input distribution have demonstrated that determining the capacity is a very challenging problem, we proposed several new upper and lower bounds that are shown to be tight in the high amplitude regime. An interesting future direction would be to study generalizations of our techniques to wireless optical MIMO channels [42] and other channels such as the wiretap channel [43].