Design of Sparse FIR Decision Feedback Equalizers in MIMO Systems Using Hybrid l₁/l₂ Norm Minimization and the OMP Algorithm

Abstract

In this paper, a novel scheme using hybrid l₁/l₂ norm minimization and the orthogonal matching pursuit (OMP) algorithm is proposed to design the sparse finite impulse response (FIR) decision feedback equalizers (DFE) in multiple input multiple output (MIMO) systems. To reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy, the problem of designing a sparse FIR DFE is transformed into an l₀ norm minimization problem, and then the proposed scheme is used to obtain the sparse solution. In the proposed scheme, a sequence of minimum weighted l₂ norm problems is solved using the OMP algorithm. The nonzero taps positions can be corrected with the different weights in the diagonal weighting matrix which is computed through the hybrid l₁/l₂ norm minimization. The simulation results verify that the sparse FIR MIMO DFEs designed by the proposed scheme get a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse optimum DFE under the minimum mean square error (MMSE) criterion. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.

1. Introduction

In high-speed wireless communication systems, the multipath effect in channels will cause severe intersymbol interference (ISI). The orthogonal frequency division multiplexing (OFDM) system [1,2,3], with the presence of the orthogonal subcarriers and the cyclic prefix, has a strong ability to combat the ISI and generally requires simple equalization. However, compared with OFDM, the single-carrier communication has the advantage of low peak-to-average power ratio of the transmitted signal and insensitivity to carrier frequency offsets [4]. Therefore, the single-carrier communication is widely applied in many communication systems, such as in underwater acoustic communication [5,6,7] and vehicular communication systems [8]. One of the widely used methods to reduce the ISI in single-carrier systems is employing the finite impulse response (FIR) decision feedback equalizers (DFE) with a large number of taps to filter the received signals. However, the design complexity of FIR DFEs grows proportional to the square of the number of nonzero taps. Especially in multiple input multiple output (MIMO) systems, the design complexity can be much higher because the number of DFE taps grows proportional to the number of multiple antennas [9]. Therefore, to reduce the design complexity, it is necessary to design the FIR DFE with only a few nonzero taps (i.e., sparse FIR DFE). In [10], the number of nonzero taps is reduced by choosing only a few significant taps of the minimum mean square error (MMSE) equalizer which is not sparse in general. Nevertheless, the non-sparse MMSE solution must be calculated first which increases the design complexity. Therefore, how to determine the positions and values of nonzero taps directly is one of the main difficulties to design the sparse FIR DFE.

In recent years, some new ways of designing sparse FIR DFE have arisen with the research and application of sparse representation and compressed sensing theory [11,12,13,14,15,16]. Among them, the compressed sensing framework for sparse signal processing is proposed in [11]. A greedy search method to determine the nonzero taps positions of a single input single output (SISO) system is presented in [12]. The l₁ norm minimization method to design sparse filters is proposed in [13], but the modeling without wireless channel is quite different from the sparse equalizer design. In [14], a sparse adaptive equalizer design approach using the stochastic gradient pursuit algorithm based on compressed sensing is proposed, but the channel impulse response in the approach is assumed to be sparse. The new design frameworks for sparse FIR MIMO equalizers based on the orthogonal matching pursuit algorithm (OMP) and sparse approximation of a vector using different dictionaries are exploited in [15,16], respectively. They achieve better performance than the method in [10] with the same number of nonzero taps. Furthermore, for the pursuit of both sparse promotion and improved accuracy, the sparse signals reconstruction algorithms inspired by l₁ and weighted l₁ regularization schemes are proposed in [17,18]; the joint smoothed l₀ norm algorithm for direction-of-arrival estimation in MIMO radar is proposed in [19].

We aim to reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy by applying the OMP algorithm to hybrid l₁/l₂ norm minimization. The OMP algorithm is a popular greedy algorithm that determines a nonzero tap position for each iteration [20]. However, there is no correction for the selected nonzero taps positions in the previous iterations, which will reduce the design accuracy for the sparse FIR DFE. The hybrid l₁/l₂ norm minimization is proposed in [21] to make the solution for data fitting less sensitive to large residuals and more robust, and it has been applied in many fields such as geophysics, image processing and machine learning. Within this context, we propose a scheme using hybrid l₁/l₂ norm minimization and OMP algorithm. In the proposed scheme, a minimum weighted l₂ norm problems is solved using the OMP algorithm in each iteration, and the nonzero taps positions can be corrected with the iteratively updated weights in the diagonal weighting matrix, then the weighting matrix is updated through the hybrid l₁/l₂ norm minimization.

In this paper, we transform the problem of designing a sparse FIR MIMO DFE into an l₀ norm minimization problem, then use the proposed scheme to solve the l₀ norm minimization problem and obtain the sparse solution. It is verified through simulations that, when compared with the non-sparse optimum MMSE DFE, the sparse FIR MIMO DFEs designed by the proposed scheme obtain a significant reduction in the number of nonzero taps with a small performance loss. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same nonzero taps percentage.

2. System Model

Consider a baseband equivalent discrete signal model in the MIMO systems with n_i transmit (input) antennas and n_o receive (output) antennas. Assuming that the channel is a linear time-invariant Rayleigh fading channel with an oversampling factor of l, the output of the MIMO channel at time k, i.e., the received samples from the n_o receive antennas which are grouped into a ln_o × 1 vector can be formed as:

$${\mathit{y}}_k=\left[\begin{array}{c}{\mathit{y}}_k^{(1)}\\ {\mathit{y}}_k^{(2)}\\ ⋮\\ {\mathit{y}}_k^{(n_o)}\end{array}\right]=\left[\begin{array}{cccc}{\mathit{H}}_0& {\mathit{H}}_1& \cdots & {\mathit{H}}_{\nu }\end{array}\right]\left[\begin{array}{c}{\mathit{x}}_k\\ {\mathit{x}}_{k-1}\\ ⋮\\ {\mathit{x}}_{k-\nu }\end{array}\right]+\left[\begin{array}{c}{\mathit{n}}_k^{(1)}\\ {\mathit{n}}_k^{(2)}\\ ⋮\\ {\mathit{n}}_k^{(n_o)}\end{array}\right],$$

(1)

where ${\mathit{y}}_k^{(j)}={[y_k^{(j)}(1),y_k^{(j)}(2),\cdots ,y_k^{(j)}(l)]}^{\mathrm{T}}(1\le j\le n_o)$ is the l × 1 output vector of the j-th receive antennas, ${\mathit{H}}_m(0\le m\le \nu )$ is the ln_o × n_i m-th MIMO channel matrix, ${\mathit{x}}_{k-m}$ is the input vector of the transmit antennas at time ($k-m$). The parameter ν is the maximum length of all the subchannel impulse response. ${\mathit{n}}_k^{(j)}$ is an additive white Gaussian noise vector of the j-th receive antennas with mean 0 and variance σ². Considering a block of N_f received vector ${\mathit{y}}_k$, (1) can be rewritten as [9]

$${\mathit{y}}_{k:k-N_f+1}=\mathit{H}{\mathit{x}}_{k:k-N_f-\nu +1}+{\mathit{n}}_{k:k-N_f+1},$$

(2)

where

$$\mathit{H}=\left[\begin{array}{cc}\begin{array}{cccc}{\mathit{H}}_0& {\mathit{H}}_1& \cdots & {\mathit{H}}_{\nu }\\ 0& {\mathit{H}}_0& {\mathit{H}}_1& \cdots \\ ⋮& \ddots & \ddots & \ddots \\ 0& \cdots & 0& 0\end{array}& \begin{array}{cccc}0& 0& \cdots & 0\\ {\mathit{H}}_{\nu }& 0& \cdots & 0\\ \ddots & \ddots & \ddots & ⋮\\ {\mathit{H}}_0& {\mathit{H}}_1& \cdots & {\mathit{H}}_{\nu }\end{array}\end{array}\right].$$

(3)

Assuming that the transmit signals are independent with the average power ${\epsilon }_x$, the input autocorrelation matrix can be expressed as

$${\mathit{R}}_{xx}\triangleq E\left[{\mathit{x}}_{k:k-N_f-\nu +1}{\mathit{x}}_{k:k-N_f-\nu +1}^{\mathrm{H}}\right]={\epsilon }_x{\mathit{I}}_{n_i(N_f+\nu )},$$

(4)

where I is the identity matrix, and the noise autocorrelation matrix can be expressed as

$${\mathit{R}}_{nn}\triangleq E\left[{\mathit{n}}_{k:k-N_f+1}{\mathit{n}}_{k:k-N_f+1}^{\mathrm{H}}\right]={\sigma }^2{\mathit{I}}_{n_0{N}_f}.$$

(5)

Furthermore, the output-input crosscorrelation and the output autocorrelation matrices are as follows

$${\mathit{R}}_{yx}\triangleq E\left[{\mathit{y}}_{k:k-N_f+1}{\mathit{x}}_{k:k-N_f-\nu +1}^{\mathrm{H}}\right]=\mathit{H}{\mathit{R}}_{xx},$$

(6)

$${\mathit{R}}_{yy}\triangleq E\left[{\mathit{y}}_{k:k-N_f+1}{\mathit{y}}_{k:k-N_f+1}^{\mathrm{H}}\right]=\mathit{H}{\mathit{R}}_{xx}{\mathit{H}}^{\mathrm{H}}+{\mathit{R}}_{nn}.$$

(7)

3. Design of Sparse FIR DFE Using Hybrid l₁/l₂ Norm Minimization and OMP Algorithm

In the FIR MIMO DFE model, in order to eliminate ISI and inter-antenna interference, then recover the input signal of the i-th transmit antenna, the feedforward filter ${\mathit{w}}_i$ with ln_oN_f taps processes the received signals ${\mathit{y}}_k$, and the feedback filter ${\mathit{b}}_i$ with n_iN_b taps convolves with the previous decision signals, then the result combined both together is applied to the decision device to obtain current decision signal. If the previous decision signals before time k are correct, the DFE error sample at time k for the i-th input antenna is given by

$$e_k^{(i)}=x_{k-\mathsf{\Delta }}^{(i)}-\underset{\triangleq {\tilde{\mathit{w}}}_i^{\mathrm{H}}}{\underbrace{\left[{\mathit{w}}_i^{\mathrm{H}}-{\mathit{b}}_i^{\mathrm{H}}\right]}}\underset{\triangleq \tilde{\mathit{y}}}{\underbrace{\left[\begin{array}{c}{\mathit{y}}_{k:k-N_f+1}\\ {\mathit{x}}_{k-△-1:k-△-N_b}\end{array}\right]}},$$

(8)

where $x_{k-\mathsf{\Delta }}^{(i)}$ is the transmit signal at time ($k-\mathsf{\Delta }$) by the i-th input antenna, and $\mathsf{\Delta }$ is an integer representing the decision delay, usually $0\le \mathsf{\Delta }\le N_f+\nu -1$. Using $N=l{n}_o{N}_f+n_i{N}_b$ to represent the number of DFE taps for simplicity, so ${\tilde{\mathit{w}}}_i$ is an $N\times 1$ vector. Then the equalizer mean square error (MSE) for the i-th input antenna is expressed as

$$\begin{array}{ll}{\xi }_i& \triangleq \mathrm{E}\left[{\left|e_k^{(i)}\right|}^2\right]=\mathrm{E}\left[{\left|x_{k-\mathsf{\Delta }}^{(i)}-{\tilde{\mathit{w}}}_i^{\mathrm{H}}\tilde{\mathit{y}}\right|}^2\right]\\ & ={\epsilon }_x-{\tilde{\mathit{w}}}_i^{\mathrm{H}}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}-{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}^{\mathrm{H}}{\tilde{\mathit{w}}}_i+{\tilde{\mathit{w}}}_i^{\mathrm{H}}{\mathit{R}}_{\tilde{y}\tilde{y}}{\tilde{\mathit{w}}}_i\end{array},$$

(9)

where ${\mathit{R}}_{\tilde{y}\tilde{y}}\triangleq \mathrm{E}\left[\tilde{\mathit{y}}{\tilde{\mathit{y}}}^{\mathrm{H}}\right]$, ${\tilde{\mathit{r}}}_{\mathsf{\Delta },i}={\mathit{R}}_{\tilde{y}x}{\mathbf{1}}_{\mathsf{\Delta },i}$, ${\mathit{R}}_{\tilde{y}x}\triangleq \mathrm{E}\left[\tilde{\mathit{y}}{\mathit{x}}_{k:k-N_f-\nu +1}^{\mathrm{H}}\right]$ and ${\mathbf{1}}_{\mathsf{\Delta },i}$ is the ($n_i\mathsf{\Delta }+i$)-th column of ${\mathit{I}}_{n_i(N_f+\nu )}$. Performing the Cholesky decomposition ${\mathit{R}}_{\tilde{y}\tilde{y}}=\tilde{\mathit{L}}{\tilde{\mathit{L}}}^H$, where $\tilde{\mathit{L}}$ is an $N\times N$ lower triangular matrix, (9) can be rewritten as

$${\xi }_i=\underset{\triangleq {\xi }_{\min ,i}}{\underbrace{{\epsilon }_x-{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}^{\mathrm{H}}{\tilde{\mathit{L}}}^{-\mathrm{H}}{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}}}+\underset{\triangleq {\xi }_{\mathrm{excess},i}}{\underbrace{{\Vert {\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}\Vert }_2^2}}.$$

(10)

From (10) it shows that the different choice of ${\tilde{\mathit{w}}}_i$ can only change ${\xi }_{\mathrm{excess},i}$, and ${\xi }_{\min ,i}$ remains unchanged. Obviously, ${\xi }_{\mathrm{excess},i}\ge 0$. Therefore, when ${\xi }_{\mathrm{excess},i}=0$, ${\tilde{\mathit{w}}}_i$ achieves the optimum solution under the MMSE criterion as

$${\tilde{\mathit{w}}}_{\mathrm{opt},i}={\tilde{\mathit{L}}}^{-H}{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}={\mathit{R}}_{\tilde{y}\tilde{y}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}.$$

(11)

${\tilde{\mathit{w}}}_{\mathrm{opt},i}$ is usually non-sparse with too much complexity to compute directly. Therefore, in order to reduce the computational complexity, considering to reduce the number of the DFE nonzero taps at the cost of a small increase in the equalizer MSE, i.e., finding the sparse solution ${\tilde{\mathit{w}}}_{s,i}$ for ${\tilde{\mathit{w}}}_i$ when ${\xi }_{\mathrm{excess},i}\le \theta (\theta >0)$, the sparse FIR MIMO DFE design problem can be transformed into the l₀ norm minimization problem as follows [15]

$${\tilde{\mathit{w}}}_{s,i}=\underset{{\tilde{\mathit{w}}}_i}{\arg \min }{\Vert {\tilde{\mathit{w}}}_i\Vert }_0\mathrm{subject}\mathrm{to}{\Vert {\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}\Vert }_2^2\le \theta .$$

(12)

In general, finding the optimal solution of the above l₀ norm minimization problem is NP-hard. One of the approaches to computing the suboptimal solution for the approximation of (12) is greedy algorithms. To design the sparse solution for the FIR MIMO DFE, in this paper, we propose the scheme based on the OMP algorithm which is a widely-used greedy algorithm.

The OMP algorithm selects an atom from the matrix ${\tilde{\mathit{L}}}^{\mathrm{H}}$ (i.e., the column vector of the matrix ${\tilde{\mathit{L}}}^{\mathrm{H}}$) that is most correlated with the current residual, then solves a constraint least squares problem to obtain all the selected atoms coefficients and updates the residual in each iteration. There is no correction for the selected nonzero taps positions in OMP algorithm, which will reduce the design accuracy for the sparse FIR DFE. To solve the problem, in this paper, we propose the scheme using hybrid l₁/l₂ norm minimization and OMP algorithm.

According to the hybrid l₁/l₂ norm minimization, at the ($k+1$)-th iteration, the proposed scheme will reweight the residual vector $\mathit{r}={\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}$ with an $N\times N$ iteratively updated diagonal weighting matrix ${\mathit{Q}}^{(k)}$, and minimize the weighted l₂ norm

$$J({\tilde{\mathit{w}}}_i)={\Vert {\mathit{Q}}^{(k)}{\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\mathit{Q}}^{(k)}{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}\Vert }_2^2.$$

(13)

Then solve the above problem using the OMP algorithm to find a sparse solution. ${\mathit{Q}}^{(k)}$ is updated from the current residual through the hybrid l₁/l₂ norm minimization. The diagonal entries of ${\mathit{Q}}^{(k)}$ are as follows

$$q_{nn}^{(k)}=\frac{1}{{\left(1+{(r_n^{(k)}/\mu )}^2\right)}^{1/4}},1\le n\le N,$$

(14)

where $r_n^{(k)}$ is the n-th entry of the current residual ${\mathit{r}}^{(k)}$ and $\mu$ is determined by the sparse solution. Substituting (14) into (13), minimizing (13) is equivalent to minimizing

$$J({\tilde{\mathit{w}}}_i)={\displaystyle \sum _{n=1}^{N}j(r_n^{(k)})}={\displaystyle \sum _{i=1}^N\frac{{(r_n^{(k)})}^2}{\sqrt{1+{(r_n^{(k)}/\mu )}^2}}}.$$

(15)

For an arbitrary value of $r_n^{(k)}$, the approximation of $j(r_n^{(k)})$ is

$$j(r_n^{(k)})\approx \{\begin{array}{c}{(r_n^{(k)})}^2,\left|r_n^{(k)}\right|\ll \mu \\ \mu \left|r_n^{(k)}\right|,\left|r_n^{(k)}\right|\gg \mu \end{array}$$

(16)

which suggests $J({\tilde{\mathit{w}}}_i)$ behaves like an l₂ norm for small residuals and like an l₁ norm for large residuals. Therefore, the hybrid l₁/l₂ norm minimization will make the sparse solution more robust and accurate by decreasing the sensitivity of the solution to large residuals.

The procedures of the proposed scheme for finding the sparse solution of the FIR MIMO DFE are as follows:

Initialization: Set the initial residual ${\mathit{r}}^{(0)}={\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}$, define the index set $\mathsf{\Omega }=\{1,2,\cdots ,N\}$, and initialize the diagonal weighting matrix ${\mathit{Q}}^{(0)}=\mathrm{diag}(1,1,\cdots ,1)$ for the first iteration.

Main iteration: Set k = 1 and perform the following steps:

Step 1: Reweight the residual vector $\mathit{r}={\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}$ and formulate the problem

$$\min {\Vert {\mathit{Q}}^{(k-1)}{\tilde{\mathit{L}}}^{\mathrm{H}}{\tilde{\mathit{w}}}_i-{\mathit{Q}}^{(k-1)}{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}\Vert }_2^2\mathrm{subject}\mathrm{to}{\Vert {\tilde{\mathit{w}}}_i\Vert }_0\le k.$$

(17)

Set ${\mathit{A}}^{(k)}={\mathit{Q}}^{(k-1)}{\tilde{\mathit{L}}}^{\mathrm{H}}$ and normalize the column vectors ${\{{\mathit{a}}_j^{(k)}\}}_{j=1}^N$ of the matrix ${\mathit{A}}^{(k)}$ as follows

$${\mathit{d}}^{(k)}={\mathit{A}}^{(k)}{\mathit{\Lambda }}^{(k)}=[{\mathit{d}}_1^{(k)},{\mathit{d}}_2^{(k)},\cdots ,{\mathit{d}}_N^{(k)}]$$

(18)

where ${\mathit{\Lambda }}^{(k)}=\mathrm{diag}\left(1/{\Vert {\mathit{a}}_1^{(k)}\Vert }_2,1/{\Vert {\mathit{a}}_2^{(k)}\Vert }_2,\cdots ,1/{\Vert {\mathit{a}}_N^{(k)}\Vert }_2\right)$. Set ${\mathit{z}}^{(k)}={({\mathit{\Lambda }}^{(k)})}^{-1}{\tilde{\mathit{w}}}_i$ and ${\mathit{\beta }}^{(k)}={\mathit{Q}}^{(k-1)}{\tilde{\mathit{L}}}^{-1}{\tilde{\mathit{r}}}_{\mathsf{\Delta },i}$, then (14) becomes

$$\min {\Vert {\mathit{D}}^{(k)}{\mathit{z}}^{(k)}-{\mathit{\beta }}^{(k)}\Vert }_2^2\mathrm{subject}\mathrm{to}{\Vert {\mathit{z}}^{(k)}\Vert }_0\le k$$

(19)

Step 2: Solve (16) with the OMP algorithm. Set the equalizer nonzero taps initial index set $I_0=\varnothing$, initialize the residual ${\mathit{r}}_0={\mathit{\beta }}^{(k)}$. In the t-th ($1\le t\le k$) iteration, the process is as follows:

(1) Calculate the correlation between the residual ${\mathit{r}}_{t-1}$ and the column vector of the matrix ${\mathit{D}}^{(k)}$, and find the index ${\lambda }_t$ to the vector ${\mathit{d}}_{{\lambda }_t}^{(k)}$ which is most correlated with ${\mathit{r}}_{t-1}$, i.e., ${\lambda }_t=\arg \underset{j}{\max }\left|\langle {\mathit{r}}_{t-1},{\mathit{d}}_j^{(k)}\rangle \right|,j\in \mathsf{\Omega }-I_{t-1}$;

(2) Update the nonzero taps index set $I_t=I_{t-1}\cup \left\{{\lambda }_t\right\}$, and then update the set of column vectors ${\mathsf{\Phi }}_t^{(k)}=[{\mathit{d}}_{{\lambda }_1}^{(k)},{\mathit{d}}_{{\lambda }_2}^{(k)},\cdots ,{\mathit{d}}_{{\lambda }_t}^{(k)}]$ corresponding to $I_t$;

(3) Compute the sparse solution ${\widehat{\mathit{z}}}_t^{(k)}=\mathrm{argmin}{\Vert {\mathsf{\Phi }}_t^{(k)}{\mathit{z}}_t^{(k)}-{\mathit{\beta }}^{(k)}\Vert }_2$, and update the residual ${\mathit{r}}_t={\mathit{\beta }}^{(k)}-{\mathsf{\Phi }}_t^{(k)}{\widehat{\mathit{z}}}_t^{(k)}$;

(4) Check the value of t. If t = k, update the sparse solution ${\widehat{\mathit{z}}}^{(k)}={\widehat{\mathit{z}}}_t^{(k)}$ and residual ${\mathit{r}}^{(k)}={({\mathit{Q}}^{(k-1)})}^{-1}{\mathit{r}}_t$, then proceed to Step 3, else set t = t + 1 and go to (1).

Step 3: Check the stopping rule whether ${\Vert {\mathit{r}}^{(k)}\Vert }_2^2\le \theta$. If met, get the sparse solution ${\tilde{\mathit{w}}}_{s,i}={\mathit{\Lambda }}^{(k)}{\widehat{\mathit{z}}}^{(k)}$ for the FIR DFE, and exit the algorithm; else update the diagonal weighting matrix ${\mathit{Q}}^{(k)}$ whose diagonal entries are determined by (14). From [22], set $\mu =\max (\left|{\widehat{\mathit{z}}}^{(k)}\right|)/100$. Then set k = k + 1 and go to Step 1.

4. Simulation Results

In the following, the simulation results are provided to verify the performance of the proposed scheme for sparse FIR MIMO DFE design in two different channels. As a comparison, the performance of the optimum MMSE DFE and the OMP method in [15] are presented respectively.

Two different multipath Rayleigh fading channels, namely ITU (International Telecommunication Union) Vehicular A channel [23] and an exponential decay channel, are simulated. ITU Vehicular A channel has a sparse power delay profile (PDP) with 6-paths spanning about 13 symbol periods. The exponential decay channel has a non-sparse PDP as $\rho (n)={10}^{-n/10},0\le n\le 12$ with 13-paths. The simulation system uses binary phase shift keying (BPSK) modulation. The decision delay is set to be $\mathsf{\Delta }=N_f–1$, which is optimum when N_b = ν for DFEs [24]. The small increase $\theta$ in the equalizer MSE of the sparse DFE compared with the MMSE DFE is determined by the maximum tolerable performance loss ${\eta }_{\max }$ as $\theta =({10}^{{\eta }_{\max }/10}-1){\xi }_{\min }$, where ${\xi }_{\min }$ is the MSE for the MMSE DFE. We define the signal-to-noise (SNR) as $\mathrm{SNR}\triangleq 10\lg ({\epsilon }_x/{\sigma }^2)$, set the oversampling factor l = 2 and MIMO DFEs with n_i = 2 and n_o = 2.

Figure 1 plots the nonzero taps percentage for the sparse SISO DFEs and MIMO DFEs versus the performance loss ${\eta }_{\max }$ using the proposed scheme respectively. It is shown that the number of nonzero taps for DFE is reduced with the increase of ${\eta }_{\max }$. When ${\eta }_{\max }$ = 0.4 dB, the nonzero taps for both the sparse SISO DFEs and MIMO DFEs are reduced by more than 70% and 60% in the two different channels respectively. Therefore, comparing the ITU Vehicular A channel with sparse PDP, the FIR DFEs needs more nonzero taps to eliminate the ISI caused by the multipath effect in the exponential decay channel with more paths. Furthermore, in MIMO systems, the DFEs needs more nonzero taps to eliminate both ISI and inter-antenna interference, as shown in Figure 1, MIMO DFEs has a larger percentage of nonzero taps when compared with SISO DFEs.

Figure 2 gives a single example of the SISO DFE by the three design methods in ITU Vehicular A channel. Note that only the nonzero taps are marked, and the tap indices below 25 are insignificant to show. It is shown that there are many taps whose values are close to zero in the non-sparse MMSE DFE. Compared with the MMSE DFE, most of the taps for sparse FIR DFEs designed by the OMP method and the proposed scheme are zero, and the tap positions selected by the two methods are not identical.

To verify the performance of the proposed scheme, we compare the bit error rate (BER) and the normalized mean square error (NMSE) for the three design methods. Figure 3 and Figure 4 compare the BER performance and the equalizer NMSE respectively in ITU Vehicular A channel. The sparse DFEs designed by the proposed scheme and the OMP algorithm are constrained to have the same sparsity level in the simulation. The simulation results show that the maximum SNR loss for the SISO DFEs and MIMO DFEs designed by the proposed scheme are only 0.4 dB and 0.6 dB when the number of nonzero taps is reduced by more than 83% and 80%, respectively. That is, the BER performance and the NMSE for the sparse DFEs designed by the proposed scheme are similar to the MMSE DFE performance. As shown in Figure 3, the BER performance for the sparse DFEs designed by the proposed scheme outperform the OMP algorithm with the same sparsity level. Besides, by comparing the NMSE of the two methods, it is known that the design accuracy of the proposed scheme is superior to the OMP method. Considering that the MIMO system has a certain diversity gain, the simulation results in Figure 3 show that the BER performance of the MIMO DFEs have a significant improvement than the SISO DFEs.

To verify the performance of the proposed scheme in different channels, Figure 5 and Figure 6 show the simulation results in the exponential decay channel. Comparing with the OMP method, the results also show that the proposed scheme has better BER performance and smaller NMSE with the same sparsity level. Therefore, the proposed scheme provides excellent performance in different channel conditions.

5. Conclusions

In this paper, a novel scheme using hybrid l₁/l₂ norm minimization and OMP algorithm has been proposed to design the sparse FIR MIMO DFE. The proposed scheme solves a of minimum weighted l₂ norm problem using OMP algorithm in each iteration, then the weights weighting matrix is updated through the hybrid l₁/l₂ norm minimization. The simulation results demonstrate that the sparse FIR MIMO DFEs designed by the proposed scheme achieve a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse MMSE DFE. Meanwhile, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.