A Novel DOA Estimation Algorithm Based on Robust Mixed Fractional Lower-Order Correntropy in Impulsive Noise

Abstract

The estimation of direction of arrival (DOA) is paramount in the realm of practical array signal processing systems. Nevertheless, traditional estimation methods often rely heavily on the Gaussian noise assumption, rendering them ineffective in achieving high-precision estimates in environments plagued by strong impulsive noise. To address this challenge, this paper introduces a novel DOA estimation algorithm that leverages mixed fractional lower-order correntropy (MFLOCR) in the context of Alpha-stable distributed impulsive noise. Correntropy is used as a measure of the similarity of the signals, using a Gaussian function to smooth extreme values and provide greater robustness against impulsive noise. By utilizing diverse kernel lengths to jointly regulate the kernel function, the concept of correntropy is expanded and implemented in the fractional lower-order moment (FLOM) algorithm for received signals. Subsequently, the MFLOCR is derived by adjusting the resulting form of correntropy. Finally, an enhanced DOA estimation algorithm is proposed that combines the MFLOCR operator with the MUSIC algorithm, specifically tailored for impulsive noise environments. Furthermore, a proof of boundedness is provided to validate the effectiveness of the proposed approach in such noisy conditions. Simulation experiments confirmed that the proposed method outperforms existing DOA estimation methods in the context of intense impulsive noise, a low generalized signal-to-noise ratio (GSNR), and a smaller number of snapshots.

1. Introduction

Direction of arrival (DOA) estimation is a critical area in array signal processing and has wide applications in sonar, 5G communication, radar, and smart antennas [1,2,3]. However, most studies on DOA estimation have been performed assuming white Gaussian conditions of ideal additive noise. As a result, many algorithms have been developed, including super-resolution DOA estimation algorithms such as Multiple Signal Classification (MUSIC) [4,5,6] and Signal Parameter Estimation Using Rotational Invariance Techniques (ESPRIT) [7,8,9]. In practical environments, many signals and noises do not exactly follow Gaussian distributions. For example, natural lightning, switch transients in car ignitions, and accidental impacts on telephone lines can induce noise with strong impulsive characteristics [10,11,12]. The probability density functions (PDFs) of these impulsive noises decay more slowly than Gaussian distributions, resulting in thicker tails. The presence of impulsive noise can significantly degrade, or even nullify, the performances of algorithms designed under ideal noise conditions [13].

Common models used to describe impulsive noise include the Alpha-stable distribution [14], the mixed Gaussian distribution [15,16], and the Middleton Class A model [17,18]. Typically, impulsive noise has infinite variance, resulting in the absence of its second- or higher-order moments, but it does exhibit finite fractional lower-order statistics (FLOS). Therefore, several FLOS-based algorithms have been developed, such as fractional low-order moments (FLOM) [19,20], robust covariance-based (ROC) algorithms [21], and phase-controlled fractional low-order moments (PFLOM) [22]. However, when the parameters of the impulse noise are unknown, it is difficult to select the optimal settings to suppress the noise. This uncertainty limits the effectiveness of FLOS-based algorithms [23].

In recent years, correntropy has gained significant attention in robust signal processing and machine learning [24,25,26]. A higher-order statistic quantifies the two random process time structures and statistical distribution. Compared to the conventional correlation function, correntropy can extract more information, offering excellent impulsive noise suppression without requiring prior knowledge of the noise [27,28]. Zhang proposed the CRCO-MUSIC algorithm [29], which combines the MUSIC algorithm with correntropy-based correlation (CRCO). Although CRCO-MUSIC has improved the denoising performance and signal application, its accuracy still needs to be improved in cases where the signal snapshots are small and the noise is highly impulsive. The authors of [30] proposed a method using cyclic correntropy (CC) in an impulsive noise environment. The algorithm analyzes cyclic signal processing in a steady state, which has good robustness to impulsive noise.

In [31], Chen devised a correntropy-based covariance matrix (CBCM) with the MUSIC technique for source localization amidst impulsive noise. Although this method notably enhances accuracy and robustness, its real-time applicability may be hampered by considerable computational demands. In [32], Dai presented an enhanced Toeplitz approximation method incorporating a correntropy-based generalized covariance (CEGC) operator. While boasting high resolution and sidestepping array aperture loss, the method might struggle with substantial computational complexity in expansive scenarios. A bi-iterative complex fixed-point algorithm (BI-CFPA) was presented by Wang in [33] to solve the non-convex optimization issue in signal subspace estimations. This method exhibits superior efficacy across various impulsive noise conditions, especially in intense impulsive noise. Nevertheless, its optimization process complexity could be a limiting factor.

In [34], a new fractional lower-order correlation (FLOCR) algorithm was proposed, which utilizes correlation and FLOM. The algorithm outperforms other algorithms in terms of the root mean square error in highly impulsive noise environments. In [35], the authors introduced the concept of mixed correntropy to enhance the suppression of impulsive noise by mixing two Gaussian functions as kernel functions. However, when the impulsive noise intensity is high, the performance of the above DOA algorithm deteriorates due to the interference of outliers. In [36], Cai introduced an adaptive weight factor based on correntropy to suppress impulsive noise and constructed a multilayer deep neural network (DNN) framework with a real-numbered vectorized pseudo-covariance matrix as input. Despite its remarkable accuracy and resilience, this DNN-based approach can require extensive training data and computational resources.

Contributions—The invention of a novel method for DOA estimation based on mixed fractional lower-order correntropy (MFLOCR) to enhance DOA estimation in environments with impulsive noise is the primary innovation of this study. The following are the primary contributions of this study:

• Development of MFLOCR: This paper proposes a new correntropy called MFLOCR, which extends the concept of correntropy using different kernel lengths to jointly regulate the kernel function. The proposed method is based on the theory of FLOS and uses mixed correntropy with an exponential kernel instead of the traditional covariance matrix, which provides greater robustness against impulsive noise and enhances the effectiveness of the algorithm in environments with Alpha-stable distributed impulsive noise.
• MFLOCR-MUSIC algorithm: An improved DOA estimation algorithm, MFLOCR-MUSIC, was developed by combining the MFLOCR operator with the traditional MUSIC algorithm. This combination is specifically tailored for improved performance in impulsive noise environments.
• Proof of Boundedness: The proof of boundedness serves to verify the efficiency and stability of the proposed MFLOCR-MUSIC algorithm under impulsive noise and ensures its reliability in real applications.
• Superior Performance in Simulations: Simulation experiments showed that the proposed MFLOCR-MUSIC algorithm outperforms existing DOA estimation methods in various characteristic indices. It was proven to be highly accurate and robust.

The rest of this paper is structured as follows. In Section 2, we formulate the problem, introduce signal and impulse noise models, and review existing methods based on FLOS. In Section 3, we provide an overview of correntropy and propose a new operator called MFLOCR for two independent and identically distributed (i.i.d.) SαS random variables. This operator can be used to create subspace techniques when there are noise and α-stable distributed signals present. In Section 4, we present the simulation results to validate the proposed algorithm. Finally, we present conclusions in Section 5.

2. Problem Formulation

2.1. Signal Model of DOA Estimation

Taking into account the impulsive noise and half-wavelength spaced uniform linear array (ULA) of M sensors that are impinged upon by K far-field narrowband sources, the fundamental observation of the mth array element at the sampling time t may be represented as follows, using the first antenna as the reference, as seen in Figure 1:

$$x_m(t)={\displaystyle \sum _{i=1}^K{s}_i(t)e^{j2\pi \sin {\theta }_i(m-1)d/\lambda }},m=1,2,\dots ,M$$

(1)

where $s_i(t)$ and ${\theta }_i$ are the ith incident signal and the direction of arrival, respectively. $\lambda$ is the signal wavelength, and the distance d is between two sensors, where $d\le \lambda /2$, and $n_m(t)$ is the measurement noise in the mth array element followed by the symmetric Alpha-stable (SαS) distribution.

Hence, the vector of the received signals at time t is $\mathit{x}\left(t\right)={\left[{\mathit{x}}_1\left(t\right),\dots ,{\mathit{x}}_M\left(t\right)\right]}^T$. This vector can be expressed as

$$\mathit{x}\left(t\right)=\mathit{A}\left(\mathit{\theta }\right)\mathit{s}\left(t\right)+\mathit{n}\left(t\right)$$

(2)

where $\mathit{s}(t)={\left[s_1(t),\cdots ,s_K(t)\right]}^{\mathrm{T}}$ denotes the signal vector; $\mathit{A}=\left[\mathit{a}({\theta }_1),\cdots ,\mathit{a}({\theta }_K)\right]$ is the array steering matrix, where $\mathit{a}({\theta }_k)={\left[1,\cdots ,e^{-j2\pi d\sin {\theta }_k(m-1)/\lambda },\cdots ,e^{-j2\pi d\sin {\theta }_k(M-1)/\lambda }\right]}^{\mathrm{T}}$ is the steering vector of the kth signal source; and $\mathit{n}(t)={\left[n_1(t),\cdots n_M(t)\right]}^{\mathrm{T}}$ denotes the impulsive noise vector.

2.2. Impulsive Noise Model

The impulsive noise has a stronger tail compared to the Gaussian noise, making the probability of outliers higher, which when modeled as an SαS (symmetric Alpha-stable) distribution is defined as

$$\varphi (x)=\exp \{j\mu x-\gamma |x{|}^{\alpha }[1+j\beta \mathrm{sgn}(x)\omega (x,\alpha )]\}$$

(3)

$$\omega (x,\alpha )=\left\{\begin{array}{cc}\tan (\pi \alpha /2)\hfill & \hfill \alpha \ne 1\\ (2/\pi )\log \left|x\right|\hfill & \hfill \alpha =1\end{array}\right.$$

(4)

$$\mathrm{sgn}(x)=\left\{\begin{array}{cc}1\hfill & \hfill x>0\\ 0\hfill & x=0\hfill \\ -1\hfill & \hfill x<0\end{array}\right.$$

(5)

where $\alpha \in (0,2]$ describes the characteristic exponent of the impulsive noise intensity. When $\alpha =2$, the distribution reduces to Gaussian. The symmetry parameter $\beta \in [-1,1]$ indicates the degree of skewness of the distribution. When $\beta =0$, the distribution is symmetric and is called the SαS distribution. The scale parameter $\gamma \in \left(0,+\infty \right)$ is similar to the variance of a regular distribution. $\mu \in \left(-\infty ,+\infty \right)$ is the positional parameter. The probability density functions of the SαS distribution for different characteristic exponents are shown in Figure 2. In this study, we simulated the impulsive noise environment using SαS distributions.

2.3. Correntropy

As a novel statistical measure, correntropy is employed to quantify the spatial structure and statistical distribution of two random processes. It possesses the ability to extract richer information and mitigate the impact of outliers resulting from impulsive noise. Consequently, correntropy introduces a novel metric within the sample space. The correntropy can be expressed as

$$V_{\sigma }(X,Y)=\mathrm{E}[{\kappa }_{\sigma }(X-Y)]$$

(6)

where $\mathrm{E}[\cdot ]$ and ${\kappa }_{\sigma }(\cdot )$ are the expectation and kernel functions with a width of $\sigma$, respectively. In practice, the prior probabilities are unknown. Therefore, the correntropy of random variables can only be estimated from a limited number of observations $\left\{{\left(x_i,y_i\right)}_{i=1}^N\right\}\left(i=1,2,\dots ,N\right)$, which can be expressed as

$${\widehat{V}}_{\sigma }(X,Y)=\frac{1}{N}{\displaystyle \sum _{i=1}^N{\kappa }_{\sigma }(x_i-y_i)}$$

(7)

where N is the number of observations, and the Gaussian kernel function is the most often utilized kernel function in correntropy. It is effective in suppressing outliers [37] and can be written as

$${\kappa }_{\sigma }(X,Y)=G(\epsilon )=\exp (-\frac{{\epsilon }^2}{2{\sigma }^2})$$

(8)

A Taylor series expansion of (8) is further described as

$$V_{\sigma }(X,Y)=\frac{1}{\sqrt{2\pi }\sigma }{\displaystyle \sum _{n=1}^{+\infty }\frac{{(-1)}^n}{2^{n}n!{\sigma }^{2n}}} \mathrm{E}[{(\epsilon )}^{2n}]$$

(9)

where $\epsilon =X-Y$ is the error variable. From the above equation, it can be known that the correntropy contains all the information of the second- and even higher-order moments.

Furthermore, the correntropy can induce a distance measure known as the correntropy-induced metric (CIM), denoted as

$$\mathrm{CIM}(X,Y)=sqrt\left\{\mathrm{E}\left[\kappa (0)-{\kappa }_{\sigma }(X-Y)\right]\right\}$$

(10)

where $\kappa (0)=1/\sqrt{2\pi }\sigma$. The CIM exhibits mixed-norm properties that are entirely controlled by the width of the kernel [38]. In order to estimate the DOA of the source precisely, the kernel width $\sigma$ is varied based on the intensity of impulsive noise.

3. The Proposed MFLOCR-MUSIC Algorithm

3.1. FLOM-MUSIC Algorithm

The statistical measures of signals contain abundant information about signal characteristics. In traditional signal processing, second-order or higher-order statistical measures are often used to extract signal features. However, in an impulsive noise environment, only statistical measures less than the characteristic exponent $\alpha (\alpha \le 2)$ exist for noise that follows the SαS distribution. This causes no second-order or higher-order statistical measurements of the received signals to exist, leading to degraded or even invalid DOA estimation results. In [19], a FLOM-based MUSIC estimation algorithm was proposed for DOA estimation in impulsive noise environments with an SαS distribution. The FLOM matrix is written as follows:

$${\mathit{R}}_{\mathrm{FLOM}}=\mathrm{E}\left[X{Y}^{<p-1>}\right],1<p<\alpha$$

(11)

where X and Y are two random variables that follow the SαS distribution. The operator ${\left(\cdot \right)}^{<p>}$ is defined as ${\left|\left(\cdot \right)\right|}^{p-1}{\left(\cdot \right)}^{\ast }$. The (I,j)th element of the FLOM matrix is defined as

$${\mathit{R}}_{\mathrm{FLOM}}(i,j)=\mathrm{E}\left[x_i(t){\left|x_j(t)\right|}^{p-2}{x}_j^{\ast }(t)\right],1<p<\alpha \le 2$$

(12)

where $x_i\left(\cdot \right)$ is the ith row of the element; if p = 2, the FLOM matrix turns into a conventional correlation. With the EVD of ${\mathit{R}}_{\mathrm{FLOM}}$, we obtain

$${\mathit{R}}_{\mathrm{FLOM}}={\mathit{U}}_S{\mathit{\Sigma }}_S{\mathit{U}}_S^H+{\mathit{U}}_N{\mathit{\Sigma }}_N{\mathit{U}}_N^H$$

(13)

where ${\mathit{\Sigma }}_S$ and ${\mathit{\Sigma }}_N$ are the diagonal matrices that have the remaining (M − K) lesser eigenvalues and the K biggest eigenvalues of ${\mathit{R}}_{\mathrm{FLOM}}$, respectively. ${\mathit{U}}_S$ and ${\mathit{U}}_N$ are the matrices composed of eigenvectors corresponding to ${\mathit{\Sigma }}_S$ and ${\mathit{\Sigma }}_N$.

$$P_{\mathrm{FLOM}-\mathrm{MUSIC}}=\frac{1}{{\mathit{a}}^{\mathrm{H}}(\mathsf{\theta }){\mathit{U}}_N{\mathit{U}}_N^{\mathrm{H}}\mathit{a}(\mathsf{\theta })}$$

(14)

Then, the K greatest peaks of the spatial spectrum might be found in order to estimate the DOAs.

3.2. MFLOCR-MUSIC Algorithm

In order to further improve the suppression effect of the correntropy on the effect of the outliers caused by impulsive noise, a new estimator named MFLOCR-MUSIC is proposed that can be applied in SαS distributions for a wide range of $1<\alpha <2$.

Mixed correntropy is more effective in capturing the diverse interference scales introduced by impulsive noise, which uses two Gaussian functions with different kernel widths. The length of the kernel can influence the sensitivity of the system to transient changes in impulsive noise. A shorter kernel length may be sensitive to transient changes in impulsive noise, whereas a longer kernel length may assist in identifying long-term trends in the background signal and therefore facilitate a more effective separation of noise and signal. The utilization of kernel functions with varying kernel lengths enables a more effective differentiation of transient alterations induced by impulsive noise.

By adjusting the kernel length with different parameters, an analytical model that is more insensitive to impulsive noise can be constructed, which further improves the accuracy of DOA estimation under impulsive noise. This model can be considered a promotion of the original correlation, which contains more statistical information, defined as

$$M(X,Y)=\mathrm{E}[{\rho }_1{G}_{\sigma 1}(e)+{\rho }_2{G}_{\sigma 2}(e)]$$

(15)

The mixture factor satisfies conditions ${\rho }_1\le {\rho }_2$ and ${\rho }_1+{\rho }_2=1$. If ${\rho }_1=0$, it becomes a single kernel function with an entropy of either $G_{\sigma 1}(\cdot )$ or $G_{\sigma 2}(\cdot )$ and ${\sigma }_1\le {\sigma }_2$ is the width of the kernel. As a more flexible similarity measure, appropriate mixing coefficients $\rho$ can improve the performance of mixed correntropy. Given that the Gaussian kernel function performs well in reducing impulsive noise, by substituting (8) into (15), we have

$$M(X,Y)=\mathrm{E}[{\rho }_1\exp (-\frac{e^2}{2{{\sigma }_1}^2})+{\rho }_2\exp (-\frac{e^2}{2{{\sigma }_2}^2})]$$

(16)

It is seen that, as the kernel length increases, the higher-order moment information decays faster, resulting in the second-order statistic becoming the primary component. When $n=1$, the correntropy is proportional to $\mathrm{E}[{(X-Y)}^{2n}]$, which indicates that all conventional correlation information is contained in the correntropy. In contrast to conventional correntropy, mixed correntropy allows the simultaneous consideration of signal features at different time scales which improves the performance and stability of the DOA estimation and reduces the risk of overfitting.

As a result, to effectively mitigate outliers stemming from impulsive noise and enhance the DOA estimation, a novel estimator named MFLOCR was introduced. This estimator builds upon the foundation of the FLOM algorithm and is motivated by the concept of mixed correntropy mentioned above. The matrix can be defined as

$${\mathit{R}}_{\mathrm{MFLOCR}}=\mathrm{E}\left[{\rho }_1(\exp \left(-\frac{\left|X-Y^{\ast }\right|}{2{\sigma }_1^2}\right)X{Y}^{<p-1>})+{\rho }_2\exp \left(-\frac{\left|X-Y^{\ast }\right|}{2{\sigma }_2^2}\right)Y^{\ast }{X^{\ast }}^{<p-1>}\right]$$

(17)

where $x_i\left(\cdot \right)$ is the ith row of the element. The (I,j)th element $Q_{i,j}$ of ${\mathit{R}}_{\mathrm{MFLOCR}}$ is denoted as

$$Q_{i,j}=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left[{\rho }_1\exp \left(-\frac{\left|x_i(t)-x_j^{\ast }(t)\right|}{2{\sigma }_1^2}\right)x_i(t)x_j^{<p-1>}(t)+{\rho }_2\exp \left(-\frac{\left|x_i(t)-x_j^{\ast }(t)\right|}{2{\sigma }_2^2}\right)x_j^{\ast }(t){x_i^{\ast }}^{<p-1>}(t)\right]}$$

(18)

Based on the proposed algorithm’s exceptional performance, we replace the covariance matrix in MUSIC with ${\mathit{R}}_{\mathrm{MFLOCR}}$ to estimate the DOAs.

3.3. The Implementation of MFLOCR–MUSIC

In this section, the conventional subspace-based methods are extended from Gaussian-distributed noise to SαS-distributed noise environments by applying MFLOCR. We propose a new algorithm, MFLOCR-MUSIC, which combines MFLOCR with the subspace decomposition of the MUSIC algorithm. The algorithm used to estimate DOAs in an environment with impulsive noise consists of the following main steps:

Step 1. Utilizing (17) and (18), the matrix ${\mathit{R}}_{\mathrm{MFLOCR}}$ with dimensions of M × M is constructed from the data with the snapshot number N.

Step 2. Execute the eigenvalue decomposition (EVD) of the covariance ${\mathit{R}}_{\mathrm{MFLOCR}}$ to obtain the M × (M − K) dimensional noise subspace matrix ${\mathit{U}}_n$:

$${\mathbf{U}}_n=\left[{\mathit{u}}_{K+1},{\mathit{u}}_{K+2},\cdots ,{\mathit{u}}_M\right]$$

(19)

where $\left\{{\mathit{u}}_{K+1},{\mathit{u}}_{K+2},\cdots ,{\mathit{u}}_M\right\}$ is the eigenvector corresponding to the M − K smaller eigenvalues.

Step 3. Compute the spatial spectrum of the MFLOCR-MUSIC algorithm:

$$P_{\mathrm{MFLOM}-\mathrm{MUSIC}}=\frac{1}{{\mathit{a}}^{\mathrm{H}}(\mathsf{\theta }){\mathit{U}}_N{\mathit{U}}_N^{\mathrm{H}}\mathit{a}(\mathsf{\theta })}$$

(20)

Step 4. Search for the K peaks of $P_{\mathrm{MFLOCR}-\mathrm{MUSIC}}(\mathsf{\theta })$ to estimate the DOAs of the K sources $\left\{{\theta }_1,{\theta }_2,\cdots ,{\theta }_K\right\}$.

3.4. The Boundedness of MFLOCR

To ensure that the MFLOCR can be effectively applied to the DOA estimation problem under impulsive noise, ${\mathit{R}}_{\mathrm{MFLOCR}}$ needs to be bounded. Assuming that X and Y, which are two independent random variables with similar distributions and characteristic indices $1<\alpha \le 2$, follow the SαS distribution, the boundedness of ${\mathit{R}}_{\mathrm{MFLOCR}}$ is proved as follows:

$$\begin{array}{ll}{\mathit{R}}_{\mathrm{MFLOCR}}& \le \mathrm{E}\left[\left|\exp \left(-\frac{\left|\mathrm{X}-{\mathrm{Y}}^{\ast }\right|}{2{\mathsf{\sigma }}_1^2}\right)\mathrm{X}{\mathrm{Y}}^{<\mathrm{p}-1>}\right|+\left|\exp \left(-\frac{\left|\mathrm{X}-{\mathrm{Y}}^{\ast }\right|}{2{\mathsf{\sigma }}_2^2}\right){\mathrm{Y}}^{\ast }{{\mathrm{X}}^{\ast <}}^{\mathrm{p}-1>}\right|\right]\\ & \le \mathrm{E}\left[\left|\mathrm{X}{\mathrm{Y}}^{<\mathrm{p}-1>}\right|+\left|{\mathrm{Y}}^{\ast }{{\mathrm{X}}^{\ast <}}^{\mathrm{p}-1>}\right|\right]\\ & \le \mathrm{E}\left[\left|\mathrm{X}\right|\left|{\mathrm{Y}}^{<\mathrm{p}-1>}\right|+\left|{\mathrm{Y}}^{\ast }\right|\left|{{\mathrm{X}}^{\ast <}}^{\mathrm{p}-1>}\right|\right]\\ & \le \mathrm{E}\left[\left|\mathrm{X}\right|{\left|\mathrm{Y}\right|}^{\mathrm{p}-2}\left|{\mathrm{Y}}^{\ast }\right|+\left|{\mathrm{Y}}^{\ast }\right|{\left|{\mathrm{X}}^{\ast }\right|}^{\mathrm{p}-2}\left|\mathrm{X}\right|\right]\\ & =\mathrm{E}\left[\left|\mathrm{X}\right|{\left|\mathrm{Y}\right|}^{\mathrm{p}-1}+\left|\mathrm{Y}\right|{\left|\mathrm{X}\right|}^{\mathrm{p}-1}\right]\end{array}$$

(21)

For when the characteristic parameter satisfies $1<\alpha <2$, it is stated in [39] that the expectation of ${\left|X\right|}^p$ can be defined as

$$\mathrm{E}[{\left|X\right|}^p]=\left\{\begin{array}{cc}\hfill C(p,\alpha ){\gamma }^{\frac{p}{\alpha }}\hfill & -1<\mathrm{p}<\alpha \hfill \\ \hfill \infty \hfill & \mathrm{p}\ge \alpha \hfill \end{array}\right.$$

(22)

where

$$C(p,\alpha )=2^{p+1}\frac{\mathsf{\Gamma }(\frac{p+1}{2})\mathsf{\Gamma }(-\frac{p}{\alpha })}{\alpha \sqrt{\pi }\mathsf{\Gamma }(-\frac{p}{2})}$$

(23)

where $\mathsf{\Gamma }(x)={\displaystyle {\int }_0^{\infty }{t}^{x-1}\exp (-t)dt}$ is the function of Gamma. Suppose that $h_1=\frac{\mathsf{\Gamma }((p+1)/2)\mathsf{\Gamma }(-p/\alpha )}{\alpha \sqrt{\pi }\mathsf{\Gamma }(-p/2)}$.

In practice, the fractional low-order moment parameter $\xi$ satisfies $-1<\xi <(p-1)$ and is infinitely close to −1. Since the function $h_1$ is monotonically decreasing and $\xi <(p-1)<1$, this leads to

$$h_1\le \frac{\mathsf{\Gamma }(\frac{\xi +1}{2})\mathsf{\Gamma }(-\frac{\xi }{\alpha })}{\mathsf{\Gamma }(-\frac{\xi }{2})}<\infty$$

(24)

For $\mathrm{E}\left[\left|X\right|\right]$, we can prove its boundedness by replacing the parameter p with 1 in the above equation:

$$\mathrm{E}[\left|X\right|]\le -\frac{2{\gamma }^{\frac{1}{\alpha }}}{\alpha \pi }\mathsf{\Gamma }(-\frac{1}{\alpha })$$

(25)

Combining this with (24) and (25), we can obtain

$$\mathrm{E}[\left|X{Y}^{<p-1>}\right|]\le \frac{-2^{\xi +2}{\gamma }^{\frac{\xi +1}{\alpha }}\mathsf{\Gamma }(-\frac{1}{\alpha })\mathsf{\Gamma }(-\frac{\xi }{\alpha })\mathsf{\Gamma }(\frac{\xi +1}{2})}{{\alpha }^2{\pi }^{\frac{3}{2}}\mathsf{\Gamma }(-\frac{\xi }{2})}<\infty$$

(26)

The boundedness of $\mathrm{E}[Y^{\ast }{X^{\ast <}}^{p-1>}]$ can be proved in the same way. Based on the above, ${\mathit{R}}_{\mathrm{MFLOCR}}$ is bounded. The boundedness proof of MFLOCR above provides a theoretical basis for its application to impulsive noise DOA estimation.

4. Simulation

In this study, a comparative analysis of the proposed method against PFLOM-MUSIC [22], CRCO-MUSIC [29], and FLOCR-MUSIC [34] was conducted through numerous simulations to demonstrate the efficacy of the algorithm. Specifically, a scenario was considered where two far-field narrowband signals possessing equal power and a frequency of 1 GHz, were incident on a linear array comprising M = 8 sensors spaced at half-wavelength intervals. The noise in this scenario followed the SαS distribution. The DOAs were ${\theta }_1=-{20}^{\circ }$ and ${\theta }_1={45}^{\circ }$. The widths of the correntropy kernel were ${\sigma }_1=0.5$ and ${\sigma }_2=2$, and the mixing coefficients of the mixed correntropy were ${\rho }_1=0.4$ and ${\rho }_2=0.6$. In our experiment, we set the parameter p to 1.2.

The GSNR was used in the scheme of impulsive noise, in place of the signal-to-noise ratio (SNR), as follows:

$$\mathrm{GSNR}=10\lg (\mathrm{E}({\left|s(t)\right|}^2)/\gamma )$$

(27)

The following simulation was subjected to a comprehensive evaluation framework comprising 500 independent Monte Carlo trials. The performance is assessed using two key metrics: the probability of successful resolution and the RMSE. Specifically, the ratio of successfully resolved cases to the total number of Monte Carlo trials was used to calculate the probability of successful detection. The successful resolution was defined as $\left|{\widehat{\theta }}_i(k)-{\theta }_i\right|\le 1^{\circ } ,i=1,2$, where in the kth Monte Carlo trial, ${\widehat{\theta }}_i(k)$ represents the predicted DOA of the ith target. The RMSE is defined as

$$\mathrm{RMSE}=\frac{1}{K}{\displaystyle \sum _{i=1}^k\sqrt{\frac{1}{L}{\left({\widehat{\theta }}_i(l)-{\theta }_i\right)}^2}}$$

(28)

where L is the number of Monte Carlo trials, and K is the number of signals. The results from the simulations were acquired using a personal computer equipped with an Intel Core i5 @ 3.0 GHz CPU and MATLAB R2021b.

4.1. Effect of GSNR

The purpose of this experimental study was to investigate the algorithm performance in relation to the GSNR. The algorithm was tested from −4 dB to 8 dB to determine the effect of changing the GSNR on the method’s performance. The snapshot number was fixed at N = 200, and the characteristic exponent of $\alpha =1.5$ was kept constant during the analysis.

As depicted in Figure 3a, the RMSE exhibits a gradual decline as the GSNR increases, indicating a significant improvement in the performance of all algorithms. Notably, when the GSNR surpasses 2 dB, the RMSEs of all algorithms fall below 0.5. However, the proposed algorithm consistently achieves a lower RMSE compared to the other algorithms, demonstrating its superior performance. Turning to Figure 3b, it is evident that with an increasing GSNR, the probability of successful resolution also rises, indicating an increase in the accuracy of these algorithms. Notably, when the GSNR reaches or exceeds 0 dB, the proposed algorithm achieves a probability of successful resolution over 0.9. This indicates that the MFLOCR-MUSIC algorithm outperforms other methods and exhibits the highest accuracy.

4.2. Effect of the Number of Snapshots

The objective of this experimental investigation was to assess the influence of an increasing snapshot number, ranging from 100 to 800, on the performance of the algorithms, while maintaining a constant characteristic exponent $\alpha =1.5$ and a GSNR = 5 dB.

As shown in Figure 4a, as the snapshot number increases, the observation matrix becomes more comprehensive, containing more useful information. This results in a continuous decline in the RMSE for all four direction estimation methods. Notably, the proposed novel method exhibits the best performance among all the simulated algorithms.

In Figure 4b, it is evident that with an increasing snapshot number, the proposed algorithm achieves significant improvements in accuracy and tends to stabilize compared to the other three algorithms. Furthermore, it is particularly effective at lower snapshot counts, underscoring the advantages of the proposed algorithm.

4.3. Characteristic Exponent

With a fixed GSNR of 5 dB and N = 200 snapshots, the aim of this series of tests was to investigate the impact of the characteristic exponent on the algorithm’s performance. As the characteristic exponent increases, the effect of impulsive noise diminishes.

Figure 5a illustrates that all algorithms perform better as the characteristic exponent increases. Notably, the RMSE of the proposed algorithm consistently remained below 0.6, and the algorithm outperformed the other three algorithms.

Figure 5b shows that for all algorithms, the probability of a successful resolution increases when the characteristic exponent increases. However, the proposed algorithm demonstrates superior performance in the presence of strong impulsive noise. Specifically, the accuracy of the suggested approach had already reached 0.9 when the characteristic exponent goes above a certain threshold, whereas the probability of successful resolution for the other algorithms remains below 0.6 with $\alpha =1.0$.

5. Conclusions

In order to overcome the performance drawbacks of conventional estimation techniques in environments with impulsive noise, this work introduced the MFLOCR-MUSIC algorithm. The proposed algorithm leverages the mixed correntropy of FLOS, effectively replacing the covariance matrix in conventional approaches and providing robust suppression of impulsive noise. The MUSIC method performs exceptionally well when used for DOA estimation with the proposed correntropy. Notably, it performs exceptionally well even in situations with few snapshots and severe impulsive noise, and it does not require previous knowledge of the steady distribution noise. A thorough analysis showed that the suggested MFLOCR-MUSIC algorithm works better than alternative techniques in a number of ways, achieving a high accuracy and low RMSE. The experimental results demonstrate that the MFLOCR-MUSIC algorithm outperforms the other three algorithms tested. Under low GSNR conditions, the MFLOCR-MUSIC algorithm achieved an RMSE improvement from 0.1 to 0.3 and an estimation accuracy improvement from 5% to 15% compared to the other algorithms. In scenarios with a low snapshot number, the MFLOCR-MUSIC algorithm showed an RMSE improvement from 0.05 to 0.15 and an estimation accuracy improvement from 1% to 3%. Under conditions of intense impulsive noise, the MFLOCR-MUSIC algorithm exhibited an RMSE improvement from 0.05 to 0.4 and an estimation accuracy improvement from 17% to 40% over the competing methods.

In summary, the MFLOCR-MUSIC algorithm significantly enhances the DOA estimation performance in challenging environments characterized by a low GSNR, a small snapshot number, and strong impulsive noise.