Scalar-on-Function Relative Error Regression for Weak Dependent Case

Abstract

Analyzing the co-variability between the Hilbert regressor and the scalar output variable is crucial in functional statistics. In this contribution, the kernel smoothing of the Relative Error Regression (RE-regression) is used to resolve this problem. Precisely, we use the relative square error to establish an estimator of the Hilbertian regression. As asymptotic results, the Hilbertian observations are assumed to be quasi-associated, and we demonstrate the almost complete consistency of the constructed estimator. The feasibility of this Hilbertian model as a predictor in functional time series data is discussed. Moreover, we give some practical ideas for selecting the smoothing parameter based on the bootstrap procedure. Finally, an empirical investigation is performed to examine the behavior of the RE-regression estimation and its superiority in practice.

1. Introduction

This paper focuses on nonparametric prediction in Hilbertian statistics, which is an intriguing area of research within nonparametric Hilbertian statistics. Various approaches exist for modeling the relationship between the input Hilbertian variable and the output real variable. Typically, this relationship is modeled through a regression model, where the regression operators are estimated using the least square error. However, this rule is not relevant for some practical cases. Instead, we consider in this paper the relative square error. The primary advantage of this regression is the possibility of reducing the effect of the outliers. This kind of relative error is used as a performance measure in many practical situations, namely in time series forecasting. The literature on the subject of nonparametric analysis is limited. Most existing works consider a parametric approach. In particular, Narula and Wellington [1] were the first to investigate the use of the relative square error in the estimation method. For practical purposes, relative regression has been applied in areas such as medicine by Chatfield [2] and financial data by Chen et al. [3]. Yang and Ye [4] considered the estimation by RE-regression in multiplicative regression models. Jones et al. [5] also focused on the use of this model but with the nonparametric estimation method and stated the convergence of the local linear estimator obtained by the relative error as a loss function. The RE-regression estimation has been deeply studied for time series data in the last few years, specifically by Mechab and Laksaci [6] for the quasi-associated time series, and Attouch et al. [7] for the spatial process. The nonparametric Hilbertian RE-regression was first developed by Demongeot et al. [8], who focused on strong consistency and gave the asymptotic law of the RE-regression. To summarize, functional statistics is an attractive subject in mathematical statistics; the reader may refer to some survey papers, such as [9,10,11,12,13,14,15], for recent advances and trends in functional data analysis and/or functional time series analysis.

In this article, we focus on the Hilbertian RE-regression for weak functional time series data. In particular, the correlation of our observations is modeled by using the quasi-association assumption. This correlation includes many important Hilbertian time series cases, such as the linear and Gaussian processes, as well as positive and negative associated processes. Our ambition in this contribution is to build a new Hilbertian predictor in the Hilbertian time series. This predictor is defined as the ratio of the first and the second inverted conditional moments. We use this explicit expression to construct two estimators based on the kernel smoothing and/or k-Neighbors Number (kNN). We prove a strong consistency of the constructed estimator, which provides good mathematical support for its use in practice. Thus, treating the functional RE-regression by the kNN method under quasi-associated assumption is a great theoretical development which requires nonstandard mathematical tools and techniques. On the one hand, it is well known that the establishment of the asymptotic property in the kNN method is more difficult than the classical kernel estimation due to the random feature of the bandwidth parameter. On the other hand, our weak structure of the functional time series data requires additional techniques and mathematical tools alternative to those used in the mixing case. Clearly, this theoretical development is very useful in practice because the kNN estimator is more accurate than the kernel method and the quasi-association structure is sufficiently weak to cover a large class of functional time series data. Furthermore, the applicability of this estimator is highlighted by giving some selection procedures to determine the parameters involved in the estimator. Then, real data are used to emphasize the superiority and impact of this contribution in practice.

This paper is organized as follows. We introduce the estimation algorithms in Section 2. The required conditions, as well as the main asymptotic results, are demonstrated in Section 3. We discuss some selectors for the smoothing parameter in Section 4. The constructed estimator’s performance over the artificial data is evaluated in Section 5. Finally, we state our conclusion in Section 6 and demonstrate proofs of the technical results in the Appendix A.

2. The Re-Regression Model and Its Estimation

As discussed in the introduction, we aim to evaluate the relationship between an exogenous Hilbertian variable X and a real endogenous variable Y. Specifically, the variables ${\displaystyle (X,Y)}$ belong in $\mathcal{H}\times \mathrm{I}\mathrm{R}$. The set $\mathcal{H}$ constitutes a separable Hilbert space. We assume that the norm $\parallel ·\parallel$ in $\mathcal{H}$ is associated with the inner product $\langle ·,·\rangle$. Furthermore, we define on $\mathcal{H}$ a complete orthonormal basis ${\left(e_k\right)}_{k\ge 1}$. In addition, we suppose that Y is strictly positive, and we suppose that the Hilbertian operators $\mathrm{I}\mathrm{E}[Y^{-1}|X]$ and $\mathrm{I}\mathrm{E}[Y^{-2}|X]$ exist and are, almost surely, finite. The RE-regression is defined by

$$\begin{array}{c}\hfill R\left(x\right)=arg\underset{\theta }{min}\mathrm{I}\mathrm{E}\left({\left(\frac{Y-\theta }{Y}\right)}^2|X=x\right).\end{array}$$

(1)

By differentiating with respect to $\theta$, we prove that

$$R\left(x\right)=\frac{\mathrm{I}\mathrm{E}\left[Y^{-1}|X=x\right]}{\mathrm{I}\mathrm{E}\left[Y^{-2}|X=x\right]}.$$

(2)

Clearly, the RE-regression $R(·)$ is a good alternative to the traditional regression, in the sense that, the traditional regression, based on the least square error, treats all variables with equal weight. This is inadequate when the observations contain some outliers. Thus, the traditional regression can lead to irrelevant results in the presence of outliers. Thus, the main advantage of the RE-regression $R(·)$ compared to the traditional regression is the possibility to reduce the effect of the outliers (see Equation (1)). So, we can say that the robustness feature is one of the main advantages of the RE-regression. Additionally, unlike the classical robust regression (the M-egression), the RE-regression is very easy to implement in practice. It has an explicit definition based on the ratio of the first and the second inverted conditional moments (see Equation (2)).

Now, consider ${(X_i,Y_i)}_{i-1,\dots ,n}$ strictly stationary observations, as copies of a couple ${\displaystyle (X,Y)}$. The Hilbertian time series framework of the present contribution is carried out using the quasi-association setting (see Douge [16] for the definition of the Hilbert space). We use the kernel estimators of the inverse moments $\mathrm{I}\mathrm{E}[Y^{-1}|X]$ and $\mathrm{I}\mathrm{E}[Y^{-2}|X]$ as conditional expectations of $Y^{-\gamma }$ ($\gamma =1,2$), given $X=x$, to estimate $R\left(x\right)$ by

$$\tilde{R}\left(x\right)=\frac{{\sum }_{i=1}^n{Y}_i^{-1}K\left(\frac{\parallel x-X_i\parallel }{h_n}\right)}{{\sum }_{i=1}^n{Y}_i^{-2}K\left(\frac{\parallel x-X_i\parallel }{h_n}\right)},$$

(3)

where $h_n$ is a positive sequence of real numbers, and K is a real-function so-called kernel. The choice of $h_n$ is the determining issue of the applicability of the estimator $\tilde{R}$. A common solution is to utilize kernel smoothing with the kNN estimation, for which

$$\widehat{R}\left(x\right)=\frac{{\sum }_{i=1}^n{Y}_i^{-1}K\left(\frac{\parallel x-X_i\parallel }{A_n^k}\right)}{{\sum }_{i=1}^n{Y}_i^{-2}K\left(\frac{\parallel x-X_i\parallel }{A_n^k}\right)},$$

(4)

where

$$A_n^k\left(x\right)=min\left\{{\displaystyle a_n>0;\sum _{i=1}^{n}1{\mathrm{I}}_{B(x,a_n)}\left(X_i\right)=k}\right\},$$

where $B(x,a_n)$ is an open ball of radius $a_n>0$ centered x. In $\widehat{R}$, the smoothing parameter is the number k. Once again, the selection of k is crucial.

3. The Consistency of the Kernel Estimator

We demonstrate the almost complete convergence of $\tilde{R}(·)$ to $R(·)$ at the fixed point x in $\mathcal{H}$. Hereafter, $N_x$ is the given neighborhood of x, and $C_1,C_2,C_{,}\dots$ are strictly positive constants. In the sequel, we put $K_i\left(x\right)=K(h_n^{-1}\parallel x-X_i\parallel ),i=1,\dots ,n,$ $R_{\gamma }\left(u\right)=\mathrm{I}\mathrm{E}\left[Y^{-\gamma }|X=u\right]$, and $\gamma =1,2$, and we denote this by

$${\displaystyle {\lambda }_k:=\underset{s\ge k}{sup}\sum _{|i-j|\ge s}\sum _{k=1}^{\infty }\sum _{l=1}^{\infty }|Cov(X_j^k,X_i^k)|+\sum _{k=1}^{\infty }|Cov(Y_j,X_i^k)|+\sum _{l=1}^{\infty }|Cov(X_j^l,Y_i)|+|Cov(Y_j,Y_i)|,}$$

where $X_i^k:=<X_i,e_k>$. Moreover, we assume the following conditions:

(D1)

For all $d>0$${\displaystyle {\varphi }_x\left(d\right):=\mathrm{I}\mathrm{P}(X\in B(x,d))>0}$, and ${\displaystyle \underset{d\to 0}{lim}{\varphi }_x\left(d\right)=0}$.

(D2)

For all $(x_1,x_2)\in {\mathcal{N}}_x^2$,

$$|R_{\gamma }\left(x_2\right)-R_{\gamma }\left(x_1\right)|\le C{d}^{k_{\gamma }}(x_2,x_1)\mathrm{for}{k}_1,k_2>0.$$

(D3)

The covariance coefficient is ${\left({\lambda }_k\right)}_{k\in \mathrm{I}\mathrm{N}}$, such that ${\lambda }_k\le C{e}^{-ak},a>0,C>0.$

(D4)

K is the Lipschitzian kernel function, which has $(0,1)$ as support and satisfies the following:

$$0<C_2\le K(·)\le C_3<\infty .$$

(D5)

The endogenous variable Y gives:

$$\mathrm{I}\mathrm{E}\left[\exp (|Y{|}^{-{\gamma }_1})\right]<C\mathrm{and}\forall i\ne j\mathrm{I}\mathrm{E}\left(|Y_i^{-{\gamma }_2}{Y}_j^{-{\gamma }_3}|\mid X_i,X_j\right)\le C^{\prime }<\infty ,{\left({\gamma }_i\right)}_{i=1,2,3}=1,2.$$

(D6)

For all ${\displaystyle i\ne j}$,

$$0<\underset{i\ne j}{sup}\mathrm{I}\mathrm{P}\left[(X_j,X_i)\in B(x,d)\times B(x,d)\right]=\le C({\varphi }_x^{\frac{a+1}{a}}\left(d\right)).$$

(D7)

There exist $\xi \in (0,1)$ and ${\xi }_1\in (0,1-\xi ),x{i}_2\in (0,a-1), \mathrm{such\; that}$

$$\frac{log{n}^5}{n^{1-\xi -{\xi }_1}}\le {\varphi }_x\left(h_n\right)\le \frac{1}{log{n}^{1+{\xi }_2}}.$$

• Brief comment on the conditions: Note that the required conditions stated above are standard in the context of Hilbertian time series analysis. Such conditions explore the fundamental axes of this contribution. The functional path of the data is explored through the condition (D1), the nonparametric nature of the model is characterized by (D2), and the correlation degree of the Hilbertian time series is explored by conditions (D3) and (D6). The principal parameters used in the estimator, namely the kernel and the bandwidth parameter, are explored through the conditions, (D4), (D5), and (D6). Such conditions are of a technical nature. They allow for retaining the usual convergence rate in nonparametric Hilbertian time series analysis.

Theorem 1.

Based on the conditions (D1)–(D7), we get

$$|\tilde{R}\left(x\right)-R\left(x\right)|=O\left(h_n^{k_0}\right)+O_{a.co.}\left(\sqrt{\frac{logn}{n^{1-\xi }{\varphi }_x\left(h_n\right)}}\right),$$

(5)

where $k_0=min(k_1,k_2)$.

Proof of Theorem 1.

Firstly, we write

$$\tilde{R}\left(x\right)=\frac{\widetilde{R_N}\left(x\right)}{\widetilde{R_D}\left(x\right)},$$

where

$$\widetilde{R_N}\left(x\right)=\frac{1}{n\mathrm{I}\mathrm{E}\left[K\right(h_n^{-1}\parallel x-X_1\parallel \left)\right]}\sum _{i=1}^n{Y}_i^{-1}K(h_n^{-1}\parallel x-X_i\parallel ),$$

$$\mathrm{and}\widetilde{R_D}\left(x\right)=\frac{1}{n\mathrm{I}\mathrm{E}\left[K\right(h_n^{-1}\parallel x-X_1\parallel \left)\right]}\sum _{i=1}^n{Y}_i^{-2}K(h_n^{-1}\parallel x-X_i\parallel ).$$

We use a basic decomposition (see Demongeot et al. [8] to deduce that Theorem 1 is a consequence result of the below lemmas). □

Lemma 1.

Using the conditions (D1) and (D3)–(D7), we get

$$|\widetilde{R_N}\left(x\right)-\mathrm{I}\mathrm{E}\widetilde{R_N}\left(x\right)|=O_{a.co.}\left(\sqrt{\frac{logn}{n^{1-\xi }{\varphi }_x\left(h_n\right)}}\right),$$

and

$$|\widetilde{R_D}\left(x\right)-\mathrm{I}\mathrm{E}\widetilde{R_D}\left(x\right)|=O_{a.co.}\left(\sqrt{\frac{logn}{n^{1-\xi }{\varphi }_x\left(h_n\right)}}\right).$$

Lemma 2.

Under conditions (D1),(D2), (D4), and (D7), we get

$$\left|\mathrm{I}\mathrm{E}\widetilde{R_N}\left(x\right)-R_1\left(x\right)\right|=O\left(h_n^{k_1}\right),$$

and

$$\left|\mathrm{I}\mathrm{E}\widetilde{R_D}\left(x\right)-R_2\left(x\right)\right|=O\left(h_n^{k_2}\right).$$

Corollary 1.

Using the conditions of Theorem 1, we obtain

$$\sum _{n=1}^{\infty }P\left(\widetilde{R_D}\left(x\right)<\frac{R_2\left(x\right)}{2}\right)<\infty .$$

Next, to prove the consistency of $\widehat{R}\left(x\right)$, we adopt the following postulates:

(K1)

$K(·)$ has a bounded derivative on $[0,1]$;

(K2)

The function ${\varphi }_x(·)$, such that

$${\displaystyle {\varphi }_x\left(a\right)=\varphi \left(a\right)L\left(x\right)+O\left(a^{\alpha }\varphi \left(a\right)\right)\mathrm{and}\underset{a⟶0}{lim}\frac{\varphi \left(ua\right)}{\varphi \left(a\right)}=\zeta \left(u\right),}$$

where $L(·)$, $\zeta$ are positive and bounded functions, and $\varphi$ is an invertible function;

(K3)

There exist $\xi \in (0,1)$ and ${\xi }_1,{\xi }_2>0,$ such that

$$n^{\xi +{\xi }_1}log{n}^5\le k\le nlog{n}^{-1-{\xi }_2}.$$

Theorem 2.

Under conditions (D1)–(D6) and (K1)–(K3), we have

$$|\widehat{R}\left(x\right)-R\left(x\right)|=O\left({\left({\varphi }^{-1}\left(\frac{k}{n}\right)\right)}^{k_0}\right)+O_{a.co.}\left(\sqrt{\frac{n^{\xi }log\left(n\right)}{k}}\right).$$

(6)

Proof of Theorem 2.

Similarly to Theorem 1, write

$$\widehat{R}\left(x\right)=\frac{\widehat{R_N}\left(x\right)}{\widehat{R_D}\left(x\right)},$$

where

$$\widehat{R_N}\left(x\right)=\frac{1}{n\mathrm{I}\mathrm{E}\left[K\right(A_n^{k^{-1}}\parallel x-X_1\parallel \left)\right]}\sum _{i=1}^n{Y}_i^{-1}K(A_n^{k^{-1}}\parallel x-X_i\parallel ),$$

$$\mathrm{and}\widehat{R_D}\left(x\right)=\frac{1}{n\mathrm{I}\mathrm{E}\left[K\right(A_n^{k^{-1}}\parallel x-X_1\parallel \left)\right]}\sum _{i=1}^n{Y}_i^{-2}K(A_n^{k^{-1}}\parallel x-X_i\parallel ),$$

and we define, for a sequence ${\beta }_n\in (0,1)$, such that ${\beta }_n-1=O\left({\left({\varphi }^{-1}\left(\frac{k}{n}\right)\right)}^{k_0}+\sqrt{\frac{n^{\xi }log\left(n\right)}{k}}\right)$, $h_n^{-}={\varphi }^{-1}\left(\frac{\sqrt{{\beta }_n}k}{n}\right)$, and $h_n^{+}={\varphi }^{-1}\left(\frac{k}{n\sqrt{{\beta }_n}}\right)$. Using standard evidence (see Bouzebda et al. [17] ), we deduce that Theorem 2 is the outcome of Theorem 1 and the two lemmas below. □

Lemma 3.

Under the conditions of Theorem 2, we have

$$|\frac{{\sum }_{i=1}^{n}K(h_n^{{-}^{-1}}\parallel x-X_i\parallel )}{{\sum }_{i=1}^{n}K(h_n^{{+}^{-1}}\parallel x-X_i\parallel )}-{\beta }_n|=O\left({\left({\varphi }^{-1}\left(\frac{k}{n}\right)\right)}^{k_0}\right)+O_{a.co.}\left(\sqrt{\frac{n^{\xi }log\left(n\right)}{k}}\right).$$

Lemma 4.

Based on the conditions of Theorem 2, we obtain

$$1{\mathrm{I}}_{h_n^{-}\le {\varphi }^{-1}\left(\frac{k}{n}\le h_n^{+}\right)}\to 1,a.co..$$

Corollary 2.

Using the conditions of Theorem 2, we get

$$|\widehat{R_N}\left(x\right)-R_1\left(x\right)|=O\left({\left({\varphi }^{-1}\left(\frac{k}{n}\right)\right)}^{k_0}\right)+O_{a.co.}\left(\sqrt{\frac{n^{\xi }log\left(n\right)}{k}}\right),$$

and

$$|\widehat{R_D}\left(x\right)-R_2\left(x\right)|=O\left({\left({\varphi }^{-1}\left(\frac{k}{n}\right)\right)}^{k_0}\right)+O_{a.co.}\left(\sqrt{\frac{n^{\xi }log\left(n\right)}{k}}\right).$$

4. Smoothing Parameter Selection

The applicability of the estimator is related to the selection of the parameters used for the construction of the estimator $\tilde{R}$. In particular, the bandwidth parameter $h_n$ has a decisive effect on the implementation of this regression in practice. In the literature on nonparametric regression analysis, there are several ways to achieve this issue. In this paper, we adopt two approaches common in classical regression to the relative one. The two selections are the cross-validation rule and the bootstrap algorithm.

4.1. Leave-One-Out Cross-Validation Principle

In classical regression, the leave-one-out cross-validation rule is obtained using the mean square error. This criterion has been employed for predicting Hilbertian time series by several authors in the past (see Feraty and View [18] for some references). The leave-one-out cross-validation rule is easy to execute and has shown good behavior in practice. However, it is a relatively time-consuming rule. We overcome this inconvenience by reducing the cardinal of the optimization set of the rule. Thus, we adopt this rule for this kind of regression analysis. Specifically, we consider some subset of smoothing parameters (resp. number of the neighborhood) $H_n$ (resp. $K_n$), and we select the best bandwidth parameter as follows.

$$h_n^{opt}=arg\underset{h_n\in H_n}{min}\sum _{i=1}^n\left(\frac{{(Y_i-{\tilde{R}}^{-i}\left(X_i\right))}^2}{Y_i^2}\right)$$

(7)

or

$$k^{opt}=arg\underset{k\in K_n}{min}\sum _{i=1}^n\left(\frac{{(Y_i-{\widehat{R}}^{-i}\left(X_i\right))}^2}{Y_i^2}\right),$$

where ${\tilde{R}}^{-i}\left(X_i\right)$ (resp. ${\widehat{R}}^{-i}\left(X_i\right)$) is the leave-out-one estimator of $\tilde{R}$ (resp. $\widehat{R}$). The latter is calculated without the observation $(X_i,Y_i)$. It is worth noting that the efficiency of this estimator is also linked to the determination of the subset $H_n$, where the rule (7) is optimized. Often, we distinguish two cases, the local case and the global case. In the local one, the subset $H_n$ is defined with respect to the number of neighborhoods near the location point. For the global case, the subset $H_n$ is the quantile of the vector distance between the Hilbertian regressors. The choice of $K_n$ is easier, and it suffices to take $K_n$ as a subset of a positive integer. This selection procedure has shown good behavior in practice, but there is no theoretical result concerning its asymptotic optimality. This will be a significant prospect for the future.

4.2. Bootstrap Approach

In addition to the leave-one-out cross-validation rule, the bootstrap method constitutes another important selection method. The principle of the latter is based on the plug-in estimation of the quadratic error. In the rest of this subsection, we describe the principal steps of this selection procedure.

Step 1.

We choose an arbitrary bandwidth $h_0$ (resp. $k_0$), and we calculate ${\tilde{R}}_{h_0}\left(x\right)$ (resp. ${\widehat{R}}_{k_0}\left(x\right)$).

Step 2.

We estimate $\widetilde{ϵ}=Y-{\tilde{R}}_{h_0}\left(x\right)$ (resp. $\widehat{ϵ}=Y-{\widehat{R}}_{k_0}\left(x\right)$).

Step 3.

We create a sample of residual ${ϵ}^{\ast }$ (resp. ${ϵ}^{\ast \ast }$ ) from the distribution

$$G^{\ast }=((\sqrt{5}+1)/2\sqrt{5}){\delta }_{\widetilde{ϵ}(1-\sqrt{5})/2}-((1-\sqrt{5})/2\sqrt{5}){\delta }_{\widetilde{ϵ}(\sqrt{5}+1)/2},$$

$$(\mathrm{resp}.G^{\ast }=((\sqrt{5}+1)/2\sqrt{5}){\delta }_{\widehat{ϵ}(1-\sqrt{5})/2}-((1-\sqrt{5})/2\sqrt{5}){\delta }_{\widehat{ϵ}(\sqrt{5}+1)/2}),$$

where $\delta$ is the Dirac measure (see Hardle and Marron [19] for more details).

Step 4.

We reconstruct the sample ${(Y_i^{\ast },X_i^{\ast })}_i=({ϵ}^{\ast }-{\tilde{R}}_{h_0}\left(X_i\right),X_i),$ (resp. ${(Y_i^{\ast \ast },X_i^{\ast \ast })}_i=({ϵ}^{\ast \ast }-{\widehat{R}}_{k0}\left(X_i\right),X_i),$

Step 5.

We use the sample ${(Y_i^{\ast },X_i^{\ast })}_i$ to calculate ${\tilde{R}}_{h_0}\left(X_i\right)$ and ${(Y_i^{\ast \ast },X_i^{\ast \ast })}_i$ to calculate ${\widehat{R}}_{k_0}\left(X_i\right)$.

Step 6.

We repeat the previous steps $N_B$ times and put ${\tilde{R}}_{h_0}^r\left(X_i\right)$ (resp. ${\widehat{R}}_{k_0}^r\left(X_i\right)$), the estimators, at the replication r.

Step 7.

We select h (resp. k) according to the criteria

$$h_{op{t}^{Boo}}=arg\underset{h\in H_n}{min}\sum _{r=1}^{N_B}\sum _{i=1}^n{({\tilde{R}}_h^r\left(X_i\right)-{\tilde{R}}_{h_0}^r\left(X_i\right))}^2,$$

(8)

and

$$k_{op{t}^{Boo}}=arg\underset{k\in K_n}{min}\sum _{r=1}^{N_B}\sum _{i=1}^n{({\widehat{R}}_k^r\left(X_i\right)-{\tilde{R}}_{k_0}^r\left(X_i\right))}^2.$$

Once again, the choice of the subset $H_n$ (resp. $K_n$) and the pilot bandwidth $h_0$ (resp. $k_0$) have a significant impact on the performance of the estimator. It will be very interesting to combine both approaches in order to benefit from the advantage of both selections. However, the time cost of this idea is very important.

5. Computational Study

5.1. Empirical Analysis

As a theoretical contribution, we wish in this empirical analysis to inspect the easy implementation of the built estimator $\tilde{R}$ in practice. As the determination of $h_n$ is the principal challenge of the computation ability of $\tilde{R}$, we compared in this computational study the two selections discussed in the previous section. For this purpose, we conducted an empirical analysis based on artificial data generated through the following nonparametric regression

$$Y_i=\tau \left(X_i\right)+{ϵ}_i,i=1,\dots ,n$$

(9)

where $\tau \left(\right)$ is known regression operator r and $\left({ϵ}_i\right)$ sequence of independent random variable generated from a Gaussian distribution $\mathcal{N}(0,0.5)$. The model, in (9), shows the relationship between an endogenous and exogenous variable.

On the other hand, in order to prospect the dependency of the data, we generated the Hilbertian regressor by using the Hilbertian GARCH process through dgp.fgarch from the R-package rockchalk. We plotted, in Figure 1, a sample of the exogenous curves $X\left(t\right)$.

The endogenous variable Y was generated by

$$\tau \left(x\right)=4{\int }_0^{\pi }\frac{x^2\left(t\right)}{1+x^2\left(t\right)}dt.$$

For this empirical analysis, we compared the two selectors (7) and (8) with the mixed one obtained by using the optimal h of the rule (7) as the pilot bandwidth in the bootstrap procedure (8). For a fair comparison between the three algorithms, we optimized over the same subset $H_n$. We selected the optimal h for the three selectors, and the subset $H_n$ of the quantiles of the vector distance between the Hilbertian curves observations of $X_i$ (the order of the quantiles was $\in \{1/5,1/10,1/15,0.5\}$. Finally, based on a quadratic kernel on $(0,1)$, the estimator was computed, and we utilized the $L^2$ metric associated with the PCA definition based on the $m=3$ first eigenfunctions of the empirical covariance operator associated with the $m=3$ greatest eigenvalues (see Ferraty and Vieu [18]).

The efficiency of the estimation method was evaluated by plotting the true response value ${\left(Y_i\right)}_i$ versus the predicted values $\widehat{R}\left(X_i\right)$. In addition, we used the relative error defined by

$${\displaystyle RSE=\sum _{i=1}^n\left(\frac{{(Y_i-{\tilde{R}}^{-i}\left(X_i\right))}^2}{Y_i^2}\right)}$$

to evaluate the performance of this simulation study, which performed over 150 replications. The prediction results are depicted in Figure 2.

It shows clearly that the $\tilde{R}$ of the relative regression was very easy to implement in practice, and both selection algorithms had satisfactory behaviors. Typically the mixed approach performed better compared to the two separate approaches. It had an $RSE=0.35$. On the other hand, the cross-validation rule had a small superiority ($RSE=0.52$) over the bootstrap approach $RSE=0.65$) in this case. Of course, this small superiority was justified by the fact that the efficiency of the bootstrap approach was based on the pilot bandwidth parameter $h_0$, whereas the cross-validation rule was strongly linked to the relative error loss function.

5.2. A Real Data Application

We devote this paragraph to the real application of the RE-regression as a predictor. Our ambition is to emphasize the robustness of this new regression. To do this, we compared it to the classical regression defined by the conditional expectation. For this purpose, we considered physics data corresponding to the monthly number of sunspots in the years 1749–2021. These data were available at the website of WDC-SILSO, Royal Observatory of Belgium, Brussels, http://www.sidc.be (accessed on 1 April 2023). The prediction of sunspots is very useful in real life. It can be used to forecast the space weather, assess the state of the ionosphere, and define the appropriate conditions of radio shortwave propagation or satellite communications. It is worth noting that these kinds of data can be viewed as a continuous time process, which is the principal source of a Hilbertian time series by cutting the continuous trajectory into small intervals with fixed larger intervals. To fix these ideas, we plotted the initial data in Figure 3.

To predict the value of a sunspot in the future, given its past observations in a continuous path, we use ${\left(Z_t\right)}_{t\in [0,b)}$ the whole data set, as a real-valued process in continuous time. We then constructed, from $Z_t$, n Hilbertian variables ${\left(X_i\right)}_{i=1,\dots ,n}$, where

$$\forall t\in [0,b),X_i\left(t\right)=Z_{n^{-1}((i-1)b+t)},Y_i=X_i\left(b\right).$$

Thus, our objective was to predict $Y_n$, knowing ${(X_i,Y_i)}_{i=1,\dots ,n-1}$ and $X_n$. At this stage, $\tilde{R}\left(X_n\right)$ was the predictor of $Y_n$. In this computational study, we aimed to forecast the sunspot number one year ahead, given the observation of the past years. Thus, we fixed on month j in 1, …, 12, and computed the estimator $\tilde{R}$ by the sample ${(Y_i^j,X_i)}_{i=1\dots 272}$, with $Y_i^j$ as the sunspot number of jth months in the $(i+1)$th year, and we repeated this estimation procedure for all $j=1,\dots ,12$.

As the main feature of the RE-regression is its insensitivity to the outliers, we examined this property by detecting the number of outliers in each prediction step j. To do this, we used a MAD-Median rule (see Wilcox and Rand [20]). Specifically, the MAD-Median rule considers an observation $Y_i$ as an outlier if

$$\frac{|Y_i-M|}{\mathrm{MAD}\ast 0.6745}>C,$$

where M and MAD are the medians of ${\left(Y_i\right)}_i$, and ${(Y_i-M)}_i$ respectively, and $C=\sqrt{{{\chi }^2}_{0.975}}$ (with one degree of freedom).

Table 1. Number of outliers with respect to j.

Months	1	2	3	4	5	6	7	8	9	10	11	12
Outliers	15	26	13	5	24	25	7	9	11	8	9	15

summarizes the number of outliers for each step j.

Both estimators $\tilde{R}$ and

$$\widehat{R}\left(x\right)=\frac{{\sum }_{i=1}^n{Y}_{i}K(h_n^{-1}\parallel x-X_i\parallel )}{{\sum }_{i=1}^{n}K(h_n^{-1}\parallel x-X_i\parallel )}$$

were simulated using the quadratic kernel $K\left(x\right)$, where

$$K\left(x\right)=\frac{3}{2}(1-x^2)\mathrm{1}{\mathrm{I}}_{[0,1]}$$

and norm $L^2$ was associated with the PCA-metric with $m=3$. The cross-validation rule (7) is used to choose the smoothing parameter h. Figure 4 shows the prediction results, where we drew two curves showing the predicted values (the dashed curve for the relative regression and the point curve for the classical regression) and the observed values (solid curve).

Figure 4 shows that $\tilde{R}$ performed better in terms of the prediction results compared to $\widehat{R}$. Even though both predictors had good behavior, the ASE of the relative regression (2.09) was smaller than the classical regression, which was equal to 2.87.

6. Conclusions

In the current contribution, we focused on the kernel estimation of the RE-regression when the observations exhibited quasi-associated autocorrelation. It constituted a new predictor in the Hilbertian time series, an alternative to classical regression based on conditional expectation. Clearly, this new estimator increased the robustness of the classical regression because it reduced the effect of the largest variables. Therefore, it made this Hilbertian model insensitive to the outlier observations; this is the main feature of this kind of regression. We provided in this contribution two rules to select the bandwidth parameter. The first was based on adapting the cross-validation rule to the relative error loss function. The second was obtained by the adaptation of the wild bootstrap algorithm. The simulation experiment highlighted the applicability of both selectors in practice. In addition to these features, the present work opened an important number of questions for the future. First, establishing the asymptotic distribution of the present estimator allows extending the applicability of this model to other applied issues in statistics. The second natural prospect focuses on the treatment of some alternative Hilbertian time series, including the ergodic case, the spatial case, and the $\beta$-mixing case, among others. It will also be very interesting to study another type of data (missing, censored, …) or another estimation method, such as the kNN, local linear method, …, etc.