Kolmogorov Entropy for Convergence Rate in Incomplete Functional Time Series: Application to Percentile and Cumulative Estimation in High Dimensional Data

Abstract

The convergence rate for free-distribution functional data analyses is challenging. It requires some advanced pure mathematics functional analysis tools. This paper aims to bring several contributions to the existing functional data analysis literature. First, we prove in this work that Kolmogorov entropy is a fundamental tool in characterizing the convergence rate of the local linear estimation. Precisely, we use this tool to derive the uniform convergence rate of the local linear estimation of the conditional cumulative distribution function and the local linear estimation conditional quantile function. Second, a central limit theorem for the proposed estimators is established. These results are proved under general assumptions, allowing for the incomplete functional time series case to be covered. Specifically, we model the correlation using the ergodic assumption and assume that the response variable is collected with missing at random. Finally, we conduct Monte Carlo simulations to assess the finite sample performance of the proposed estimators.

1. Introduction

Statistical problems associated with the study of functional random variables, that is, variables with values in an infinite-dimensional space, have garnered increasing attention in the statistical literature over the past few decades. The abundance of data measured on increasingly fine temporal/spatial grids, as is the case in meteorology, medicine, satellite imagery, and numerous other research fields, has inspired the development of this research theme. Thus, the statistical modeling of these data as random functions resulted in a number of difficult theoretical and numerical research concerns; we may refer to [1,2,3,4,5,6] for parametric and nonparametric models. For the latest contributions in FDA and its related topics, one can refer to [7,8,9,10,11,12,13,14].

Quantile regression has emerged as a significant statistical technique for data analysis since Koenker and Bassett’s [15] seminal work. But, concerns about quantile crossing and model misspecification [16,17,18] have led to the development of the nonparametric estimation of conditional quantile functions [19,20]. This estimate originates from [21], who proved the convergence using the probability of the empirical conditional law function; refer to [22,23]. This technique is an alternative to mean regression, and it possesses many desirable properties, such as being more efficient than mean regression when the data follow a heavy-tailed distribution, and it is frequently used to characterize the entire conditional distribution of the response. Conditional percentile estimation has been studied extensively by several researchers [24,25,26]. However, most of them used the kernel method approach. For instance, Ferraty et al. [4] have studied the uniform convergence of such an estimation in the analogously distributed and independent case. However, in the same case, Samanta [27] acquired the almost complete (a.co.) convergence of the conditional percentile estimation.

Studies of local linear estimation conditional quantile function (LLECQF) are still limited. For example, Messaci et al. [28] have studied the local linear estimation (LLE) of the conditional quantile function (CQF) by reversing the estimator shown in [29]. Al-Awadhi et al. [30] have proved the a.co. convergence and the asymptotic law of the CQF by considering an estimator based on the $L_1$ approach. Despite its importance in functional data analysis, the LLE has various merits over the kernel technique. Generally, this method can reduce the bias properties of the kernel approach [31,32]. Furthermore, the LLE has been lately introduced in the FDA by [33]. The latter concentrates on the LLE of the curve regression while the insert variable is a Hilbert class. Barrientos-Marin et al. [34] have proposed the LLE of the nonparametric regression operator and studied their asymptotic properties. In particular, this operator can be applied to functional covariable.

In this article, we are interested in studying the local linear estimation of the conditional cumulative distribution function (LLECCDF) and LLECQF under the assumption of ergodicity. We are interested in the uniform a.co. convergence of the constructed sequences using the Kolmogorov entropy function. In addition to Kolmogorov entropy, other information measures such as Kullback–Leibler divergence have been considered for a convergence rate study of estimators in multivariate time series modelling; refer to [35,36].

From a practical point of view, the ergodic framework is an essential condition in statistical physics, number theory, Markov chains, and other fields. The concept of ergodicity is fundamental in the research of stochastic processes. Note also that one of the arguments invoked by [37] motivating the introduction of the concept of ergodicity is that, for certain classes of processes, it can be much easier to prove ergodic properties rather than the mixing condition. Hence, the ergodicity hypothesis seems to be the most naturally adapted and provides a better framework to study data series such as those generated by noisy chaos.

In their discussion, [38] provided an example of processes that are ergodic but not mixing, which may be summarized as follows: let $\{(T_i,{\lambda }_i):i\in \mathbb{Z}\}$ be a strictly stationary process such that $T_i\mid {\mathcal{T}}_{i-1}$ is a Poisson process with a parameter ${\lambda }_i$, where ${\mathcal{T}}_i$ is the $\sigma$-field generated by $(T_i,{\lambda }_i,T_{i-1},{\lambda }_{i-1},\dots )$. Assume that ${\lambda }_i=f({\lambda }_{i-1},T_{i-1})$, where $f:[0,\infty )\times \mathbb{N}\to (0,\infty )$ is a given function. This process does not mix in general (see Remark 3 of [39]). It is known that any sequence ${\left({\epsilon }_i\right)}_{i\in \mathbb{Z}}$ of i.i.d. random variables is ergodic. Hence, it is immediately clear that ${\left(Y_i\right)}_{i\in \mathbb{Z}}$ with $Y_i=\vartheta ((\dots ,{\epsilon }_{i-1},{\epsilon }_i),({\epsilon }_{i+1},{\epsilon }_{i+2},\dots )),$ for some Borel-measurable function $\vartheta (·)$, is also ergodic; see Proposition 2.10 on page 54 in [40]. Under the condition of ergodicity, [41] have studied the conditional quantile for ergodic data by considering an iterative model. However, under the same conditions, the authors in [42] have considered the nonparametric estimation of quantile for censored data.

All these studies were concerned with the complete-data situation. This work investigates the question of when data are missing at random (MAR); for instance, see [43]. In contemporary statistics, missing data are pervasive, posing a significant obstacle for various applications. The concept of missing data in statistics occurs when a data value of the variable is not stored in the observation. For example, missingness may occur when our data are compiled from sources that have measured various variables; for instance, in the healthcare industry, the data routinely collected on patients may vary between clinics and hospitals. Among numerous other reasons are sensor failure, data censorship, privacy concerns, pharmaceutical tracing tests, and reliability tests. It can occur in any experimental setting where contamination of the treatment or subject mortality is possible. This topic has been extensively examined in numerous statistical problems; see [43,44] for a thorough overview. In the present work, we only observe Y in cases where some indicator B equals one and the indicator B is conditionally independent of Y given X. This assumption is useful when information in the form of covariate data is available to explain the missingness; refer to [45,46,47]. The first studies on MAR are presented by [48]. They established an approximation of the regression operator and studied their asymptotic consistency when the curve regressor is observed and the interest response is missing at random. However, in the ergodic data case, Ling et al. [49] have studied the asymptotic distribution of the estimator, which is proposed in [48]; also, refer to [50,51].

The main objective of this paper is to evaluate the convergence velocity of some functional estimators through the Kolmogorov entropy function. More precisely, we focus on the local linear smoothing of the distribution function and its inverse, the quantile function. The constructed estimators’ asymptotic properties are evaluated when the data are correlated as functional ergodic time series data and the response variable is observed under the MAR structure. The efficiency of these estimators is uniformly specified using the entropy metric, allowing us to assess the impact of the functional path of the data. In particular, the Kolmogorov entropy gives a trade-off between the data’s sparsity and the approximation’s efficiency. Moreover, the Kolmogorov entropy explores the functional space’s topological structure and its spectral property. Thus, stating the uniform consistency concerning the Kolmogorov entropy is more beneficial than the classical pointwise case. Although the uniform convergence of LLECCDF and LLECQF in the functional ergodic time series (FETS) structure is purely mathematically challenging, the obtained results are also pivotal for many applied issues, such as the smoothing parameter choice, bootstrapping, and the single index modeling. The second challenging issue of this contribution concerns the MAR feature of the response variable. Our results can be used to determine an estimator of the unconditional distribution of the scalar response, even if it is not completely observed. All these challenging issues will be discussed using specific examples in Section 6. In addition to the uniform consistency, we prove the asymptotic normality of the LLECCDF, which is important to provide a confidence interval comparable with the predictive interval deduced from LLECQF. Once again, this prediction using the predictive subset is also primordial in the context of incomplete functional time series data. Finally, we point out that, to our best knowledge, this problem of uniform consistency of local linear approximation under MAR and FETS structures was open up to the present, giving the main motivation to our paper.

The rest of the paper is organized as follows. In Section 2, we state the formal setup and define the estimators. More precisely, Section 2.1 is devoted to the LLECCDF estimator, while Section 2.2 introduces the LLECQF estimator. The convergence rate of the two approximation sequences is established in Section 3. In Section 4, we derive the limiting distribution of the proposed estimators. In Section 5, we conduct Monte Carlo simulations to assess the finite sample performance of the proposed estimators. Section 6 is devoted to highlighting the principal features of our contribution. In Section 7, we give some concluding remarks. All the proofs are gathered in the last section.

2. Model and Estimators

2.1. LLECCDF: Numerical Approximation of CCDF-Model

Let $\left\{\left(X_i,Y_i\right):1\le i\le n\right\}$ be a sequence of stationary ergodic functional random variables identically distributed as $(X,Y)$, where X takes values in a some semi-metric abstract space $\mathcal{F}$ with a semi-metric $d(·,·)$ and Y takes values in $\mathbb{R}$. For reader convenience, we introduce some details defining the ergodic property of processes and its link with the mixing one. Let $\left\{X_n,n\in \mathbb{Z}\right\}$ be a stationary sequence. Consider the backward field ${\mathcal{A}}_n=\sigma \left(X_k:k\le n\right)$ and the forward field ${\mathcal{B}}_m=\sigma \left(X_k:k\ge m\right)$. The sequence is strongly mixing if

$$\underset{A\in {\mathcal{A}}_0,B\in {\mathcal{B}}_n}{sup}|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)|\to 0,\mathrm{as}n\to \infty .$$

The sequence is ergodic if, for any two measurable sets $A,B$,

$$\underset{n\to \infty }{lim}\frac{1}{n}{{\displaystyle \Sigma }}_{k=0}^{n-1}\left|\mathbb{P}\left(A\cap {\tau }^{-k}B\right)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\right|=0,$$

where $\tau$ is the time evolution or shift transformation taking $X_k$ into $X_{k+1}$. We shall also use the same symbol $\tau$ to denote the induced set transformation, which takes, for example, sets $B\in {\mathcal{B}}_m$ into sets $\tau B\in {\mathcal{B}}_{m+1}$; for instance, see [52]. The naming of strong mixing in the above definition is more stringent than what is ordinarily referred to (when using the vocabulary of measure preserving dynamical systems) as strong mixing, namely to that ${lim}_{n\to \infty }\mathbb{P}\left(A\cap {\tau }^{-n}B\right)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)$ for any two measurable sets $A,B$; see, for instance [52,53] and more recent references [54,55,56,57,58,59,60]. Hence, strong mixing implies ergodicity, whereas the inverse is not always true (see, e.g., Remark 2.6 in page 50 in connection with Proposition 2.8 in page 51 in [40]). For every $x\in \mathcal{F}$, the function of conditional law $CFD\left(y\right|x)$ of Y when $X=x$ is defined by

$$CFD\left(y\right|x)=\mathbb{P}\left(Y\le y|X=x\right).$$

The LLECCDF is obtained by assuming for every z in the vicinity of x

$$CDF\left(y\right|z)={\beta }_1+{\beta }_2\alpha (z,x)+o\left(\alpha (z,x)\right)with\alpha (z,z)=0,$$

(1)

where $\alpha (·,·)$ is a bilinear locating function such that

 (i) 

For all $x^{\prime }\in {\mathcal{C}}_{\mathcal{F}}$, $C^{\prime }|\delta (x,x^{\prime })|\le |\alpha (x^{\prime },x)|\le C|\delta (x,x^{\prime })|;$

 (ii) 

For all $z_1,z_2\in {\mathcal{C}}_{\mathcal{F}}$, $|\alpha (z_1,x)-\alpha (z_2,x)|\le C^{{}^{\prime }}\left|\delta (z_1,z_2)\right|$,

where $\delta (·,·)$ is a bilinear function such that $\left|\delta \right(·,·\left)\right|=d(·,·)$. In the rest of the paper, we suppose that the $CDF$ is of the $C^1$-class with respect to y and its derivative is the conditional density function denoted by $cdf(·)$. However, in the case of a missing response, one has an incomplete sample of size n from $(X,Y,B)$, which is usually denoted by $\left\{\left(X_i,Y_i,B_i\right):1\le i\le n\right\}$, where $B_i=1$ if $Y_i$ is observed, and $B_i=0$ otherwise. The Bernoulli random variable B is supposed to be such that

$$P\left(x\right)=\mathbb{P}(B=1|X=x)=\mathbb{P}(B=1|X=x,Y=y)$$

is a continuous function. Under this smoothing consideration, we define the LLECCDF of $CDFF(·\mid ·)$ by finding the minimizers $(\widehat{{\beta }_1},\widehat{{\beta }_2})$ of

$$\begin{array}{c}\hfill {\displaystyle \underset{({\beta }_1,{\beta }_2)\in {\mathbb{R}}^2}{min}{\Sigma }_{i=1}^n{\left(\mathcal{J}\left({\displaystyle \frac{y-Y_i}{{\lambda }_J}}\right)-{\beta }_1-{\beta }_2\alpha (X_i,x)\right)}^2\mathcal{K}er\left({\displaystyle \frac{\delta (x,X_i)}{{\lambda }_K}}\right)B_i,}\end{array}$$

(2)

where ${\lambda }_K:={\lambda }_{K,n}$ and ${\lambda }_J:={\lambda }_{J,n}$ are bandwidth parameters and $\mathcal{J}(·)$ and $\mathcal{K}er(·)$ are, respectively, distribution and kernel functions. The explicit solution to this minimization is

$${\displaystyle \widehat{{\beta }_1}=\widehat{CDF}\left(y\right|x)=\frac{{\displaystyle {\Sigma }_{j=1}^n{\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)B_j{\mathcal{J}}_j\left(y\right)}}{{\displaystyle {\Sigma }_{j=1}^n{\Gamma }_j\left(x\right)B_j\mathcal{K}e{r}_j\left(x\right)}},}$$

with

$$\begin{array}{ccc}\hfill {\Gamma }_j\left(x\right)& =& {\displaystyle {\Sigma }_{i=1}^n{\alpha }_i^2\left(x\right)\mathcal{K}e{r}_i\left(x\right)B_i-\left({\displaystyle {\Sigma }_{i=1}^n{\alpha }_i\left(x\right)\mathcal{K}e{r}_i\left(x\right)B_i}\right){\alpha }_j,}\hfill \\ \hfill \mathcal{K}e{r}_j\left(x\right)& =& \mathcal{K}er\left({\displaystyle \frac{\delta (x,X_j)}{{\lambda }_K}}\right)\hfill \end{array}$$

and

$${\mathcal{J}}_j\left(y\right)=\mathcal{J}\left({\displaystyle \frac{y-Y_j}{{\lambda }_J}}\right),{\alpha }_j\left(x\right)=\alpha (X_j,x).$$

Furthermore, we can write

$$\widehat{CDF}(y\mid x)=\frac{{\widehat{CDF}}_N^x\left(y\right)}{{\widehat{CDF}}_D\left(x\right)},$$

where

$${\displaystyle {\widehat{CDF}}_N^x\left(y\right):=\frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n{\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right){\mathcal{J}}_j\left(y\right)B_j,}$$

and

$${\displaystyle {\widehat{CDF}}_D\left(x\right):=\frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n{\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)B_j.}$$

The first main contribution of this work is a precise convergence rate of the approximation $\widehat{CDF}(·\mid ·)$ uniformly over a non-necessary compact subset ${\mathcal{C}}_{\mathcal{F}}$ of $\mathcal{F}$. We use the notation

$$\mathcal{B}a(x,h)=\left\{x^{\prime }\in \mathcal{F}:d\left(x^{\prime },x\right)\le h\right\}.$$

Definition 1.

Let ${\mathcal{C}}_{\mathcal{F}}$ be a subset of a semi-metric space $\mathcal{F}$, and let $\epsilon >0$ be given. A finite set of points $x_1,x_2,\dots ,x_N$ in $\mathcal{F}$ is called an ε-net for ${\mathcal{C}}_{\mathcal{F}}$ if

$${\mathcal{C}}_{\mathcal{F}}\subset \bigcup _{k=1}^N\mathcal{B}a\left(x_k,\epsilon \right).$$

The quantity ${\psi }_{{\mathcal{C}}_{\mathcal{F}}}\left(\epsilon \right)=log\left(N_{\epsilon }\left({\mathcal{C}}_{\mathcal{F}}\right)\right)$, where $N_{\epsilon }\left({\mathcal{C}}_{\mathcal{F}}\right)=:d_n$ is the minimal number of open balls in $\mathcal{F}$ of radius ε which is necessary to cover ${\mathcal{C}}_{\mathcal{F}}$, is called Kolmogorov’s ε-entropy of the set ${\mathcal{C}}_{\mathcal{F}}$.

This concept was introduced by Kolmogorov in the mid-1950s (refer to [61]). It serves as a measure of the complexity of a set, indicating that high entropy implies that a significant amount of information is required to accurately describe an element within a given tolerance, $\epsilon$. Consequently, the selection of the topological structure (specifically, the choice of semi-metric) plays a crucial role when examining asymptotic results that are uniform over a subset, ${\mathcal{C}}_{\mathcal{F}}$ of $\mathcal{F}$. In particular, we subsequently observe that a well-chosen semi-metric can enhance the concentration of the probability measure for the functional variable, X, while minimizing the $\epsilon$-entropy of the subset, ${\mathcal{C}}_{\mathcal{F}}$. Ferraty and Vieu [6] emphasized the phenomenon of concentration of the probability measure for the functional variable by calculating small ball probabilities in different standard scenarios; refer to [62]. For readers interested in these concepts (entropy and small ball probabilities) and/or the utilization of Kolmogorov’s $\epsilon$-entropy in dimensionality reduction problems, we recommend referring to [63] or/and [64], respectively.

Let $\left(u_n\right)$ for $n\in \mathbb{N}$, be a sequence of real r.v.s. We say that $\left(u_n\right)$ converges almost-completely (a.co.) toward zero if, and only if, for all $ϵ>0,$

$${\displaystyle {\Sigma }_{n=1}^{\infty }\mathbb{P}\left(\left|u_n\right|>ϵ\right)<\infty .}$$

Moreover, we say that the rate of the almost-complete convergence of $\left(u_n\right)$ toward zero is of order $v_n$ (with $v_n\to 0$), and we write $u_n=O_{a.co.}\left(v_n\right)$ if, and only if, there exists $ϵ>0$ such that

$${\displaystyle {\Sigma }_{n=1}^{\infty }\mathbb{P}\left(\left|u_n\right|>ϵv_n\right)<\infty .}$$

This kind of convergence implies both the almost-sure convergence and the convergence in probability.

2.2. LLECQF: Numerical Approximation of CQF-Model

The second approximation concerns the CQF-Model of order p, for $p\in (0,1)$, denoted as $CQ{F}_p\left(x\right)$:

$$CDF\left(CQ{F}_p\left(x\right)\right|x)=p.$$

The natural estimator is obtained by inverting the LLECCDF. However, unlike the local constant estimator, the LLECCDF is not monotone and invertible. To overcome this issue, we use the robust definition of the $CQ{F}_p\left(x\right)$:

$$CQ{F}_p\left(x\right)=arg\underset{t}{min}\mathrm{I}\mathrm{E}\left[L_p(Y-t)|X=x\right],$$

(3)

where the scoring function ${\displaystyle L_p\left(y\right)=\left|y\right|(p-{11}_{[y<0]})}$, with ${11}_S$ as the indicator of S. Once again, the LLECQF is obtained via a smoothing approximation of the model locally in the neighborhood of the location x. Therefore, we suppose that $CQ{F}_p\left(x\right)$ is such that

$$CQ{F}_p\left(z\right)={\beta }_1+{\beta }_2\alpha (z,x)+o\left(\alpha (z,x)\right).$$

(4)

Therefore, ${\beta }_1$ and ${\beta }_2$ are estimated as

$${\displaystyle arg\underset{({\beta }_1,{\beta }_2)\in \mathrm{I}{\mathrm{R}}^2}{min}{\Sigma }_{j=1}^n{L}_p(Y_j-{\beta }_1-{\beta }_2\alpha (X_j,x))\mathcal{K}e{r}_j\left(x\right)B_j.}$$

(5)

So, the LLECQF of $CQ{F}_p\left(x\right)$ is denoted by

$$\widehat{CQ{F}_p}\left(x\right)=\widetilde{{\beta }_1},$$

where

$${\displaystyle (\widetilde{{\beta }_1},\widetilde{{\beta }_2})=arg\underset{({\beta }_1,{\beta }_2)\in \mathrm{I}{\mathrm{R}}^2}{min}{\Sigma }_{j=1}^n{L}_p(Y_j-{\beta }_1-{\beta }_2\alpha (X_j,x))\mathcal{K}e{r}_j\left(x\right)B_j.}$$

Once again, our main focus is to establish the convergence rate using the Kolmogorov entropy and the uniform-Lipschitzian condition: For all $(y_1,y_2)\in {\mathcal{N}}_y^2$ and $x_1,x_2\in {\mathcal{C}}_{\mathcal{F}}$, we have

$$\left|CD{F}^{\left(j\right)}\left(y_1\right|x_1)-CD{F}^{\left(j\right)}\left(y_1\right|x_2)\right|\le C_2\left(|\delta (x_2,x_1){|}^{k_2}+{|y_2-y_1|}^{k_1}\right),$$

(6)

for $j=0,1$ and $C_2$, $k_1,k_2>0.$

3. Uniform Convergence Rate

Let ${\mathfrak{F}}_i$ and ${\mathcal{G}}_i$$(i=1,\dots ,n)$ be the $\sigma$-algebras generated by $((Y_1,X_1),\dots ,(Y_i,X_i))$ and $((Y_1,X_1),\dots ,(Y_i,X_i),X_{i+1})$, respectively. Let ${\mathcal{N}}_y$ denote a fixed neighborhood of a point y in $\mathrm{I}\mathrm{R}$. Define

$${\psi }_x({\lambda }_1,{\lambda }_2)={\displaystyle \mathrm{I}\mathrm{P}}({\lambda }_2\le \delta (X,x)\le {\lambda }_1).$$

The function ${\psi }_x(·,·)$ plays a similar role to small ball probability as in [34]. Finally, let C and $C^{{}^{\prime }}$ be some generic constants that are strictly positive. To obtain the convergence rate in this functional ergodic MAR scheme, we consider the following assumptions:

(H1)

 The first-order derivative of $\psi (·)$ exists and is bounded in ${\mathcal{C}}_{\mathcal{F}}$. The function $\psi (·)$ is such that

$$0<C^{\prime }\psi \left(\varsigma \right)\le \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}{\displaystyle \mathrm{I}\mathrm{P}}(X\in \mathcal{B}a(x,\varsigma ))\le \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\displaystyle \mathrm{I}\mathrm{P}}(X\in \mathcal{B}a(x,\varsigma ))\le C\psi \left(\varsigma \right).$$

(H2)

 There exists a non-random function ${\psi }_i(·)$ satisfying

 (i) 

   $0<C{\psi }_i\left(\varsigma \right)<\mathrm{I}\mathrm{P}(X_i\in \mathcal{B}a(x,\varsigma )|{\mathfrak{F}}_{i-1})\le C^{{}^{\prime }}{\psi }_i\left(\varsigma \right)$,

 (ii) 

 ${\displaystyle \frac{1}{n\psi \left(\varsigma \right)}{\Sigma }_{i=1}^n{\psi }_i\left(\varsigma \right)⟶1,\mathrm{a}.\mathrm{co}.,}$

 (iii) 

There exists a non-random function $\mathsf{\Psi }(·)$ satisfying

$${\displaystyle \forall t\in [-1,1],\underset{{\lambda }_K\to 0}{lim}\frac{\psi \left(t{\lambda }_K\right)}{\psi \left({\lambda }_K\right)}=\mathsf{\Psi }\left(t\right).}$$

(H3)

 The distribution function $\mathcal{J}(·)$ and the kernel $\mathcal{K}er(·)$ fulfill the following:

 (i) 

$\mathcal{K}er(·)$ is a Lipschitz on its support $[-1,1]$ satisfying

$$D=\left(\begin{array}{cc}\mathcal{K}er\left(1\right)-{\int }_{-1}^1\mathcal{K}e{r}^{\prime }\left(t\right)\Psi \left(t\right)dt& \mathcal{K}er\left(1\right)-{\int }_{-1}^1{\left(t\mathcal{K}er\left(t\right)\right)}^{\prime }\Psi \left(t\right)dt\\ \\ \mathcal{K}er\left(1\right)-{\int }_{-1}^1{\left(t\mathcal{K}er\left(t\right)\right)}^{\prime }\Psi \left(t\right)dt& \mathcal{K}er\left(1\right)-{\int }_{-1}^1{\left(t^2\mathcal{K}er\left(t\right)\right)}^{\prime }\Psi \left(t\right)dt\end{array}\right),$$

is a positive definite matrix.

 (ii) 

The function $\mathcal{J}(·)$ has a derivative satisfying

$${\displaystyle {\int \left|t\right|}^{k_1}{\mathcal{J}}^{\left(1\right)}\left(t\right)dt<\infty .}$$

 (iii) 

We almost surely have $\mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j\left(y\right)\right|{\mathcal{G}}_{j-1}]=\mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j\left(y\right)\right|X_j].$

(H4)

 The real sequence $d_n$ associated with $r_n=O\left({\displaystyle \frac{lnn}{n}}\right)$ satisfies

$${\displaystyle \frac{{(lnn)}^2}{n\psi \left({\lambda }_K\right)}<ln{d}_n<\frac{n\psi \left({\lambda }_K\right)}{lnn}\mathrm{and}{\Sigma }_{n=1}^{\infty }{d}_n^{1-\varrho }<\infty \mathrm{for}\mathrm{some}\varrho >1.}$$

(H5)

 The bandwidth ${\lambda }_K$ is linked to $\alpha (·)$ and $\psi (·)$ by

 (i) 

$${\displaystyle \forall n>n_0-\frac{1}{\psi \left({\lambda }_K\right)}{\int }_{-1}^1\psi (z{\lambda }_K,{\lambda }_K)\frac{d}{dz}\left(z^2(\mathcal{K}er\left(z\right)\right)dz>C;}$$

 (ii) 

${\displaystyle \underset{n\to \infty }{lim}{\lambda }_K=0,}$ and ${\displaystyle \underset{n\to \infty }{lim}{\lambda }_H=0,}$ and ${\displaystyle \underset{n\to \infty }{lim}\frac{lnn}{n\psi \left({\lambda }_K\right)}=0;}$

 (iii) 

$${\displaystyle {\lambda }_K{\int }_{\mathcal{B}a(x,{\lambda }_K)}\alpha (u,x)d\mathbf{P}\left(u\right)=O\left({\displaystyle {\int }_{\mathcal{B}a(x,{\lambda }_K)}{\alpha }^2(u,x)d\mathbf{P}\left(u\right)}\right),}$$

where $\mathbf{P}$ is the law of X.

We first recall that our assumptions are not restrictive and may be considered as standard in the functional local linear analysis context. They are similar to those used in the local linear estimation of the quantile regression in [30]. In particular, Assumption (H1) concerns the usual concentration property of the functional variable. It is well documented that this property allows us to explore the functionality nature of the data. Assumption (H3)(iii) is a Markov-type condition and characterizes the conditional moments. It is satisfied when considering, for instance, the model

$${11}_{(Y\le y)}=m_{\psi }\left(X_i\right)+{ϵ}_i,$$

where $\left({ϵ}_i\right)$ is a square-integrable process independent of $\left(X_i\right)$. Finally, Assumptions (H3)(i)-(ii), (H4), and (H5) are technical conditions, similar to those used by [30].

The following theorem gives the uniform, almost complete convergence of $\widehat{CDF}(y\mid x)$ with the rate.

Theorem 1.

Under Conditions (H1)–(H5), we have

$$\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|\widehat{CDF}(y\mid x)-CDF(y\mid x)\right|=O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)+O_{a.co.}\left({\left(\frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}\right)}^{\frac{1}{2}}\right).$$

(7)

The proof of Theorem 1 is postponed until the last section. The following theorem gives the uniform, almost complete convergence of $\widehat{CQ{F}_{\alpha }}\left(x\right)$ with the rate.

Theorem 2.

Under Conditions (H1)–(H5) and if $cdf\left(CQ{F}_{\alpha }\left(x\right)\right|x)>0$, we have

$$\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|\widehat{CQ{F}_{\alpha }}\left(x\right)-CQ{F}_{\alpha }\left(x\right)\right|=O\left({\lambda }_K^{min(k_2,k_2)}\right)+O_{a.co.}\left({\left(\frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}\right)}^{\frac{1}{2}}\right).$$

(8)

The proof of Theorem 2 is postponed until the last section.

4. Asymptotic Normality

The second asymptotic result concerns the asymptotic law of the sequence $\widehat{CDF}(y\mid x)$. To do that, we enhance Assumption (H5) by assuming that the kernel $\mathcal{K}er(·)$ satisfies (H4) and has a first derivative $\mathcal{K}e{r}^{{}^{\prime }}(·)$ such that

$${\displaystyle \mathcal{K}e{r}^2\left(1\right)-{\int }_{-1}^1{\left(\mathcal{K}e{r}^2\left(u\right)\right)}^{{}^{\prime }}\mathsf{\Psi }\left(u\right)du>0.}$$

Let us now state the following theorem, which gives the central limit theorem of the estimator $\widehat{CDF}(y\mid x)$. Below, we write $Z\stackrel{\mathcal{D}}{=}\mathcal{N}(\mu ,{\sigma }^2)$ whenever the random variable Z follows a normal law with expectation $\mu$ and variance ${\sigma }^2$; $\stackrel{\mathcal{D}}{\to }$ denotes the convergence in distribution.

Theorem 3.

Under the assumptions of Theorem 1 and if the smoothing parameters ${\lambda }_K$ satisfy ${\displaystyle \underset{n\to \infty }{lim}n{\lambda }_K^{2k_1}\psi \left({\lambda }_K\right)=0}$, then

$$\sqrt{n\psi \left({\lambda }_K\right)}\left(\widehat{CDF}(y\mid x)-CDF(y\mid x)\right)\stackrel{D}{\to }N(0,V_{JK}(y\mid x)),$$

where

$${\displaystyle M_i=\mathcal{K}e{r}^i\left(1\right)-{\int }_{-1}^1{\left(\mathcal{K}e{r}^i\left(u\right)\right)}^{{}^{\prime }}\Psi \left(u\right)du,}$$

for $i=1,2$ and

$${\displaystyle V_{JK}(y\mid x)=\frac{M_2}{M_1^2}P\left(x\right)CDF(y\mid x)(1-CDF(y\mid x)).}$$

The proof of Theorem 3 is postponed until the last section.

Clearly, this asymptotic result has many applications in practice. In particular, it can be used to build a confidence interval for the true value of $CDF(y\mid x)$. The latter is obtained by estimating the asymptotic variance $V_{JK}(y\mid x)$ using a plug-in approach. Indeed, $M_1$ and $M_2$ are estimated via

$${\displaystyle \widehat{M_1}=\frac{1}{n\psi \left({\lambda }_K\right)}{\Sigma }_{i=1}^n\mathcal{K}e{r}_i\left(x\right),\widehat{M_2}=\frac{1}{n\psi \left({\lambda }_K\right)}{\Sigma }_{i=1}^n\mathcal{K}e{r}_i^2\left(x\right),}$$

and

$$\widehat{P\left(x\right)}=\frac{{\displaystyle {\Sigma }_{i=1}^n{B}_i\mathcal{K}e{r}_i\left(x\right)}}{{\displaystyle {\Sigma }_{i=1}^n\mathcal{K}e{r}_i\left(x\right)}}.$$

Therefore, the natural estimator of the asymptotic variance $V_{JK}(y\mid x)$ is

$${\displaystyle \widehat{V_{JK}}(y\mid x)=\frac{{\widehat{M}}_2}{{\widehat{M}}_1^2}\widehat{P\left(x\right)}\widehat{CDF}(y\mid x)(1-\widehat{CDF}(y\mid x)).}$$

Under this consideration, the ${\displaystyle \left(1-\zeta \right)}$ confidence interval for $CDF(y\mid x)$ is

$$\widehat{CDF}(y\mid x)\pm t_{1-\zeta /2}\times {\left(\frac{\widehat{V_{JK}}(y\mid x)}{n\psi \left({\lambda }_K\right)}\right)}^{1/2},$$

where $t_{1-\zeta /2}$ denotes the $1-\zeta /2$ quantile of the standard normal distribution. We note that the function $\psi (·)$ does not appear in the calculation of the confidence interval since it will be simplified.

5. Numerical Results

In this computational study, we illustrate the three fundamental axes of our topic that are the functional structure, the asymptotic normality, and the local linear smoothing. In the first illustration, we examine the functional structure’s impact on the constructed estimators’ convergence rate. To cover the general features of our study, the ergodicity, and the MAR features, we generate an artificial functional time series using the Hilberthian autoregressive processes. Of course, this kind of process’s linearity allows us to incorporate our theoretical assumption concerning the ergodicity structure. To do this, we employ the routine-code fts.rar from the R package freedom.fda. The latter has a nice feature that is based on the dynamic functional principal component analysis (DFPCA); for instance, see [65]. In this empirical study, we use DFPCA to generate the ergodic function using specific basis functions. Specifically, we have used the Fourier basis functions (FBF) to obtain the functional $\left(X_i\right)$. Formally,

$$X_i=\mathrm{{\rm Y}}\left(X_{i-1}\right)+{\epsilon }_i,$$

where $\mathrm{{\rm Y}}$ is an operator with kernel $\psi (·,·)$ and ${\epsilon }_i$ is the white noise. The kernel operator is expressed by

$$\mathrm{{\rm Y}}\left(X(·)\right)={\int }_0^1\psi (t,s)X\left(s\right)ds.$$

In the cited routine code $\mathrm{{\rm Y}}$ is constructed from FBF $\{u_j:j=1,\dots ,d\}$ by taking ${\left({\psi }_{ij}\right)}_{ij}=\left(\langle \mathsf{\Psi }\left(u_i\right),u_j\rangle \right)$ as the corresponding matrix. The function op.norms controls the degree of dependency in these functional observations for practical purposes. The functional observed regressors are plotted in Figure 1.

Specifically, such a sample was obtained by taking arbitrary, values allowing it to cover various degrees of ergodicity. Secondly, the variable of interest is generated using the regression equation

$$Y=exp\left\{-{\int }_0^1\frac{1}{1+X^2\left(t\right)}dt\right\}+ϵ,$$

where $ϵ$ is drown from $N(0,0.5)$. With this consideration, the conditional law of Y given X is a normal distribution with a mean equal

$$exp\left\{-{\int }_0^1\frac{1}{1+X^2\left(t\right)}dt\right\}.$$

The missing feature is controlled using the conditional probability of observation:

$$P(X=x)=\mathtt{expit}\left(\frac{2\gamma }{\pi }{\int }_0^{2\pi }{x}^2\left(t\right)dt\right),$$

where

$$\mathtt{expit}\left(u\right)=e^u/(1+e^u).$$

Under this consideration, the scalar $\gamma$ controls the missing rate. Once again, we simulate with several values of $\gamma$ to evaluate this characteristic’s influences in the estimators. The sequences LLECCDF and LLECQF are computed using the $(-1,1)$-quadratic kernel and the locally cross-validation on the number of neighborhoods using the mean square error (MSE)

$${\displaystyle \mathtt{MSE}=\frac{1}{n}{\Sigma }_{i=1}^n{\left(Y_i-{\widehat{CQ{F}_{0.5}}}^{-i}\left(X_i\right)\right)}^2,}$$

with ${\widehat{CQ{F}_{0.5}}}^{-i}$ referring to the estimation leave-one-out-curve of the conditional median. Clearly, the Kolmogorov entropy is an important factor of the topological structure. The latter is controlled through the locating functions $\alpha$ and $\delta$. For the sake of shortness, we simulate $\alpha =\delta$ equal to the $L_2$-distance using three between the qth derivatives of the curves based on the basis function, as well as the PCA-semi metric associated with the eigenfunctions of the empirical version covariance operator. Exactly, we compute this metric using the m first one. The MSE was evaluated over various values of q and m.

In

Table 1. MSE error for LLECCDF.

The Metric Type	op.norms	$\mathit{\gamma }$	q or m	MSE
Fourier Basis Functions
	0.98	5	2	0.196
			0	0.193
		0.5	2	0.184
			0	0.187
		0.05	2	0.156
			0	0.155
	0.48	5	2	0.172
			0	0.174
		0.5	2	0.168
			0	0.161
		0.05	2	0.149
			0	0.151
B-spline Basis Functions
	0.48	5	2	0.266
			0	0.264
		0.5	2	0.257
			0	0.259
		0.05	2	0.251
			0	0.248
	0.08	5	2	0.243
			0	0.242
		0.5	2	0.239
			0	0.237
		0.05	2	0.227
			0	0.230
PCA-metric
	0.98	5	3	0.408
			1	0.407
		0.5	3	0.398
			1	0.394
		0.05	3	0.387
			1	0.376
	0.08	5	3	0.143
			1	0.292
		0.5	3	0.295
			1	0.283
		0.05	3	0.284
			1	0.279

and

Table 2. MSE error for LLECQF.

The Metric Type	op.norms	$\mathit{\gamma }$	q or m	MSE
Fourier Basis Functions
	0.98	5	2	0.091
			0	0.093
		0.5	2	0.081
			0	0.085
		0.05	2	0.056
			0	0.054
	0.08	5	2	0.051
			0	0.04
		0.5	2	0.033
			0	0.039
		0.05	2	0.031
			0	0.023
B-spline Basis Functions
	0.98	5	2	0.128
			0	0.123
		0.5	2	0.121
			0	0.122
		0.05	2	0.117
			0	0.114
	0.48	5	2	0.109
			0	0.107
		0.5	2	0.099
			0	0.095
		0.05	2	0.085
			0	0.074
PCA-metric
	0.48	5	3	0.104
			1	0.108
		0.5	3	0.102
			1	0.101
		0.05	3	0.099
			1	0.097
	0.08	5	3	0.086
			1	0.092
		0.5	3	0.081
			1	0.078
		0.05	3	0.064
			1	0.061

, we summarize the MSE of both estimators for various values of mentioned parameters, the level of dependency (op.norms), the missing rate $\gamma$, and the metric parameter q or m.

It is clear that the behavior of the estimator is strongly impacted by different parameters of the estimators, including the dependency-level, the missing rate, and the topological structure. Without surprise, the topological structure has an important role. In a sense, the variability of the MSE as a function of the metric type is more important than the other parameters. All in all, the effectiveness of the estimator is also affected by the quantities (op.norms), $\gamma$, and q or m. In conclusion, we can say that the computational study confirms the theoretical statement that the convergence rate is strongly affected by the topological structure of the functional regressor.

The second illustration concerns the quality of the asymptotic normality result in Theorem 3. Specifically we aim to examine the behavior of the asymptotic distribution with respect to the degree of correlation, as well as the missing rate. For this purpose, we repeat the previous sampling processes independently m times, and for each time, we calculate the quantity:

$$\sqrt{{\displaystyle \frac{n\psi \left({\lambda }_K\right){\widehat{M}}_1^2}{{\widehat{M}}_2\widehat{P\left(x\right)}CDF(y\mid x)(1-CDF(y\mid x))}}}\left(\widehat{CDF}(y\mid x)-CDF(y\mid x)\right).$$

(9)

Recall that the true conditional law of Y given X is known justifying the use of the true conditional cumulative distribution function of $CDF(y\mid x)$. Observe also that the estimation of the function $\psi \left({\lambda }_K\right)$ is not necessary. It will be simplified using the definition of ${\widehat{M}}_1$ and ${\widehat{M}}_2$ (see their definitions in Section 4).

Now, the m-sample of the quantity (9) is calculated using the ideas of the first illustration concerning the construction of $\widehat{CDF}(y\mid x)$. Moreover, the estimators ${\widehat{M}}_1$,   ${\widehat{M}}_2$, and $\widehat{P\left(x\right)}$ are obtained in the same manner. Specifically, we use the same bandwidth, the same kernel, and the metric associated the FBF with $q=0$. Of course, the m-sample of (9) is drawn for a fixed location curve $x=X_{i_0}$ and fixed point $y=Y_{i_0}$. The index $i_0$ is randomly chosen independently from the sampling process. Furthermore, the behavior of the asymptotic distribution of the quantity (9) is examined by estimating the density of the obtained m-sample and we compare it to the density of the standard normal distribution. In order to evaluate the effect of the dependency degree and the missing rate on the accuracy of the asymptotic normality, we perform our sampling process using various values of op.norms and $\gamma$. Exactly, we keep the same value as that of the first illustration. Finally, we plot in Figure 2, Figure 3, Figure 4 and Figure 5 the estimated density, obtained via the routine code density with $m=120$, against the density of $N(0,1)$. The continuous line represents the estimated density, and the dashed line represents the true density.

Once again, this empirical analysis confirms the theoretical development. In a sense, the estimation approach is strongly impacted by the degree of dependency, as well as the missing rate. Typically, even if the curve of the estimated density is relatively close to the normal density, the accuracy of the asymptotic normality is significantly varied with respect to the values of op.norms and $\gamma$. It appears that the effect of the missing rate is more important than the degree of dependency. This conclusion is justified by the fact that the missing rate impacts both the bias and the variance terms, whereas the dependency feature impacts only the variance part. To confirm this statement, we report in

Table 3. Bias error for LLECQF.

op.norms	$\mathit{\gamma }$	$\mathit{Bias}$
0.98	5	0.121
	0.5	0.093
0.48	5	0.109
	0.5	0.091

the bias quantities of the different values of (op.norms) and $\gamma$. The bias term is

$${\displaystyle Bias=\frac{1}{m}{\Sigma }_{j=1}^m{\widehat{CDF}}^j(Y_{i_0}\mid X_{i_0})-CDF(Y_{i_0}\mid X_{i_0}),}$$

where ${\widehat{CDF}}^j(Y_{i_0}\mid X_{i_0})$ is the LLECCDF obtained with the $j^{th}$ sample. The result of Table 3 shows that the effect of the data correlation is too small compared with the variability with respect to the missing rate.

The third illustration concerns the used estimation method that is the local linear approach. More precisely, we compare it with the classical kernel method. We concentrate on this part in the second model, which is the quantile regression. It is well known that this kind of model has a many scopes of application. One of the important application areas is the prediction problem. At this stage, the quantile regression can be used as single-point predictor when $p=0.5$ or as a predictive interval $[CQ{F}_{p/2},CQ{F}_{1-p/2}]$. The latter ensures the existence of the response variable Y in $[CQ{F}_{p/2}\left(x\right),CQ{F}_{1-p/2}\left(x\right)]$ with a probability equal to $1-p$. In order to show the easy implantation of the estimator $\widehat{CQ{F}_{\alpha }}(·)$ and to highlight its advantages over the classical kernel method studied by [4], we compare the efficiency of both estimation methods on the construction of the $[CQ{F}_{p/2}\left(x\right),CQ{F}_{1-p/2}\left(x\right)]$ interval. Undoubtedly, the performance of any predictive interval is measured using two factors that are the coverage probability and the length of the interval. However, for the sake of brevity, we focus in this third illustration only on the coverage probability factor that measures the belonging percentage to the approximated predictive interval. For this aim, we employ the same sampling process as the second illustration, and for each sampling time j, we split the observations into learning and testing sample. We determine $[{\stackrel{-}{CQF}}_{p/2}\left(x_0\right),{\stackrel{-}{CQF}}_{1-p/2}\left(x_0\right)]$ for all point $x_0$ in the testing sample, and $\stackrel{-}{CQF}$ means either the local linear or kernel estimator of the quantile $CQF$. For the computation aspect, we use the routine code funopare.quantile.lcv for the kernel method, whereas that for the local linear method is obtained by minimizing (3) on a regular grid of 100 points, from $(0.9\ast min\left(Y_i\right),1.1\ast max\left(Y_i\right))$. To the end, let us point out that we have used the same selection strategies as the previous illustrations, namely, the same bandwidth, the same kernel, and the same FBF metric. In order to give a comprehensive comparison, we examine the efficiency of the predictive interval processing for various values of p. Exactly, we examine 30 values of $p_i$ in $(0,1)$. Such values are compared with the coverage probabilities of both estimators in the four previous situations covering the strong and weak dependencies, as well as the two levels of missing rates. The results are presented in Figure 6, Figure 7, Figure 8 and Figure 9 for which we plot the different values of $q_i=1-p_i$ versus the coverage probability $C{P}_i$.

Once again, the estimation quality is strongly affected by the two principal features that are the dependency and the missing phenomena. However, it seems that the local linear algorithm is more robust and more preferment in the sense that its behavior is more stable than the kernel method. To confirm this statement, we summarise in

Table 4. Coverage probability error.

Method	op.norms	$\mathit{\gamma }$	$\mathit{ACPE}$
Local linear estimator	0.98	5	0.051
		0.5	0.042
	0.48	5	0.044
		0.5	0.037
Kernel method	0.98	5	0.19
		0.5	0.098
	0.48	5	0.13
		0.5	0.067

, the absolute coverage probability error defined by

$${\displaystyle ACPE=\frac{1}{30}{\Sigma }_{i=1}^{30}|C{P}_i-q_i|,}$$

where $C{P}_i$ represents the coverage probability of the predictive interval associated with the threshold ${\alpha }_i$.

6. Discussion and Comments

• Gap between the pointwise and uniform convergence in FDA: We specify the convergence rate over some known functional structures to highlight the gap between the pointwise and uniform convergence in functional statistics. The first one concerns the resealed version ${(W_c\left(t\right)=W(t/c))}_{t\ge 0}$ of Gaussian process $W{\left(t\right)}_{t\ge 0}$. If we assume that the spectral measure $\mu$ of W such that, for some $a>0$,

  $$\int e^{a\left|\lambda \right|}\mu \left(du\lambda \right)<\infty ,$$

  then the Kolmogorov’s $ϵ$-entropy of ${\mathcal{C}}_{\mathcal{F}}$, the unit ball in reproducing kernel Hilbert spaces of the process $W_c(·)$ as a subset of $(C\left([0,1]\right),\parallel ·{\parallel }_{\infty })$, is of the order

  $$\frac{1}{c}{log}^2\left(\frac{1}{ϵ}\right);$$

  for instance, see [66], implying an uniform convergence rate over ${\mathcal{C}}_{\mathcal{F}}$ asymptotically equal to

  $$O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)+O_{a.co.}\left({\left(\frac{{ln}^{2}n}{n\psi \left({\lambda }_K\right)}\right)}^{\frac{1}{2}}\right).$$
  Secondly, if we put ${\mathcal{C}}_{\mathcal{F}}$ as the closed unit ball of Cameron–Martin spaces associated with the covariance operator of the standard stationary Ornstein–Uhlenbeck process defined by

  $$Cov(s,t)=exp\left(-a|s-t|\right),a>0,$$

  the Kolmogorov’s $ϵ$-entropy of this subset with respect to the norm of the Sobolev space $W^{1,2}\left([0,1]\right)$ is of order

  $$\left(\sqrt{(}2a)\left(\frac{1}{\pi ϵ}\right)\right);$$

  for instance, refer to [66], and the convergence rate is

  $$O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)+O_{a.co.}\left({\left(\frac{lnn}{n\psi \left({\lambda }_K\right)}\right)}^{\frac{1}{2}}\right).$$
  Thirdly, it is shown in [67] that any closed ball in a Sobolev space $W^{1,1}\left([0,T]\right)$ endowed with the norm $L^1\left([0,T]\right)$ has a Kolmogorov’s $ϵ$-entropy of the order

  $$\frac{1}{ϵ}log\frac{1}{ϵ},$$

  implying a uniform convergence rate asymptotically equal to

  $$O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)+O_{a.co.}\left({\left(\frac{{ln}^{2}n}{n\psi \left({\lambda }_K\right)}\right)}^{\frac{1}{2}}\right).$$
  So, it is clear that the uniform convergence rate differs among functional subsets. However, in the nonfunctional case, where all norms are equivalent, the $ϵ$-entropy for any compact subset in $\mathrm{I}\mathrm{R}$ is of the order $log\left(\frac{1}{ϵ}\right)$, allowing us to keep the usual convergence rate in the finite-dimensional case, that is

  $$O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)+O_{a.co.}\left({\left(\frac{lnn}{n{\lambda }_K}\right)}^{\frac{1}{2}}\right),$$

  which also coincides with the pointwise convergence rate. In conclusion, unlike the finite-dimensional case, there is a real gap between the pointwise and the uniform convergence in FDA. Thus, treating uniform consistency in the FDA is a challenging question; refer to [9,68,69,70].
• The effect of the basis function on the Kolmogorov’s entropy:
  Similarly to the previous paragraph, Kolmogorov’s entropy is also affected by the spectral decomposition of the functional variable over specific basis functions. Of course, this relationship is justified by the fact that both tools have similar interpretations. In a sense, the Kolmogorov’s entropy of a given subset ${\mathcal{C}}_{\mathcal{F}}$ is the number of binary bits of information required to describe any $x\in {\mathcal{C}}_{\mathcal{F}}$ with error $ϵ$, while the spectral decomposition of $x\in {\mathcal{C}}_{\mathcal{F}}$ given a basis functions $\left(f_i\right)$ can be viewed as the reconstruction of $x\in {\mathcal{C}}_{\mathcal{F}}$. Furthermore, the minimal cardinal of elements of the basis $\left(f_i\right)$, sufficient to reconstruct any $x\in {\mathcal{C}}_{\mathcal{F}}$ with error $ϵ$, is so-called the sampling $ϵ$-entropy of ${\mathcal{C}}_{\mathcal{F}}$; for instance, see [71]. Theorem 4.1 in this cited work provides, under some general conditions, that the class of bounded piecewise $C^k$ smooth functions as a subspace of $L^2$ has a Kolmogorov’s $ϵ$-entropy of the order

  $${\left(\frac{1}{ϵ}\right)}^{1/k}.$$
  In contrast, Kolmogorov’s $ϵ$-entropy of the class of periodic real analytic functions on $[-\pi ,\pi ]$, as a subspace of $C^0\left([-\pi ,\pi ]\right)$ is

  $${log}^2\left(\frac{1}{ϵ}\right).$$
  As a consequence of this statement (see Corollary 4.2 in [71]), the sampling $ϵ$-entropy of the class spanned by the B-splines basis function is greater than the sampling $ϵ$-entropy of the class of periodic real analytic functions that is spanned for the Fourier basis function. In conclusion, in practice, the estimator’s accuracy is also affected by the choice of basis functions and the cardinal of the basis used in the metric.
• Estimation of the unconditional distribution of the MAR response: One of the fundamental applications of conditional modeling in the MAR structure is the possibility of reconstructing the feature of the MAR variable. In our context, we use the fact that

  $$F_Y\left(y\right)=\mathbb{P}(Y\le y)=\mathrm{I}\mathrm{E}\left[CDF\left(y\right|X)\right].$$
  Thus, the natural estimator of the cumulative distribution function is

  $${\displaystyle \widehat{F_Y}\left(y\right)=\frac{1}{n}{\Sigma }_{i=1}^n\widehat{CD{F}^{-i}}(y\mid X_i),}$$

  where $\widehat{CD{F}^{-i}}(y\mid X_i)$ is the estimator $\widehat{CDF}(·\mid ·)$ without the observation $(X_i,Y_i,B_i)$. An alternative estimator of $\widehat{F_Y}\left(y\right)$ can be obtained by replacing only the missing observation with the conditional expectation. Specifically, the second estimator is expressed by

  $${\displaystyle \widetilde{F_Y}\left(y\right)=\frac{1}{n}{\Sigma }_{i=1}^n\left(B_i{11}_{[Y_i<y]}+(1-B_i)\widehat{CD{F}^{-i}}(y\mid X_i)\right).}$$
  We return to [48] for more ideas to construct other estimators. At this stage, the uniform consistency obtained in this paper is an important preliminary tool to derive the root n-consistency of this kind of estimator; we refer to [48] for the regression case.

7. Concluding Remarks

The convergence rate for a free-distribution functional data analysis is challenging. It requires some advanced tools for functional analysis in pure mathematics. This paper gives several contributions to the existing literature on functional data analysis. First, this paper demonstrated that the Kolmogorov entropy is an essential tool for describing the convergence rate of local linear estimation (LLE). To determine the uniform convergence rate of the LLE of the conditional cumulative distribution function (LLECCDF) and the LLE conditional quantile function (LLECQF), we have used this device. Second, a central limit theorem is established for the proposed estimators. These results are demonstrated under general assumptions that permit coverage of the case of incomplete functional time series. Specifically, we model the dependence using the ergodic assumption and assume that the response variable is missing at random (MAR). Finally, we have evaluated the finite sample performance of the proposed estimators using Monte Carlo simulations. In addition to the previous issues, the present paper opens some important paths for the future. It will be natural to consider in a future investigation of the functional kNN local linear approach quantile regression estimators to obtain an alternative estimator that benefits from the advantages of both methods, the local linear method, and the kNN approach. Extending nonparametric functional concepts to local stationary processes is a relatively underdeveloped field. It would be intriguing to extend our work to the case of the functional local stationary process, which requires nontrivial mathematics; however, doing so would be far beyond the scope of this paper.

8. Proofs

This section contains the proof of our results. The notation introduced previously will be employed in the following. All of the proofs rely on applying the exponential inequality of the martingale difference. The proofs are quite lengthy; we limit ourselves to the main arguments.

8.1. Proof of the Main Theorems

Proof of Theorem 1.

For (7), we write

$$\begin{array}{ccc}\hfill {\stackrel{-}{CDF}}_N^x\left(y\right)& :=& {\displaystyle \frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right){\mathcal{J}}_j\left(x\right)B_j\right|{\mathfrak{F}}_{j-1}),}\hfill \\ \hfill {\stackrel{-}{CDF}}_D\left(x\right)& :=& {\displaystyle \frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)B_j\right|{\mathfrak{F}}_{j-1}).}\hfill \end{array}$$

Then, the statement (7) is based on the following decomposition:

$$\begin{array}{l}\widehat{CDF}(y\mid x)-CDF(y\mid x)\\ =B_n(y\mid x)+\frac{1}{{\widehat{CDF}}_D\left(x\right)}\left[(B_n(y\mid x)+CDF(y\mid x))A_n(y\mid x)+R_n(y\mid x)\right],\end{array}$$

(10)

where

$${\displaystyle B_n(y\mid x)=\frac{{\stackrel{-}{CDF}}_N^x\left(y\right)}{{\stackrel{-}{CDF}}_D\left(x\right)}-CDF(y\mid x),}$$

$$A_n(y\mid x)={\stackrel{-}{CDF}}_D\left(x\right)-{\widehat{CDF}}_D\left(x\right),$$

and

$$R_n(y\mid x)={\widehat{CDF}}_N^x\left(y\right)-{\stackrel{-}{CDF}}_N^x\left(y\right).$$

So, all that remains is to demonstrate Lemmas 1–3:

Lemma 1.

Under Conditions (H1)–(H5), we have, as $n\to \infty$,

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|B_n(y\mid x)\right|=O\left({\lambda }_K^{b_1}+{\lambda }_J^{b_2}\right).}$$

Lemma 2.

Under Conditions (H1)–(H5), we have, as $n\to \infty$,

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|R_n(y\mid x)\right|=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right).}$$

Lemma 3.

Under Conditions (H1)–(H5), we have, as $n\to \infty$,

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|A_n(y\mid x)\right|=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right),}$$

and

$${\displaystyle {\Sigma }_{n=1}^{\infty }\mathbb{P}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}{\widehat{CDF}}_D\left(x\right)<\frac{1}{2}}\right)<\infty .}$$

□

Proof of Theorem 2.

For (8), we define for vector ${\displaystyle \overrightarrow{\delta }=\left(\begin{array}{c}c\\ d\end{array}\right)}$, we put

$$\mathsf{\Phi }\left(\overrightarrow{\delta }\right)=\alpha -{11}_{{\displaystyle Y_j\le (c+{\beta }_1)+({\lambda }_K^{-1}d+{\beta }_2){\alpha }_j}},$$

and we employ Lemma 3 in [30] for

$${\displaystyle V_n(\overrightarrow{\delta },x)=\frac{1}{n\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n\Phi \left(\overrightarrow{\delta }\right)\left(\begin{array}{c}1\\ {\lambda }_K^{-1}{\alpha }_j\end{array}\right)\mathcal{K}e{r}_j\left(x\right)B_j,}$$

$${\displaystyle A_n\left(x\right)=V_n(\overrightarrow{{\delta }_0},x)with\overrightarrow{{\delta }_0}=\left(\begin{array}{c}0\\ 0\end{array}\right)and{\delta }_n=\left(\begin{array}{c}\widetilde{{\beta }_1}-{\beta }_1\\ {\lambda }_K(\widetilde{{\beta }_2}-{\beta }_2)\end{array}\right).}$$

Thus, (8) is a consequence of Lemmas 4 and 5.

Lemma 4.

Under Conditions (H1)–(H5), we have, as $n\to \infty$,

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel A_n\left(x\right)\parallel =O\left({\lambda }_K^{min(k_2,k_2)}\right)+O_{a.co.}\left({\left(\frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}\right)}^{1/2}\right).}$$

Lemma 5.

Under Conditions (H1)–(H5), we have, as $n\to \infty$,

$$\begin{array}{l}{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}}{\displaystyle \underset{\parallel \overrightarrow{\delta }\parallel \le M}{sup}}\parallel V_n(\overrightarrow{\delta },x)+{\lambda }_{0}D\overrightarrow{\delta }-A_n\left(x\right)\parallel \\ =O\left({\lambda }_K^{min(k_2,k_2)}\right)+O_{a.co.}\left({\left(\frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}\right)}^{1/2}\right),\end{array}$$

where

$${\lambda }_0=cdf\left(CQ{F}_p\left(x\right)\right|x)P\left(x\right).$$

□

Proof of Theorem 3.

Using the decomposition

$$\begin{array}{c}\hfill {\displaystyle \widehat{CDF}(y\mid x)-CDF(y\mid x)=\frac{C_n(y\mid x)+Q_n(y\mid x)}{{\widehat{CDF}}_D\left(x\right)}+B_n(y\mid x),}\end{array}$$

(11)

with

$$Q_n(y\mid x)=R_n(y\mid x)+CDF(y\mid x)A_n(y\mid x),$$

and

$$C_n(y\mid x)=B_n(y\mid x)A_n(y\mid x).$$

Then, the asymptotic normality can be demonstrated via the following Lemmas.

Lemma 6.

Under the assumptions of Theorem 3, we have, as $n\to \infty$,

$$\begin{array}{c}\hfill \sqrt{{\displaystyle \frac{n\psi \left({\lambda }_K\right)}{V_{JK}(y\mid x)}}}{Q}_n(y\mid x)\stackrel{D}{\to }N(0,1).\end{array}$$

(12)

□

8.2. Proof of the Technical Lemmas

Proof of Lemma 1.

Writing

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|B_n(y\mid x)\right|=\frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|{\tilde{B}}_n(y\mid x)\right|}}{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}\left|{\stackrel{-}{CDF}}_D\left(x\right)\right|}},}$$

where

$${\tilde{B}}_n(y\mid x)={\stackrel{-}{CDF}}_N^x\left(y\right)-CDF(y\mid x){\stackrel{-}{CDF}}_D\left(x\right).$$

The latter is

$$\begin{array}{ccc}\hfill {\tilde{B}}_n(y\mid x)& =& {\displaystyle \frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n\left\{\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right){\mathcal{J}}_j{B}_j\right|{\mathfrak{F}}_{j-1})\right.}\hfill \\ & & \left.-CDF(y\mid x)\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)B_j\right|{\mathfrak{F}}_{j-1})\right\}\hfill \\ & \le & {\displaystyle \frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}}\hfill \\ & & {\displaystyle {\Sigma }_{j=1}^n\left\{P\left(x\right)\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)\right|\mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j\left(y\right)\right|X_j]-CDF(y\mid x)\left|\right|{\mathfrak{F}}_{j-1})\right\}.}\hfill \end{array}$$

(13)

The integration by part gives

$${\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)|\mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j\right|X_j]-CDF(y\mid x)|\le C\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right).$$

(14)

Hence, we obtain

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|{\tilde{B}}_n(y\mid x)\right|=O\left({\lambda }_K^{k_1}+{\lambda }_J^{k_2}\right)\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\stackrel{-}{CDF}}_D\left(x\right).}$$

This completes the proof. □

Proof of Lemma 2.

Firstly, observe that, for all $x\in {\mathcal{C}}_{\mathcal{F}}$,

$$|{\Gamma }_j\left(x\right)|\le nC{\lambda }_K^2+nC{\lambda }_K\left|{\alpha }_j\left(x\right)\right|.$$

(15)

Next, we let ${\displaystyle k\left(x\right)=arg\underset{k\in \{1,2,\dots ,d_n\}}{min}\left|\delta \right|(x,x_k)}$. We have

$$\begin{array}{ccc}\hfill {\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\left|R_n(y\mid x)\right|}& \le & \underset{Q_1}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid {\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid }}}\hfill \\ & & +\underset{Q_2}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid {\widehat{CDF}}_N^x\left(y\right)-{\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid }}}\hfill \\ & & +\underset{Q_3}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid {\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^x\left(y\right)\mid }}}.\hfill \end{array}$$

We start by treating $Q_2$. By using the boundeness on $\mathcal{K}er(·)$ and $\mathcal{J}(·)$, we write

$$\begin{array}{ccc}\hfill Q_2& \le & {\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\frac{1}{n}{\Sigma }_{j=1}^n\mid {\mathcal{J}}_j\left(y\right)\mid \mid \frac{1}{\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Gamma }_j\left(x\right)\mathcal{K}e{r}_j\left(x\right)B_j{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)}\hfill \\ & & {\displaystyle -\frac{1}{\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x_k\left(x\right)\right)\mathcal{K}e{r}_1\left(x_k\left(x\right)\right)\right)}{\Gamma }_j\left(x_k\left(x\right)\right)B_j\mathcal{K}e{r}_j\left(x_k\left(x\right)\right){\mathbb{I}}_{\mathcal{B}a(x_k\left(x\right),{\lambda }_K)}\left(X_j\right)\mid }\hfill \\ & \le & \left({\displaystyle \frac{C}{n^2{\lambda }_K^2\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n\mid {\Gamma }_j\left(x\right)B_j{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\mid \times }\right.\hfill \\ & & {\displaystyle \left.\mid \mathcal{K}e{r}_j\left(x\right)-\mathcal{K}e{r}_j\left(x_k\left(x\right)\right){\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)\mid \right)}\hfill \\ & & +\left({\displaystyle \frac{C}{n^2{\lambda }_K^2\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n\mathcal{K}e{r}_j\left(x_{k\left(x\right)}\right)B_j{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)\times }\right.\hfill \\ & & \left.\mid {\Gamma }_j\left(x\right){\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)-{\Gamma }_j\left(x_k\left(x\right)\right)\mid \right)\hfill \\ & :=& F_1+F_2.\hfill \end{array}$$

By using the fact that the $\mathcal{K}er(·)$ satisfies the Lipschitz condition, we obtain via (15)

$$\begin{array}{l}\left|{\Gamma }_j\left(x\right)\mid {\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\mid \mathcal{K}e{r}_j\left(x\right)-\mathcal{K}e{r}_j\left(x_k\left(x\right)\right){\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)\right|\\ \le {\displaystyle nC{\lambda }_K^2(\frac{r_n}{{\lambda }_K}{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)}\\ +{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)),\end{array}$$

from which we infer that

$$\begin{array}{ccc}\hfill F_1& \le & {\displaystyle \frac{C{r}_n}{n{\lambda }_K\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)B_j}\hfill \\ & & {\displaystyle +\frac{C}{n\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)B_j.}\hfill \end{array}$$

Concerning $F_2$ we put, for $l=0,1$ and $k=1,2,$

$$\begin{array}{ccc}\hfill T^{k,l}& =& {\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\hfill \\ & & \times \left|\left({\displaystyle {\Sigma }_{i=1}^n{\alpha }_i^k\left(x\right)\mathcal{K}e{r}_i\left(x\right)B_i}\right){\alpha }_j^l\left(x\right)-\left({\displaystyle {\Sigma }_{i=1}^n{\alpha }_i^k\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_i\left(x_{k\left(x\right)}\right)B_i}\right){\alpha }_j^l\left(x_{k\left(x\right)}\right)\right|.\hfill \end{array}$$

Therefore, we obtain

$$T^{k,l}\le T_1^{k,l}+T_2^{k,l},$$

where

$$\begin{array}{c}\hfill T_1^{k,l}={\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\left({\displaystyle {\Sigma }_{i=1}^n\mid {\alpha }_i^k\left(x\right)\mid \mathcal{K}e{r}_i\left(x\right)B_i\times \mid {\alpha }_j^l\left(x\right)-{\alpha }_j^l\left(x_{k\left(x\right)}\right)\mid }\right),\end{array}$$

and

$$\begin{array}{ccc}\hfill T_2^{k,l}& =& {\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)(\mid {\alpha }_j^l\left(x_{k\left(x\right)}\right)\mid B_i\hfill \\ & & \times \left|{\displaystyle {\Sigma }_{i=1}^n({\alpha }_i^k\left(x\right)\mathcal{K}e{r}_i\left(x\right)-{\alpha }_i^k\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_i\left(x_{k\left(x\right)}\right))}\right|,\hfill \end{array}$$

Making use of the condition $\left(H4\right)$, for $l=1$, we have

$${\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\mid {\alpha }_j\left(x\right)-{\alpha }_j\left(x_{k\left(x\right)}\right)\mid \le C{r}_n{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).$$

So, for $k=2,$ $l=0$,

$$\begin{array}{c}\hfill T_1^{k,l}=0,\end{array}$$

(16)

and for $k=1,$ $l=1$,

$$\begin{array}{c}\hfill T_1^{k,l}\le nC{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).\end{array}$$

Now, for $T_2^{k,l}$, we have

$$\begin{array}{ccc}\hfill T_2^{k,l}& \le & {\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\hfill \\ & & \times \left({\displaystyle {\Sigma }_{i=1}^n\mid {\alpha }_j^l\left(x_{k\left(x\right)}\right)\mid \mathcal{K}e{r}_i\left(x\right)B_i\times \mid {\alpha }_i^k\left(x\right)-{\alpha }_i^k\left(x_{k\left(x\right)}\right)\mid }\right)\hfill \\ & & +{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\hfill \\ & & \times \left({\displaystyle {\Sigma }_{i=1}^n\mid {\alpha }_j^l\left(x_{k\left(x\right)}\right)\mid \mid {\alpha }_i^k\left(x_{k\left(x\right)}\right)\mid B_i\times \mid \mathcal{K}e{r}_i\left(x\right)-\mathcal{K}e{r}_i\left(x_{k\left(x\right)}\right)\mid }\right).\hfill \end{array}$$

We have

$${\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\mid {\alpha }_i^2\left(x\right)-{\alpha }_i^2\left(x_{k\left(x\right)}\right)\mid \le C{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).$$

This implies that, for $k=1,2$,

$${\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\mid {\alpha }_i^k\left(x\right)-{\alpha }_i^k\left(x_{k\left(x\right)}\right)\mid \le C{r}_n{\lambda }_K^{k-1}{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).$$

Thus, for $(l,k)=(0,2)$, we have

$$\begin{array}{c}\hfill T_2^{k,l}\le nC{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right),\end{array}$$

(17)

and for $l=k=1$

$$\begin{array}{c}\hfill T_2^{k,l}\le nC{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).\end{array}$$

(18)

Combining (16) and (17), we find that

$$\begin{array}{l}{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\left|\left({\displaystyle {\Sigma }_{i=1}^n{\alpha }_i^2\left(x\right)\mathcal{K}e{r}_i\left(x\right)B_i-{\alpha }_i^2\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_i\left(x_{k\left(x\right)}\right)B_i}\right)\right|\\ \le nC{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right),\end{array}$$

and

$$\begin{array}{l}{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right)\\ \times \left|\left({\displaystyle \left({\Sigma }_{i=1}^n{\alpha }_i\left(x\right)\mathcal{K}e{r}_i\left(x\right)B_i\right){\alpha }_j\left(x\right)-\left({\Sigma }_{i=1}^n{\alpha }_i\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_i\left(x_{k\left(x\right)}\right)B_i\right){\alpha }_j\left(x_{k\left(x\right)}\right)}\right)\right|\\ \le nC{r}_n{\lambda }_K{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \mathcal{B}a(x,{\lambda }_K)}\left(X_j\right).\end{array}$$

Thus, we obtain

$$\begin{array}{ccc}\hfill F_2& \le & {\displaystyle \frac{C{r}_n}{n{\lambda }_K\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)B_j}\hfill \\ & & {\displaystyle +\frac{C}{n\psi \left({\lambda }_K\right)}\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)B_j.}\hfill \end{array}$$

Consequently, we infer that

$${\displaystyle Q_2\le C\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}(Q_{2.1}+Q_{2.2}+Q_{2.3}),}$$

where

$$\begin{array}{ccc}\hfill Q_{2.1}& =& {\displaystyle \frac{C}{n\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \overline{\mathcal{B}a(x,{\lambda }_K)}}\left(X_j\right)B_j,}\hfill \\ \hfill Q_{2.2}& =& {\displaystyle \frac{C{r}_n}{n{\lambda }_K\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}\left(X_j\right)B_j,}\hfill \\ \hfill Q_{2.3}& =& {\displaystyle \frac{C}{n\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)B_j.}\hfill \end{array}$$

Now, we apply an exponential inequality for the difference martingale random variables of $Z_j$ that are defined by

$$\begin{array}{c}\hfill Z_j=\left\{\begin{array}{cc}{\displaystyle \frac{1}{\psi \left({\lambda }_K\right)}\left[{\mathbb{I}}_{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)\cap \overline{\mathcal{B}a(x,{\lambda }_K)}}\left(X_j\right)\right]B_j}\hfill & \mathrm{for}{Q}_{1.1},\hfill \\ \\ {\displaystyle \frac{r_n}{{\lambda }_K\psi \left({\lambda }_K\right)}\left[{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)\right]B_j}\hfill & \mathrm{for}{Q}_{1.2},\hfill \\ \\ {\displaystyle \frac{1}{{\psi }_K}\left[{\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)\cap \overline{\mathcal{B}a(x_{k\left(x\right)},{\lambda }_K)}}\left(X_j\right)\right]B_j}\hfill & \mathrm{for}{Q}_{1.3}.\hfill \end{array}\right.\end{array}$$

Keep in mind that

$$\begin{array}{ccc}\hfill Z_j& =& O\left({\displaystyle \frac{1}{\psi \left({\lambda }_K\right)}}\right),\hfill \\ \hfill \mathrm{I}\mathrm{E}\left[Z_j\right|{\mathfrak{F}}_{j-1}]& =& O\left({\displaystyle \frac{r_n}{\psi \left({\lambda }_K\right)}}\right)\hfill \\ \hfill \mathrm{I}\mathrm{E}\left[Z_j^2\right|\left|{\mathfrak{F}}_{j-1}\right]& =& O\left({\displaystyle \frac{r_n}{\psi {\left({\lambda }_K\right)}^2}}\right).\hfill \end{array}$$

Therefore, we infer

$${\displaystyle Q_{2.1}=O\left(\frac{r_n}{\psi \left({\lambda }_K\right)}\right)+O_{a.co.}\left(\sqrt{{\displaystyle \frac{r_{n}lnn}{n\psi {\left({\lambda }_K\right)}^2}}}\right),}$$

and

$$Q_{2.2}=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right).$$

We deduce

$$Q_2=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right),$$

and $\forall \epsilon >0,$ we obtain

$$\begin{array}{l}\mathrm{I}\mathrm{P}\left(Q_1>\epsilon \sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right)\\ = {\displaystyle \mathrm{I}\mathrm{P}}\left(\underset{k\in 1,\dots ,d_n}{max}\mid {\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid >\epsilon \right)\\ \le d_n\underset{k\in 1,\dots ,d_n}{max}{\displaystyle \mathrm{I}\mathrm{P}}\left({\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid >\epsilon \sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right).\end{array}$$

Let

$${\displaystyle {\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)=\frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\left(x\right)\mathcal{K}e{r}_1\left(x\right)\right)}{\Sigma }_{j=1}^n{\mathcal{S}}_j,}$$

where

$${\mathcal{S}}_j={\Gamma }_j\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_j\left(x_{k\left(x\right)}\right){\mathcal{J}}_j\left(y\right)B_j-\mathrm{I}\mathrm{E}\left({\Gamma }_j\left(x_{k\left(x\right)}\right)\mathcal{K}e{r}_j\left(x_{k\left(x\right)}\right){\mathcal{J}}_j\left(y\right)B_j\right|{\mathfrak{F}}_{j-1}).$$

We readily obtain

$$\mathrm{I}\mathrm{E}\left({\mathcal{S}}_j^2\right|{\mathfrak{F}}_{j-1})\le 2C^{{}^{\prime }}{n}^2{\lambda }_K^4{\psi }_j\left({\lambda }_K\right).$$

Now, for all $\epsilon >0$, we infer that

$$\begin{array}{l}\mathrm{I}\mathrm{P}\left(\mid {\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid >\epsilon \sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right)\\ \le {\displaystyle \mathrm{I}\mathrm{P}}\left({\displaystyle |\frac{1}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)}{\Sigma }_{j=1}^n{\mathcal{S}}_j|>\epsilon \sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}}\right)\\ \le 2{exp}^{-C_0{\epsilon }^{2}ln{d}_n}.\end{array}$$

Choosing $\epsilon$ for which $C_0{\epsilon }^2=\varsigma$, we then infer

$$d_n\underset{k\in 1,\dots ,d_n}{max}{\displaystyle \mathrm{I}\mathrm{P}}\left(\mid {\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)\mid >\epsilon \sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right)\le C^{{}^{\prime }}{d}_n^{1-\varsigma }.$$

Since ${\displaystyle {\Sigma }_{n=1}^{\infty }{d}_n^{1-\varsigma }<\infty ,}$ we obtain that

$$Q_1=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right).$$

For $Q_3$, we have

$$Q_3\le \mathrm{I}\mathrm{E}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}|{\widehat{CDF}}_N^{x_k\left(x\right)}\left(y\right)-{\stackrel{-}{CDF}}_N^{x_k\left(x\right)}\left(y\right)|\mid {\mathfrak{F}}_{j-1}}\right).$$

We follow the proof for $Q_1$ to obtain

$$Q_3=O_{a.co.}\left(\sqrt{{\displaystyle \frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}}}\right),$$

which achieves the demonstration of Lemma 2. □

Proof of Lemma 3.

Obviously, the first statement is deduced by taking ${\mathcal{J}}_j=1$ in Lemma 2, while for the second result, we use

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}\mid {\widehat{CDF}}_D\left(x\right)\mid \le \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2},}$$

meaning that there exists $x\in {\mathcal{C}}_{\mathcal{F}}$ such that

$$\begin{array}{c}{\displaystyle P\left(x\right)-{\widehat{CDF}}_D\left(x\right)\ge \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2}}\hfill \\ {\displaystyle ⟹\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid P\left(x\right)-{\widehat{CDF}}_D\left(x\right)\mid \ge \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2}.}\hfill \end{array}$$

So, we have

$$\begin{array}{l}\mathrm{I}\mathrm{P}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}\mid {\widehat{CDF}}_D\left(x\right)\le \frac{1}{2}}\right)\\ \le {\displaystyle \mathrm{I}\mathrm{P}}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid P\left(x\right)-{\widehat{CDF}}_D\left(x\right)\mid \ge \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2}}\right).\\ \le {\displaystyle \mathrm{I}\mathrm{P}}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\mid \mathrm{I}\mathrm{E}({\widehat{CDF}}_D\left(x\right)-{\widehat{CDF}}_D\left(x\right)\mid \ge \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2}}\right).\end{array}$$

Consequently,

$${\displaystyle {\Sigma }_{i=1}^{\infty }{\displaystyle \mathrm{I}\mathrm{P}}\left({\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}\mid {\widehat{CDF}}_D\left(x\right)\mid \le \frac{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{inf}P\left(x\right)}}{2}}\right)<\infty .}$$

Hence, the proof is complete. □

Proof of Lemma 4.

Notice that

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel A_n\left(x\right)\parallel \le \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel A_n\left(x\right)-{\stackrel{-}{A}}_n\left(x\right)\parallel +\underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel {\stackrel{-}{A}}_n\left(x\right)\parallel ,}$$

where

$${\displaystyle {\stackrel{-}{A}}_n\left(x\right):=\frac{1}{n\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left(\Phi \left(\overrightarrow{{\delta }_0}\right)\left(\begin{array}{c}1\\ {\lambda }_K^{-1}{\alpha }_j\end{array}\right)\mathcal{K}e{r}_j\left(x\right)B_j|{\mathfrak{F}}_{j-1}\right).}$$

So, all that remains is to prove that

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel A_n\left(x\right)-{\stackrel{-}{A}}_n\left(x\right)\parallel =O_{a.co.}\left({\left(\frac{ln{d}_n}{n\psi \left({\lambda }_K\right)}\right)}^{1/2}\right)}$$

(19)

and

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\parallel {\stackrel{-}{A}}_n\left(x\right)\parallel =O\left({\lambda }_K^{min(k_2,k_2)}\right).}$$

(20)

The proof of (19) follows the same lines as the Lemma 2, while (20) is based on the same idea as in Lemma 2 using the fact that

$${\mathbb{I}}_{\mathcal{B}a(x,{\lambda }_K)}\left(X_1\right)\left|CDF\left(CQ{F}_p\left(x\right)\right|x)-CDF({\beta }_1+{\beta }_2{\alpha }_1|X_1)\right|\le C\left({\lambda }_K^{min(k_1,k_2)}\right).$$

This completes the proof of Lemma 4. □

Proof of Lemma 5.

We prove that

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}\parallel V_n(\overrightarrow{\delta },x)-A_n\left(x\right)-({\stackrel{-}{V}}_n(\overrightarrow{\delta },x)-{\stackrel{-}{A}}_n\left(x\right))\parallel =O_{a.co.}\left(\sqrt{\frac{logn}{n\psi \left({\lambda }_K\right)}}\right),}$$

(21)

and

$${\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}\parallel ({\stackrel{-}{V}}_n(\overrightarrow{\delta },x)-{\stackrel{-}{A}}_n\left(x\right))+cdf\left(CQ{F}_p\left(x\right)\right|x)D\delta \parallel =O\left({\lambda }_K^{min(k_1,k_2)}\right),}$$

(22)

where

$${\displaystyle {\stackrel{-}{V}}_n(\overrightarrow{\delta },x):=\frac{1}{n\psi \left({\lambda }_K\right)}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left(\Phi \left(\overrightarrow{\delta }\right)(\begin{array}{c}1\\ {\lambda }_K^{-1}{\alpha }_j\end{array}\right)\mathcal{K}e{r}_j\left(x\right)B_j|{\mathfrak{F}}_{j-1}).}$$

In order to show (21), we keep the definition of $k\left(x\right)$ as in Lemma 2, we use the compactness of the ball $B(0,M)$ in ${\mathrm{I}\mathrm{R}}^2$, and we write

$$B(0,M)\subset \bigcup _{j=1}^{d_n}B({\delta }_j,l_n),{\delta }_j=\left(\begin{array}{c}{c}_j\\ d_j\end{array}\right)\mathrm{and}{l}_n=d_n^{-1}=1/\sqrt{n}.$$

Then, we take $j\left(\delta \right)=arg{min}_j|\delta -{\delta }_j|$. Similarly to Lemma 2 we write

$$\begin{array}{l}{\displaystyle {\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}\parallel V_n(\overrightarrow{\delta },x)-A_n\left(x\right)-({\stackrel{-}{V}}_n(\overrightarrow{\delta },x)-{\stackrel{-}{A}}_n\left(x\right))\parallel }}\\ \le {\displaystyle \underset{T_1}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}∥V_n(\overrightarrow{\delta },x)-A_n\left(x\right)-(V_n(\overrightarrow{\delta },x_{k\left(x\right)})-A_n\left(x_{k\left(x\right)}\right))∥}}}}\\ +\underset{T_2}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}∥V_n(\overrightarrow{\delta },x_{k\left(x\right)})-V_n({\overrightarrow{\delta }}_{j\left(\delta \right)},x_{k\left(x\right)})∥}}}\\ +\underset{T_3}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}∥V_n({\overrightarrow{\delta }}_{j\left(\delta \right)},x_{k\left(x\right)})-A_n\left(x_{k\left(x\right)}\right)-({\stackrel{-}{V}}_n({\overrightarrow{\delta }}_{j\left(\delta \right)},x_{k\left(x\right)})-{\stackrel{-}{A}}_n\left(x_{k\left(x\right)}\right))∥}}}\\ +\underset{T_4}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}∥{\stackrel{-}{V}}_n({\overrightarrow{\delta }}_{j\left(\delta \right)},x_{k\left(x\right)})-{\stackrel{-}{V}}_n(\overrightarrow{\delta },x_{k\left(x\right)})∥}}}\\ +\underset{T_5}{\underbrace{{\displaystyle \underset{x\in {\mathcal{C}}_{\mathcal{F}}}{sup}\underset{\parallel \delta \parallel \le M}{sup}∥{\stackrel{-}{V}}_n(\overrightarrow{\delta },x_{k\left(x\right)})-{\stackrel{-}{A}}_n\left(x_{k\left(x\right)}\right)-({\stackrel{-}{V}}_n(\overrightarrow{\delta },x)-{\stackrel{-}{A}}_n\left(x\right))∥}}}.\end{array}$$

We treat $T_1$ and $T_2$ as $Q_2$ in Lemma 2. Meanwhile, $T_4$ and $T_5$ are evaluated as in $Q_3$. Finally, we use the idea of $Q_1$ to evaluate $T_3$. The statement (22) is a consequence of

$${\lambda }_K^{-a}\mathrm{I}\mathrm{E}\left[{\alpha }_i^{-a}\mathcal{K}e{r}_i^b\right]=\psi \left({\lambda }_K\right)\left(\mathcal{K}e{r}^b\left(1\right)-{\int }_{-1}^1{\left(u^a\mathcal{K}e{r}^b\left(u\right)\right)}^{\prime }\mathsf{\Psi }\left(u\right)du\right)+o\left(\psi \left({\lambda }_K\right)\right).$$

Hence, the proof is complete. □

Proof of Lemma 1.

For a fixed x, we put all $j=1,\dots ,n,$$({\Gamma }_1\left(x\right),\mathcal{K}e{r}_1\left(x\right))=({\Gamma }_1,\mathcal{K}e{r}_1)$. Next, let us denote

$${\displaystyle {\eta }_{n,j}=\frac{\sqrt{n\psi \left({\lambda }_K\right)}}{n\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)}({\mathcal{J}}_j-CDF(y\mid x)){\Gamma }_j\mathcal{K}e{r}_j{B}_j,}$$

and we define ${\xi }_{n,j}={\eta }_{n,j}-\mathrm{I}\mathrm{E}\left({\eta }_{n,j}\right|{\mathfrak{F}}_{j-1}).$ It is clear that

$$\begin{array}{c}\hfill {\displaystyle \sqrt{n\psi \left({\lambda }_K\right)}{Q}_n(y\mid x)={\Sigma }_{j=1}^n{\xi }_{n,j}.}\end{array}$$

(23)

Therefore, to demonstrate Lemma 6, we have to prove the following:

$${\displaystyle {\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\xi }_{n,j}^2\right|{\mathfrak{F}}_{j-1})\stackrel{\mathbb{P}}{\to }{V}_{JK}(y\mid x),}$$

(24)

and

$$\forall \epsilon >0n\mathrm{I}\mathrm{E}\left({\xi }_{n,j}^2{\mathbb{I}}_{\left[\right|{\xi }_{n,j}^2|>\epsilon ]}\right)=o\left(1\right).$$

(25)

For (24), we have

$$\mathrm{I}\mathrm{E}\left({\xi }_{n,j}^2\right|{\mathfrak{F}}_{j-1})=\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2\right|{\mathfrak{F}}_{j-1})-{\left(\mathrm{I}\mathrm{E}\left({\eta }_{n,j}\right|{\mathfrak{F}}_{j-1})\right)}^2.$$

Thus, it suffices to show that

$${\displaystyle \underset{n\to \infty }{lim}{\Sigma }_{j=1}^n{\left(\mathrm{I}\mathrm{E}\left({\eta }_{n,j}\right|{\mathfrak{F}}_{j-1})\right)}^2=0\mathrm{in}\mathrm{probability},}$$

(26)

and

$${\displaystyle \underset{n\to \infty }{lim}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2\right|{\mathfrak{F}}_{j-1})=V_{JK}(y\mid x)\mathrm{in}\mathrm{probability}.}$$

(27)

For (26), we use Lemma 5 of [72] to conclude

$$\begin{array}{ccc}\hfill \left|\mathrm{I}\mathrm{E}\right({\eta }_{n,j}\left|{\mathfrak{F}}_{j-1}\right)|& =& {\displaystyle \frac{\sqrt{n\psi \left({\lambda }_K\right)}}{n\mathrm{I}\mathrm{E}({\Gamma }_1\mathcal{K}e{r}_1}\left|\mathrm{I}\mathrm{E}\left(B_j({\mathcal{J}}_j-CDF(y\mid x)){\Gamma }_j\mathcal{K}e{r}_j\right|{\mathfrak{F}}_{j-1})\right|}\hfill \\ & \le & {\displaystyle C\sqrt{n\psi \left({\lambda }_K\right)}({\lambda }_K^{b_1}+{\lambda }_J^{b_2})\frac{1}{n\psi \left({\lambda }_K\right)}(P\left(x\right)+o\left(1\right)).}\hfill \end{array}$$

Thus, we have

$${\displaystyle {\Sigma }_{j=1}^n{\left(\mathrm{I}\mathrm{E}\left({\eta }_{n,j}\right|{\mathfrak{F}}_{j-1})\right)}^2=O_{a.co.}\left(n\psi \left({\lambda }_K\right){({\lambda }_K^{b_1}+{\lambda }_J^{b_2})}^2\right).}$$

For Equation (27), we have

$$\begin{array}{l}{\displaystyle {\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2\right|{\mathfrak{F}}_{j-1})}\\ = {\displaystyle \frac{\psi \left({\lambda }_K\right)}{n{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^2}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j^2\mathcal{K}e{r}_j^2{B}_j{({\mathcal{J}}_j-CDF(y\mid x))}^2\right|{\mathfrak{F}}_{j-1})}\\ = {\displaystyle \frac{\psi \left({\lambda }_K\right)}{n{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^2}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j^2\mathcal{K}e{r}_j^2{B}_j\mathrm{I}\mathrm{E}\left[{({\mathcal{J}}_j-CDF(y\mid x))}^2\right|X_j]\right|{\mathfrak{F}}_{j-1}).}\end{array}$$

Since

$$\begin{array}{ccc}\hfill \mathrm{I}\mathrm{E}\left[{({\mathcal{J}}_j-CDF(y\mid x))}^2\right|X_j]& =& \mathrm{Var}\left[{\mathcal{J}}_j\right|X_j]+{[\mathrm{I}\mathrm{E}\left({\mathcal{J}}_j\right|X_j)-CDF(y\mid x)]}^2,\hfill \end{array}$$

we have

$$\begin{array}{c}\hfill \mathrm{Var}\left[{\mathcal{J}}_j\right|X_j]=\mathrm{I}\mathrm{E}\left({\mathcal{J}}_j^2\right|X_j)-{\left(\mathrm{I}\mathrm{E}\left({\mathcal{J}}_j\right|X_j)\right)}^2.\end{array}$$

(28)

We highlight that

$$\begin{array}{ccc}\hfill \mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j\right|X_j]& =& {\displaystyle {\int }_{\mathrm{I}\mathrm{R}}{\mathcal{J}}^{\left(1\right)}\left(t\right)[CDF(y-{\lambda }_{J}t|x)-CDF(y\mid x)]dt+CDF(y\mid x)}\hfill \\ & =& CDF(y\mid x),\hfill \end{array}$$

(29)

and

$$\begin{array}{ccc}\hfill \mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j^2\right|X_j]& =& {\displaystyle {\int }_{\mathrm{I}\mathrm{R}}{\mathcal{J}}^2\left({\displaystyle \frac{y-z}{{\lambda }_J}}\right)cdf\left(z\right|x)dz}\hfill \\ & =& {\displaystyle {\int }_{\mathrm{I}\mathrm{R}}2J\left(t\right){\mathcal{J}}^{\left(1\right)}\left(t\right)[CDF(y-{\lambda }_{J}t|x)-CDF(y\mid x)]dt}\hfill \\ & & {\displaystyle +{\int }_{\mathrm{I}\mathrm{R}}2{\mathcal{J}}^{\left(1\right)}\left(t\right)\mathcal{J}\left(t\right)CDF(y\mid x)dt.}\hfill \end{array}$$

Using the fact

$${\displaystyle {\int }_{\mathrm{I}\mathrm{R}}{\mathcal{J}}^{\left(1\right)}\left(t\right)\mathcal{J}\left(t\right)CDF(y\mid x)dt=CDF(y\mid x),}$$

we readily obtain, as $n\to \infty ,$

$$\begin{array}{c}\hfill \mathrm{I}\mathrm{E}\left[{\mathcal{J}}_j^2\right|X_j]⟶CDF(y\mid x).\end{array}$$

(30)

Combining (29) with (30), we arrive at

$$\begin{array}{c}\hfill \mathrm{Var}\left[{\mathcal{J}}_j\right|X_j]=CDF(y\mid x)(1-CDF(y\mid x)).\end{array}$$

(31)

Concerning ${\beta }_{n_2},$ we obtain from (29) that, as $n\to \infty$,

$$\begin{array}{c}\hfill {\beta }_{n_2}⟶0.\end{array}$$

Thereafter, we obtain

$$\begin{array}{ccc}\hfill {\displaystyle {\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2\right|{\mathfrak{F}}_{j-1})}& =& {\displaystyle \frac{\psi \left({\lambda }_K\right)}{n{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^2}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j^2{K}_j^2{B}_j{\beta }_{n_1}\right|{\mathfrak{F}}_{j-1})}\hfill \\ & =& {\displaystyle \frac{\psi \left({\lambda }_K\right)}{n{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^2}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\Gamma }_j^2{K}_j^2\mathrm{I}\mathrm{E}\left(B_j\right|X_j){\beta }_{n_1}\right|{\mathfrak{F}}_{j-1})}\hfill \\ & =& {\displaystyle \frac{\psi \left({\lambda }_K\right)}{n{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^2}{\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left((P\left(x\right)+o\left(1\right)){\Gamma }_j^2{K}_j^2{\beta }_{n_1}\right|{\mathfrak{F}}_{j-1}).}\hfill \end{array}$$

The use of Equation (31) allow us to obtain, as $n\to \infty$,

$$\begin{array}{ccc}& & {\displaystyle {\Sigma }_{j=1}^n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2\right|{\mathfrak{F}}_{j-1})⟶\frac{M_2}{M_1^2}P\left(x\right)(1-CDF(y\mid x))CDF(y\mid x)}\hfill \\ & & =V_{JK}(y\mid x);\hfill \end{array}$$

this last part completes the proof of our first claim. Concerning (25), we have

$$n\mathrm{I}\mathrm{E}\left({\xi }_{n,j}^2\right)\le 4n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2{\mathbb{I}}_{\left[\right|{\eta }_{n,j}|>\frac{\epsilon }{2}]}\right).$$

By Markov–Hölder’s inequalities, we obtain $\epsilon >0$:

$${\displaystyle \mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2{\mathbb{I}}_{\left[\right|{\eta }_{n,j}|>\frac{\epsilon }{2}]}\right)\le \frac{\mathrm{I}\mathrm{E}\left(\right|{\eta }_{n,j}{\left|\right)}^{2a}}{{(\epsilon /2)}^{2a/b}}.}$$

Putting $a=1+\frac{{\delta }^{\prime }}{2}$ for ${\delta }^{\prime }>0$ and by recalling that the function

$$G_{2+{\delta }^{\prime }}=\mathrm{I}\mathrm{E}\left(\right|{\mathcal{J}}_j{-CDF(y\mid x)|}^{2+{\delta }^{\prime }}\left|X_j\right)$$

is continuous, we have

$$\begin{array}{ccc}\hfill 4n\mathrm{I}\mathrm{E}\left({\eta }_{n,j}^2{\mathbb{I}}_{\left[\right|{\eta }_{n,j}|>\frac{\epsilon }{2}]}\right)& \le & {\displaystyle C{\left({\displaystyle \frac{\psi \left({\lambda }_K\right)}{n}}\right)}^{\frac{2+{\delta }^{\prime }}{2}}\frac{n}{{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^{2+{\delta }^{\prime }}}}\hfill \\ & & \times \mathrm{I}\mathrm{E}\left(\right[|({\mathcal{J}}_j-CDF(y\mid x))|{\Gamma }_j\mathcal{K}e{r}_j{B}_j{]}^{2+{\delta }^{\prime }})\hfill \\ & \le & {\displaystyle C{\left({\displaystyle \frac{\psi \left({\lambda }_K\right)}{n}}\right)}^{\frac{2+{\delta }^{\prime }}{2}}\frac{n}{{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^{2+{\delta }^{\prime }}}}\hfill \\ & & \times \mathrm{I}\mathrm{E}\left(P\left(x\right)\right|{\Gamma }_j\mathcal{K}e{r}_j{|}^{2+{\delta }^{\prime }}[\mathrm{I}\mathrm{E}\left(\right|{\mathcal{J}}_j-CDF(y\mid x)){|}^{2+{\delta }^{\prime }}|X_j\left]\right)\hfill \\ & \le & {\displaystyle C(P\left(x\right)+o\left(1\right)){\left({\displaystyle \frac{\psi \left({\lambda }_K\right)}{n}}\right)}^{\frac{2+{\delta }^{\prime }}{2}}\frac{n}{{\left(\mathrm{I}\mathrm{E}\left({\Gamma }_1\mathcal{K}e{r}_1\right)\right)}^{2+{\delta }^{\prime }}}}\hfill \\ & & \times \mathrm{I}\mathrm{E}\left(\right|{\Gamma }_j\mathcal{K}e{r}_j{|}^{2+{\delta }^{\prime }})G_{2+{\delta }^{\prime }}\hfill \\ & =& O\left({\left(n\psi \left({\lambda }_K\right)\right)}^{\frac{-{\delta }^{\prime }}{2}}\right)⟶0,\mathrm{as}n\to \infty .\hfill \end{array}$$

Hence, the proof is complete. □