A Model-Free Online Learning Control for Attitude Tracking of Quadrotors

Abstract

This paper investigates the problem of attitude tracking in quadrotor unmanned aerial vehicles (UAVs) using a model-free online learning control (MFOLC) scheme. The attitude system, which is represented by unit quaternions, is considered in the presence of uncertain and unknown inertia parameters, time-varying external disturbances, and angular velocity measurement noise. A computationally low-cost control scheme consisting of a model-free baseline controller and a module capable of learning from previous control input is designed. The proposed controller does not require precise inertial parameters and does not involve feedforward terms that use these parameters and true system states. This ensures that the approach can protect the control effort from sensor noise as well as parameter uncertainty. We also show that all the signals in the closed-loop system are uniformly ultimately bounded. Comparative simulations and real-world experiments are conducted for validation, which demonstrate the effectiveness and fine performance of the proposed scheme.

1. Introduction

Multirotor UAVs play an important role in several civil and military fields because of their mechanical simplicity, vertical takeoff and landing capability, and natural stability [1]. The quadrotor is a typical multirotor UAV that is widely used in aerial photography, military reconnaissance, emergency communication, agriculture, surveying and mapping, etc. [2,3,4] Attitude tracking control is fundamental for a quadrotor to complete various tasks, yet it is also a challenging issue. In practical applications, the attitude controller must achieve control accurately and quickly under internal and external uncertainties, such as external disturbances caused by turbulence, inaccurate or even unknown model parameters, and angular velocity measurement noise. Furthermore, due to the limitation of on-board processor performance and the fact that many tasks require running complex programs online (e.g., simultaneous localization and mapping), the attitude control algorithm must utilize inexpensive online computations.

Various efforts have been made in designing quadrotor attitude controllers. Initially, linear methods such as proportional-integral-derivative (PID) and linear quadratic regulator (LQR) methods were widely used for quadrotor attitude control [5,6,7,8]. To improve robustness under unknown external disturbances, disturbance estimation-based approaches have been studied. Controllers employing these approaches consist of a baseline controller and an external disturbance estimation component, which compensate for disturbance. For instance, Chen et al. [9] proposed a disturbance observer to estimate the unknown disturbance, and by using the output of the disturbance observer, a flight controller of the quadrotor was developed to track the given signals which are generated by the reference model. However, in Chen’s work the disturbance is assumed to consist of some harmonic disturbances. More generally, Wang et al. [10] designed a finite-time extended state observer to cope with external disturbances, and a nonsingular terminal sliding-mode control scheme was developed for a quadrotor. In addition, the sliding-mode technique was also used to establish observers to estimate the external disturbances within the appointed settling time [11]. To improve the accuracy of the disturbance estimation, neural networks were integrated into controllers to provide a more accurate estimation of the external disturbances [12,13]. However, neural networks require a large amount of training data, while their generalization abilities in different environments also need to be validated. The idea of disturbance estimation-based control has also been applied to the case of model parameter uncertainty. In such a case, one first designs a baseline controller using a set of nominal parameters, and then estimates and compensates for the parameter uncertainties as external disturbances. Once the output of the estimation component can approximate the true disturbances, the stability of the closed loop system can be theoretically guaranteed. However, when the sensor noise is significant, it is difficult for the estimated disturbances to converge to the true value, and thus it is hard to ensure the stability of the system. Furthermore, the disturbance estimation component requires additional computational consumption and prior information on the quadrotor to build a nominal model.

To address the issue of unknown model parameters, learning-based control has been widely utilized in recent years. This includes offline learning methods such as reinforcement learning and deep reinforcement learning. These methods learn control policies directly from flight data and thus avoid having to use a UAV model [14,15]. However, their generalization remains difficult to ensure, and their performance and stability should be further verified when the scene that generates the data for training differs greatly from the actual scene. Because of this, offline learning methods have only achieved good performance in some static scenarios, such as drone racing [16]. In contrast, online learning approaches combine a model-based controller with a model online-learning mechanism, which can cope with uncertain model parameters or unknown situations [17,18]. This is very similar to disturbance estimation-based methods, which are also essentially a data-based method, and therefore control performance may be hard to guarantee when the measurements are noisy. In addition, the online learning mechanism increases computational consumption.

To achieve accurate and robust attitude control with compact computational power, Zhang et al. [19] proposed an online learning control (OLC) algorithm. This approach achieves robust attitude control for spacecraft by adding the online learning of previous control outputs to a baseline controller. However, a feedforward term that compensates for the gyroscopic moments requires knowing the exact inertia matrix, which is difficult to accomplish in practice.

Motivated by the need for an accurate, robust, and computation-saving attitude tracking controller for quadrotors, the present paper makes several improvements to the original OLC and applies it to a quaternion-based quadrotor attitude dynamical system. The main contributions of this paper are as follows.

• A model-free online learning control (MFOLC) scheme is proposed to achieve the attitude tracking of quadrotors. The closed-loop attitude system is uniformly ultimately bounded (UUB) stable when the control torques and the external disturbances are bounded.
• In contrast to previous studies on the robust attitude tracking control problem for quadrotors, the proposed controller is computationally inexpensive, and does not require accurate model parameters of quadrotors. Both simulation and real-world experiment show that our scheme can achieve attitude tracking in the presence of parameter uncertainties, external disturbances, and noisy angular velocity measurements.

The rest of this paper is organized as follows. In Section 2, we introduce the quaternion-based mathematical model of a rigid quadrotor and a control problem statement. Section 3 presents the control law design and a stability analysis. A comparative simulation and a real-world experiment are given in Section 4. The conclusions and directions for future works are given in Section 5.

2. Model Description and Problem Statement

2.1. Notations

Let $ℝ$ denote the set of real numbers, ${ℝ}^{m\times n}$ denotes the set of $m$ by $n$ real matrices, and ${\mathit{I}}_n\in {ℝ}^{n\times n}$ denote an $n$ by $n$ identity matrix. For a matrix $\mathit{A}\in {ℝ}^{m\times n}$, ${\mathit{A}}^T$ denotes its transpose, $‖\mathit{A}‖=\sqrt{{\lambda }_{\max }\left({\mathit{A}}^T\mathit{A}\right)}$ denotes its 2-norm, where ${\lambda }_{\max }\left(\mathit{M}\right)$ denotes the largest eigenvalue of a symmetric matrix $\mathit{M}$. For any vector $\mathit{v}={\left[v_1,v_2,\dots ,v_n\right]}^T\in {ℝ}^{n\times 1}$, $‖\mathit{v}‖=\sqrt{{\mathit{v}}^T\mathit{v}}$ denotes its Euclidean norm. The operator ${\left(\mathit{v}\right)}^{\wedge }$ for a vector $\mathit{v}={\left[v_1,v_2,v_3\right]}^T\in {ℝ}^3$ denotes a skew-symmetric matrix:

$${\mathit{v}}^{\wedge }=\left[\begin{array}{ccc}0& -v_3& v_2\\ v_3& 0& -v_1\\ -v_2& v_1& 0\end{array}\right]$$

(1)

In attitude control, we denote the inertial frame as $\mathcal{I}$, and the UAV fixed body frame as $\mathcal{B}$. The attitude of a quadrotor is defined by the state of $\mathcal{B}$ relative to $\mathcal{I}$. Unit-quaternion $\mathit{q}={\left[s,{\mathit{v}}^T\right]}^T\in {ℝ}^{4\times 1}$ is used to present the attitude of quadrotors, where $s$ is the scalar part and $\mathit{v}\in {ℝ}^{3\times 1}$ is the vector part; moreover, $‖\mathit{q}‖=1$. Furthermore, $\mathit{R}\left(\mathit{q}\right)=\left({\mathit{I}}_3+2s{\mathit{v}}^{\wedge }+2{\mathit{v}}^{\wedge }{\mathit{v}}^{\wedge }\right)\in {ℝ}^{3\times 3}$ denotes the rotation matrix from $\mathcal{B}$ to $\mathcal{I}$.

2.2. Attitude Model of a Quadrotor

The attitude of the quadrotors is modeled as an airborne rigid body in terms of the unit-quaternion, $\mathit{q}$. Thus, the attitude dynamics are given by the following [20,21]:

$$\dot{\mathit{q}}\left(t\right)=\left[\begin{array}{c}\dot{s}\left(t\right)\\ \dot{\mathit{v}}\left(t\right)\end{array}\right]=\frac{1}{2}\left[\begin{array}{c}-\mathit{v}{\left(t\right)}^T\\ \mathit{v}{\left(t\right)}^{\wedge }+s\left(t\right){\mathit{I}}_3\end{array}\right]\mathit{\omega }\left(t\right)$$

(2)

$$\mathit{J}\dot{\mathit{\omega }}\left(t\right)=-\mathit{\omega }{\left(t\right)}^{\wedge }\mathit{J}\mathit{\omega }\left(t\right)+\mathit{u}\left(t\right)+\mathit{d}\left(t\right)$$

(3)

where $\mathit{\omega }\left(t\right)\in {ℝ}^{3\times 1}$ is the angular velocity of the quadrotor expressed in the body frame; $\mathit{J}\in {ℝ}^{3\times 3}$ denotes the inertia matrix, which is a positive definite constant matrix; $\mathit{u}\left(t\right)\in {ℝ}^3$ is the control torques provided by four rotors; and $\mathit{d}\left(t\right)\in {ℝ}^3$ denotes the time-varying external disturbance torques acting on the vehicle. For the sake of brevity, time stamps will be omitted below when they do not interfere with comprehension, e.g., writing $\mathit{\omega }\left(t\right)$ as $\mathit{\omega }$.

Once given the desired attitude quaternion, ${\mathit{q}}_d={\left[s_d,{\mathit{v}}_d^T\right]}^T$, desired rotational speeds ${\mathit{\omega }}_d$ and desired rotational accelerations ${\dot{\mathit{\omega }}}_d$, the attitude tracking error quaternion ${\mathit{q}}_e={\left[s_e,{\mathit{v}}_e^T\right]}^T$ and angular velocity error ${\mathit{\omega }}_e$ can be defined as follows:

$$\left\{\begin{array}{c}{s}_e=s{s}_d+{\mathit{v}}^T{\mathit{v}}_d\\ {\mathit{v}}_e=s_d\mathit{v}-s{\mathit{v}}_d+{\mathit{v}}^{\wedge }{\mathit{v}}_d\end{array}\right.$$

(4)

$${\mathit{\omega }}_e=\mathit{\omega }-{\Omega }_d$$

(5)

where ${\Omega }_d={\left(\mathit{R}\left({\mathit{q}}_e\right)\right)}^T{\mathit{\omega }}_d$. By directly differentiating ${\mathit{q}}_e$ and ${\mathit{\omega }}_e$, the tracking error dynamics can be obtained as follows:

$${\dot{\mathit{q}}}_e=\left[\begin{array}{c}{\dot{s}}_e\\ {\dot{\mathit{v}}}_e\end{array}\right]=\frac{1}{2}\left[\begin{array}{c}-{\mathit{v}}_e^T{\mathit{\omega }}_e\\ \left(s_e{\mathit{I}}_3+{\mathit{v}}_e^{\wedge }\right){\mathit{\omega }}_e\end{array}\right]$$

(6)

$$\mathit{J}{\dot{\mathit{\omega }}}_e=-{\mathit{\omega }}^{\wedge }\mathit{J}\mathit{\omega }+\mathit{J}\left({\mathit{\omega }}_e^{\wedge }{\Omega }_d-{\dot{\Omega }}_d\right)+\mathit{u}+\mathit{d}$$

(7)

where ${\dot{\Omega }}_d={\left(\mathit{R}\left({\mathit{q}}_e\right)\right)}^T{\dot{\mathit{\omega }}}_d$.

Note that the thrust and torsional torque produced by each rotor are bounded; the desired rotational speeds and accelerations given by the user or higher-level controller are also bounded. Therefore, the following reasonable assumptions are made.

Assumption 1.

Control torque $\mathit{u}$ is bounded; i.e., there exists $\overline{u}>0$ such that $‖\mathit{u}‖\le \overline{u}$. ${\mathit{\omega }}_d$ and ${\dot{\mathit{\omega }}}_d$ are bounded, and thus ${\mathit{\omega }}_e$ and ${\dot{\mathit{\omega }}}_e$ are bounded; i.e., there exist ${\overline{\omega }}_d, {\dot{\overline{\omega }}}_d, {\overline{\omega }}_e, {\dot{\overline{\omega }}}_e>0$, such that ${‖\mathit{\omega }‖}_d\le {\overline{\omega }}_d, ‖{\dot{\mathit{\omega }}}_d‖\le {\dot{\overline{\omega }}}_d, ‖{\mathit{\omega }}_e‖\le {\overline{\omega }}_e, and ‖{\dot{\mathit{\omega }}}_e‖\le {\dot{\overline{\omega }}}_e$.

Assumption 2.

External disturbance torque $\mathit{d}$ is bounded; i.e., there exists $\overline{d}>0$ such that $‖\mathit{d}‖\le \overline{d}$.

In practice, the inertia matrix $\mathit{J}$ is unknown, but it can be estimated by weighting the individual components of the quadrotor and building a physical model [22]. Let $m$ denote the total mass of the quadrotor, and $l$ denote the approximate distance between the motor and the center of mass, both of which can be practically measured. Similar to the literature [23], we assume that the mass of each motor is $\frac{m}{4}$ and treat the four motors as point masses, then the inertia matrix of this simplified physical model is $\mathrm{diag}\left(\frac{m{l}^2}{2},\frac{m{l}^2}{2},m{l}^2\right)$. This simplified physical model has a larger moment of inertia along each axis than a quadrotor with the same size. Thus, we make the following assumption.

Assumption 3.

Inertia matrix $J$ is unknown, but the upper bound of its 2-norm can be estimated by the mass and size of the quadrotor; i.e., we can select $\overline{J}=m{l}^2>0$ such that $‖\mathit{J}‖\le \overline{J}$.

2.3. Problem Statement

The control objective can be stated as follows. Consider the rigid quadrotor attitude system given by (2) and (3), design a control law $\mathit{u}$ such that:

• the closed-loop attitude error system given by (6) and (7) is globally stable under Assumptions 1–3.
• the attitude and angular velocity error converge to a small region.

3. Model-Free Online Learning Control Design

3.1. Control Law Design

Considering the control objective, we define the attitude synthesis error $\mathit{ϵ}=k_p{\mathit{v}}_e+{\mathit{\omega }}_e$, where $k_p>0$. It is clear that, as the attitude and angular velocity error converge to a small region, $\mathit{ϵ}$ also converges to a small neighborhood of the origin. Then, the model-free online learning control (MFOLC) law is proposed as follows:

$$\mathit{u}\left(t\right)=\mathit{\eta }+\mathit{L}\left(\mathit{t}\right)$$

(8)

$$\mathit{\eta }=-\mathit{K}\mathit{ϵ}$$

(9)

$$\mathit{L}\left(t\right)={\mathit{K}}_L\mathit{u}\left(t-\tau \right)$$

(10)

where $\mathit{\eta }$ is the baseline controller, $\mathit{K}=\mathrm{diag}\left(k_1,k_2,k_3\right)$, and $k_i\ge 0,i=1,2,3$. $\mathit{L}\left(t\right)$ is the online learning term, where ${\mathit{K}}_L=\mathrm{diag}\left(k_{l,1},k_{l,2},k_{l,3}\right)$ is the learning intensity matrix, and $k_{l,i}\ge 0,i=1,2,3$. $\mathit{u}\left(t-\tau \right)$ is the control input in time $t-\tau$, and $\tau \ge 0$ is called a learning interval.

Remark 1.

Since the ordinary OLC law is

$$\mathit{u}\left(t\right)=k_1\mathit{u}\left(t-\tau \right)-k_2\left[\mathit{ϵ}-\frac{1}{{\kappa }_1}\left(-{\mathit{\omega }}^{\wedge }\mathit{J}\mathit{\omega }+\frac{k_p}{2}\mathit{J}\left(s_e{\mathit{I}}_3+{\mathit{v}}_e^{\wedge }\right){\mathit{\omega }}_e\right)\right]$$

(11)

where $k_1\ge 0$ is the learning intensity, $k_2\ge 0$ is the control gain,  ${\kappa }_1>0$ is the feedforward gain. There are two main differences between the proposed MFOLC and the ordinary OLC. (1) The baseline controller of the MFOLC discards the gyroscopic torque compensation term, which enables the controller to be independent of the inertia matrix of quadrotors and to weaken the control saturates. (2) The control parameters of each attitude axis can be adjusted independently, which enhances the control effectiveness for quadrotors with an asymmetric structure.

3.2. Stability Analysis

Substituting (8)–(10) into (7) and combining the result with (6) yields the dynamical equation of attitude synthesis error:

$$\mathit{J}\dot{\mathit{ϵ}}=-{\mathit{ϵ}}^{\wedge }\mathit{J}\mathit{\omega }+\mathit{S}{\mathit{\omega }}_e+\mathit{\beta }-\mathit{K}\mathit{ϵ}$$

(12)

where

$$\mathit{S}=\frac{1}{2}{k}_p\mathit{J}\left(s_e{\mathit{I}}_3+{\mathit{v}}_e^{\wedge }\right)+k_p{\mathit{v}}_e^{\wedge }\mathit{J}+{\left(\mathit{J}{\Omega }_d\right)}^{\wedge }-{\Omega }_d^{\wedge }\mathit{J}-\mathit{J}{\Omega }_d^{\wedge }$$

(13)

$$\mathit{\beta }=k_p{\mathit{v}}_e^{\wedge }\mathit{J}{\Omega }_d-{\Omega }_d^{\wedge }\mathit{J}{\Omega }_d-\mathit{J}{\dot{\Omega }}_d+\mathit{d}+\mathit{L}\left(t\right)$$

(14)

The following lemma is given as a preparation for the proof of the MFOLC’s stability.

Lemma 1.

Under Assumptions 1–3, the following inequalities hold:

$$‖\mathit{S}{\mathit{\omega }}_e‖\le \sigma ‖\mathit{ϵ}‖$$

(15)

$$‖\mathit{\beta }‖\le \rho$$

(16)

where

$$\sigma =\overline{J}(1.5k_p+3{\overline{\omega }}_d)$$

(17)

$$\rho =\overline{J}\left({\overline{\omega }}_d^2+k_p{\overline{\omega }}_d+{\dot{\overline{\omega }}}_d\right)+\overline{d}+{\overline{k}}_l\overline{u}$$

(18)

Proof of Lemma 1.

Note that $‖{\mathit{a}}^{\wedge }‖=‖\mathit{a}‖$ for $\mathit{a}\in {ℝ}^{3\times 1}$, $‖s_e{\mathit{I}}_3+{\mathit{v}}_e^{\wedge }‖=1$, and $‖\mathit{R}\left({\mathit{q}}_e\right)‖=1$. According to the Minkowski inequality and Assumptions 1 and 3, we have

$$‖\mathit{S}‖\le ‖\mathit{J}‖(\frac{1}{2}{k}_p‖s_e{\mathit{I}}_3+{\mathit{v}}_e^{\wedge }‖+k_p‖{\mathit{v}}_e‖+3‖{\Omega }_d‖) \le \overline{J}(1.5k_p+3{\overline{\omega }}_d)=\sigma$$

(19)

It is clear that $\sigma \ge 0$. Recall that since $‖\mathit{ϵ}‖\le k_p+‖{\mathit{\omega }}_e‖$ and $k_p>0$, $‖{\mathit{\omega }}_e‖<‖\mathit{ϵ}‖$. Therefore,

$$‖\mathit{S}{\mathit{\omega }}_e‖\le ‖\mathit{S}‖‖{\mathit{\omega }}_e‖\le \sigma ‖\mathit{ϵ}‖$$

(20)

Then Equation (15) holds. Similarly, according to the Minkowski inequality and Assumption 1,

$$‖\mathit{L}\mathit{u}\left(t-\tau \right)‖\le ‖\mathit{L}‖‖\mathit{u}\left(t-\tau \right)‖\le {\overline{k}}_l\overline{u}$$

(21)

holds, where ${\overline{k}}_l=\max \left(k_{l,1},k_{l,2},k_{l,3}\right)$. Upon combining Assumptions 2 and 3, we can conclude that

$$‖\mathit{\beta }‖\le \overline{J}\left({\overline{\omega }}_d^2+k_p{\overline{\omega }}_d+{\dot{\overline{\omega }}}_d\right)+\overline{d}+{\overline{k}}_l\overline{u}=\rho$$

(22)

As such, $\rho >0$. Therefore, this completes the proof. □

The stability analysis of the closed-loop attitude system can be stated in the following Theorem.

Theorem 1.

Consider the quadrotor attitude system governed by (2) and (3) under Assumptions 1–3. With the application of the control law (8)–(10), we suppose that the control parameters are chosen so that

$$\underset{\_}{k}>\overline{J}(1.5k_p+3{\overline{\omega }}_d)+1$$

(23)

where $\underset{\_}{k}=min\left(k_1,k_2,k_3\right)$, holds. Then, the attitude synthesis error $\mathit{ϵ}$ is UUB.

Proof of Theorem 1.

Consider the following Lyapunov function candidate:

$$V=\frac{1}{2}{\mathit{ϵ}}^T\mathit{J}\mathit{ϵ}$$

(24)

Obviously, $V\ge 0$ and $V=0$ if and only if $\mathit{ϵ}={\left[0,0,0\right]}^T$. The derivative of $V$ is

$$\dot{V}={\mathit{ϵ}}^T\mathit{J}\dot{\mathit{ϵ}}$$

(25)

Substituting (12) into (25) yields

$$\dot{V}=-{\mathit{ϵ}}^T\mathit{K}\mathit{ϵ}-{\mathit{ϵ}}^T{\mathit{ϵ}}^{\wedge }\mathit{J}\mathit{\omega }+{\mathit{ϵ}}^T\mathit{S}{\mathit{\omega }}_{\mathit{e}}+{\mathit{ϵ}}^T\mathit{\beta }$$

(26)

Note that ${\mathit{ϵ}}^T{\mathit{ϵ}}^{\wedge }\mathit{J}\mathit{\omega }=0$, and according to Lemma 1, the following inequality holds:

$$\dot{V}\le -\left(\underset{\_}{k}-\sigma \right)‖{\mathit{ϵ}}^2‖+\rho ‖\mathit{ϵ}‖$$

(27)

Since $‖\mathit{ϵ}‖\ge 0$ and $\rho >0$, we have

$$\rho ‖\mathit{ϵ}‖\le {‖\mathit{ϵ}‖}^2+\frac{{\rho }^2}{4}$$

(28)

Substituting (28) into (27) yields

$$\dot{V}\le -\pi {‖\mathit{ϵ}‖}^2+\delta$$

(29)

where $\pi =\underset{\_}{k}-\sigma -1$, and $\delta =\frac{{\rho }^2}{4}$.

It is seen from (25) that $\dot{V}<0$ when $\mathit{ϵ}$ are outside of set $\mathcal{D}=\left\{\mathit{ϵ}|‖\mathit{ϵ}‖\le \frac{\rho }{2\sqrt{\underset{\_}{k}-\sigma -1}}\right\}$. This implies that $V\left(t\right)$ decreases monotonically outside set $\mathcal{D}$. Hence, the attitude synthesis error in the closed-loop system is bounded. Moreover, we can choose a sufficiently small ${\epsilon }_0>0$, let ${\epsilon }^{\prime }={\epsilon }_0+\frac{\rho }{2\sqrt{\underset{\_}{k}-\sigma -1}}$, and define set ${\mathcal{D}}^{\prime }=\left\{\mathit{ϵ}|‖\mathit{ϵ}‖\le {\epsilon }^{\prime }\right\}$ to guarantee that

$$\underset{t\to \infty }{\lim }‖\mathit{ϵ}\left(t\right)‖\in {\mathcal{D}}^{\prime }$$

(30)

It can be concluded from (30) that there exists a $T\left({\epsilon }^{\prime }\right)>0$, such that $‖\mathit{ϵ}\left(t\right)‖\le {\epsilon }^{\prime }$ for $t\ge T\left({\epsilon }^{\prime }\right)$. This shows that $\mathit{ϵ}$ is UUB from its definition [24].

According to Theorem 2 in [25], when $ϵ$ converges to a small region ${\epsilon }^{\prime }$, $\left|{\omega }_{e,i}\left(t\right)\right|\le 2{\epsilon }^{\prime }$, $\left|q_{e,i}\left(t\right)\right|\le {\epsilon }^{\prime }/k_p$, $i=1,2,3$.

This completes the proof. □

Remark 2.

The proposed MFOLC is more intuitive than the ordinary OLC in control parameter tuning. The following steps can be used for parameter tuning in the MFOLC. Firstly, one can set an appropriate ${\overline{\omega }}_d$ according to the quadrotor’s task and estimate $\overline{J}$ through the weight and size of the quadrotor. Secondly, one can find the appropriate $k_p$ according to the system response and then calculate the minimum values of $k_1,$ $k_2,$ and $k_3$ using Equation (22). Finally, $k_p,$ $k_1,$ $k_2,$ $k_3,$ $k_{l,1},$ $k_{l,2},$ and $k_{l,3}$ should be fine-tuned according to the attitude responses of the quadrotor.

4. Validation

4.1. Comparative Simulation

The efficacy of the proposed method is illustrated with numerical simulations. The parameters of the quadrotor are chosen according to a quadrotor developed in [23]. The key parameters for the quadrotor used for simulation are as follows:

• Total mass of the quadrotor: $m=4.14$ kg;
• The distance from the center of mass to each motor: $l=0.315$ m;
• The inertia matrix:

$${\mathit{J}}_0=\left[\begin{array}{ccc}0.082& 0& 0\\ 0& 0.0845& 0\\ 0& 0& 0.1377\end{array}\right]\left(\mathrm{kg}\cdot {\mathrm{m}}^2\right)$$

(31)

To test the robustness of the MFOLC algorithm, the uncertainty of inertia and external disturbances are also considered in the simulation.

$$\Delta \mathit{J}\left(t\right)=\mathrm{diag}\left(\begin{array}{c}5\cos 0.5t-1\sin 0.5t-3\\ 3\cos 0.5t+2\sin 0.5t-4\\ 4\cos 0.5t-1.5\sin 0.5t+5\end{array}\right)e^{-0.1t}\times {10}^{-2}\left(\mathrm{kg}\cdot {\mathrm{m}}^2\right)$$

(32)

$$\mathit{d}\left(t\right)=\left[\begin{array}{c}2\sin \varphi t+5\cos 0.15t+3\\ -3\cos \varphi t-4\sin 0.7t-4\\ 8\cos \varphi t-4\sin 0.5t-1\end{array}\right]\times {10}^{-3}\left(\mathrm{N}\cdot \mathrm{m}\right)$$

(33)

where $\varphi =0.5+‖\mathit{\omega }‖$. The external disturbance is shown in Figure 1.

The proposed MFOLC and the ordinary OLC developed in [19] are compared. The desired attitude quaternion is ${\mathit{q}}_d={\left[0.9353,0.2273,0.2708,0.01459\right]}^T$, which is equivalent to a $30°$ roll angle, a $30°$ pitch angle, and a $10°$ yaw angle. The initial state of the quadrotor is $\mathit{q}\left(0\right)={\left[1,0,0,0\right]}^T$, $\mathit{\omega }\left(0\right)={\left[0,0,0\right]}^T$, and ${\mathit{\omega }}_d\left(0\right)={\dot{\mathit{\omega }}}_d\left(0\right)={\left[0,0,0\right]}^T$. The simulation starts from $t=0$ and $\mathit{u}\left(t\le 0\right)={\left[0,0,0\right]}^T$. The control parameters are listed in

Table 1. Control parameters chosen for simulation.

Controller	MFOLC	OLC
Parameters	$k_p=1$	$k_p=2.15$
	${\mathit{K}}_L=\mathrm{diag}\left(0.8,0.85,0.8\right)$	$k_1=0.8$
	$\mathit{K}=\mathrm{diag}\left(1.625,1.7,1.725\right)$	$k_2=0.7$
	$\tau =0.01$	$\tau =0.01$
		${\kappa }_1=3.5$

.

Note that according to Assumption 3, $\overline{J}$ can be selected by $\overline{J}=m{l}^2=0.4108$ $\mathrm{kg}\cdot {\mathrm{m}}^2$, such that $‖{\mathit{J}}_0‖=0.1377<\overline{J}$. According to Table 1, $\underset{\_}{k}>\overline{J}(1.5k_p+3{\overline{\omega }}_d)+1=1.6162$ is also satisfied. In addition, to maintain hover, attitude control torques must be limited, with the maximum roll, pitch, and yaw control torque of 1 Nm, 1 Nm and 0.1 Nm, respectively.

4.1.1. Scenario 1: Without Measurement Noise

In this scenario, a relative ideal situation is simulated in which only external disturbances are acting on the quadrotor. Figure 2a–c show the simulation results for the attitude angle, the angular velocity, and the control input torque, respectively. The attitude transient-response specifications are listed in

Table 2. Attitude transient-response specifications without measurement noise.

Specifications		MFOLC	OLC	Units
Maximum overshoot	Roll	$0.11$	$0$	%
	Pitch	$0.19$	$0$
	Yaw	$0.03$	$0$
Settling time	Roll	$3.27$	$4.12$	s
	Pitch	$3.125$	$3.81$
	Yaw	$3.465$	$4.375$

. To compare steady-state performance, the root mean square error (RMSE) of the Euler angles are calculated for both controllers.

Table 3. Steady state RMSE without measurement noise.

	MFOLC	OLC	Units
roll angle error	$2.3508$	$40.1740$	${10}^{-12} \mathrm{degree}$
pitch angle error	$2.2847$	$9.0077$
yaw angle error	$4.5797$	$23.1220$
${\omega }_{e,1}$	$0.4892$	$35.8330$	${10}^{-12} \mathrm{degree}/s$
${\omega }_{e,2}$	$0.1774$	$22.4560$
${\omega }_{e,3}$	$5.5551$	$15.3720$

presents the steady state RMSEs of the MFOLC and OLC from 10 to 100 s.

In Figure 2a,b, both the MFOLC and OLC can achieve a control objective within 5 s. In Table 2, the proposed MFOLC has a shorter settling time and a slight overshoot, and a smoother angular velocity trajectory. It is worth noting that the MFOLC achieves such fine performance without priori information, compared to the OLC which uses the exact inertia matrix. This is because the MFOLC uses a more flexible and intuitive parameter system.

In Table 3, the attitude steady-state error of the MFOLC is improved by 87.26% compared to that of the OLC. This is because the OLC algorithm contains the feedforward term that depends on the inertia matrix. Additionally, in the simulation, the inertia matrix has some unknown uncertainty, which affects the steady-state performance of the OLC algorithm.

In Figure 2c, the MFOLC also requests a smaller control torque than the OLC, and the duration of the control torque to reach the limit is significantly less than that of the OLC. This means a lower energy consumption for the quadrotor.

4.1.2. Scenario 2: With Angular Velocity Measurement Noise

In this scenario, not only external disturbances but also angular velocity measurement noise are acting on the quadrotor. For the consideration of engineering applications, the noisy angular velocity measurement ${\mathit{\omega }}_m$ is modeled as follows:

$${\mathit{\omega }}_m\left(t\right)=\mathit{\omega }\left(t\right)+\mathit{w}\left(t\right)$$

(34)

where $\mathit{w}\left(t\right)\in {ℝ}^3$ is the angular velocity sensor noise, modeled as zero-mean Gaussian random variables, with variance matrix ${\Sigma }_w=\mathrm{diag}\left(0.0025,0.0025,0.001\right)$.

Figure 3a–c show the simulation results for attitude angle, angular velocity, and control input torque, respectively. The attitude transient-response specifications and steady state RMSEs from 10 to 100 s are listed in

Table 4. Attitude transient-response specifications with angular velocity measurement noise.

Specifications		MFOLC	OLC	Units
Maximum overshoot	Roll	$1.88$	$2.0$	%
	Pitch	$0.93$	$0.98$
	Yaw	$3.59$	$3.69$
Settling time	Roll	$3.2$	$4.145$	s
	Pitch	$3.07$	$3.79$
	Yaw	$3.08$	$3.745$

and

Table 5. Steady state RMSE with angular velocity measurement noise.

	MFOLC	OLC	Units
roll angle error	$0.1624$	$0.1706$	$\mathrm{degree}$
pitch angle error	$0.0821$	$0.0842$
yaw angle error	$0.1037$	$0.1049$
${\omega }_{e,1}$	$2.2417$	$2.2601$	$\mathrm{degree}/\mathrm{s}$
${\omega }_{e,2}$	$2.2424$	$2.2601$
${\omega }_{e,3}$	$0.7159$	$0.7243$

, respectively.

Comparing the simulation results in Section 4.1.1, the addition of angular velocity measurement noise leads to a significant increase in the overshoot and steady state RMSE for both controllers. However, the reduced performances are still in a fine performance range.

Overall, the comparative simulation shows that, both the MFOLC and OLC can realize attitude tracking control of the quadrotor in the presence of angular velocity measurement noise, model uncertainty and external disturbance. However, the MFOLC has better performance in terms of settling time, steady state control accuracy, and energy consumption.

4.2. Real-World Experiment

For a real-world experiment, we validate the proposed MFOLC in an outdoor environment. The quadrotor we used in the experiment is shown in Figure 4a. It is a small quadrotor designed by our team. Its power system consists of four sets of Emax RS2205 motors, kangkong 5045 three-blade propellers, and an XRotor 30A electronic speed controller (ESC). The flight controller is also self-developed, based on an STM32F407 microprocessor, as shown in Figure 4b. The onboard angular velocity sensors are three ADXRS646 micro-electro-mechanical system (MEMS) gyroscopes. The quadrotor is powered by a 3-cell LiPo battery and has a total mass of 0.9 kg. Figure 4c presents the outdoor flight picture.

The attitude control loop ran at 200 Hz, and the control parameters were as follows: $k_p=2$, ${\mathit{K}}_L=\mathrm{diag}\left(0.8,0.8,0.8\right), \mathit{K}=\mathrm{diag}\left(1.225,1.325,1.35\right), \mathrm{and} \tau =0.01$. By using an anemometer, the wind speed at the flight field was 3.7 m/s from the east. The quadrotor performed a route flight and the proposed MFOLC tracked the target attitude provided by the position controller, and the target angular velocity and acceleration were always set to zero.

The corresponding experimental results are illustrated in Figure 5. Overall, the MFOLC exhibits a good attitude command tracking performance, while the yaw angle tracking error is relatively large. This is because the yawing torque is produced by the weak reactive torque form each rotor. To maintain hovering and other axial attitude stabilizations, the yawing torque produced by speed difference of rotors is much smaller than rolling and pitching torques, resulting in relatively large attitude tracking errors.

5. Conclusions

Although many nonlinear controllers can be used for quadrotor attitude tracking, none of them have addressed external disturbances, uncertain or even unknown model parameters, and low computational consumption simultaneously. In this paper, we developed a novel model-free online learning control scheme to achieve attitude tracking in the presence of time-varying external disturbances, uncertainties in the inertia matrix, and angular velocity sensor noises. The proposed approach guaranteed the closed-loop attitude error system to be uniformly ultimately bounded stable. An accurate inertia matrix and expensive computational consumption were not needed to implement the control law. Thus, the proposed MFOLC is quite suited for small quadrotors with compact arithmetic, which is illustrated by a real-world experiment. A remaining problem of our method is that it is not specifically designed for the input saturation limit case, especially when the quadrotor needs to maintain enough vertical thrust to resist gravity when performing attitude control tasks. Moreover, the control margin that can be allocated to attitude control is limited. Hence, in future work, we intend to further investigate the MFOLC algorithm for the input saturation limit case.