Feedforward Enhancement through Iterative Learning Control for Robotic Manipulator

This work presents an iterative learning control (ILC) algorithm to enhance the feedforward control (FFC) for robotic manipulators. The proposed ILC algorithm enables the cooperation between the ILC, inverse dynamics, and a PD feedback control (FBC) module. The entire control scheme is elaborated to guarantee the control accuracy of the first implementation; to improve the control performance of the manipulator progressively with successive iterations; and to compensate both repetitive and non-repetitive disturbances, as well as various uncertainties. The convergence of the proposed ILC algorithm is analysed using a well established Lyapunov-like composite energy function (CEF). A trajectory tracking test is carried out by a seven-degree-of-freedom (7-DoF) robotic manipulator to demonstrate the effectiveness and efficiency of the proposed control scheme. By implementing the ILC algorithm, the maximum tracking error and its percentage respect to the motion range are improved from 5.78° to 1.09°, and 21.09% to 3.99%, respectively, within three iterations.


I. INTRODUCTION
The involvement of internet of things (IoT) and internet of services (IoS) in the fourth industrial revolution (Industry 4.0) facilitates the development of the smart manufacturing, which has contributed to the widespread use of the robotic manipulators. At the same time, the smart manufacturing programme also puts forward new requirements and challenges on robotic manipulators, particularly in the area of precision control [1], [2]. Although the robotic manipulators provide extra flexibility and possibility of humanrobot collaboration in manufacturing process, its absolute accuracy becomes a major factor that preventing its adoption in high precision industries, e.g. aerospace manufacturing. Improving the control accuracy of the robotic manipulators is challenging due to both internal and external uncertainties in the operation scenarios such as automated inspection, composite fabrication, additive manufacturing, etc. As a result, introducing advanced control scheme into manipulator controller becomes a generic and essential need for improving the robot manipulation precision.
The conventional controller of robotic manipulators has two streams: feedforward control (FFC) and feedback control (FBC), depending on whether the current measurement signal is used. The FFC provides computed joint input from the desired robot motion in real time, and the FBC gives joint input compensation by taking advantage of both the desired and the measured motion.
In the literature, due to the ability in handling system uncertainties and external disturbances, most of the existing works focus on the design and improvement of the FBC module in the control of robotic manipulators. For instance, many advanced FBC schemes, such as model predictive control [3], adaptive fuzzy control [4], and neural network based control [5], have been adopted to improve the control precision. However, the FBC approaches encounter difficulties due to disturbances and modelling errors (e.g. the hysteresis effects [6]), which motivates us to explore the ability of the FFC and thus improve the control performance of the whole system.
In robotic manipulators, the traditional FFC usually adopts the pure inverse dynamics, in which the control performance highly depends on the accuracy of the inverse dynamics. While in practice the accurate inverse dynamics is difficult to be obtained. To overcome this inherent disadvantage in the traditional FFC, various advanced control approaches have been investigated to compensate the inaccuracy in system model and thus to improve the performance of the FFC. For example, a fuzzy adaptation algorithm is developed in [7] to compensate the uncertainties existing in dynamics. In [8], the authors present an adaptive compensation method to handle the dynamic friction of the system. However, it should be noted that the above mentioned control methods generally require an intensive tuning process to improve the control performance. They also ignore the system repetition and do not possess the ability of learning from the past control experience. Inspired by these observations, several learning-based control approaches have been recently proposed, e.g. reinforcement learning [9] and neural networks [10], by considering the repetitive operation manners. These approaches have shown their superiority in handling the system uncertainties for robotics. While the main disadvantages of these methods lie in the time-consuming training process and the unavoidable nonsmoothness in control signals, which motivates the present work.
As an intelligent control approach, iterative learning control (ILC) has been proven to be effective to deal with highprecision control problems. By considering the system repetition, ILC is able to learn from the past control experience and thus improve the current control performance, despite the presence of various system uncertainties. From control theory point of view, ILC is actually a partial model-free control method, which may achieve a high-accuracy control performance by just utilising the system input and output information without requiring the precise system dynamics. Owing to these impressive characteristics, ILC has been intensively implemented in various practical applications in robotics [11]- [13]. Moreover, with the learning ability, ILC is capable of compensating repetitive disturbances and suppressing fast hysteresis effects [14]. Recently, targeting at the FFC of industrial manipulators, several works have been reported to show the superiority of ILC. For instance, an ILC approach combined with a H ∞ FBC is proposed in [15] to improve the control performance of nanopositioning system. In [16], the authors establish an ILC scheme in combination with low-gain FBC to improve tracking accuracy for robotic manipulators. A norm-optimal ILC scheme for articulated soft manipulators is presented in [17] to improve the tracking control performance. It is worth to note that in most of the above mentioned works, the ILC schemes are designed directly to replace the role of the pure inverse dynamics in the traditional controllers, which however did not fully take the advantage of the available system dynamics. Motivated by this observation, it is of practical importance to develop new controllers which fully utilise the available system knowledge as well as the learning ability.
In the present work, a novel learning-based controller is developed for the tracking control of robotic manipulators. It consists of three parts including a FBC module, an inverse dynamics module, as well as an ILC strategy. In the proposed control approach, the ILC algorithm is used to enhance FFC performance achieved by the inverse dynamics through compensating the inaccuracies in the system model, while the FBC is adopted to nullify the effect of non-repeatable external disturbances. In contrast to existing works, the main contributions of this work can be summarised as follows: 1) A novel learning-based controller is developed to enable the coordinating work of the inverse dynamics, the FBC and the ILC strategy. By fully utilising the system knowledge and the system repetition, the proposed controller possesses the ability to achieve high-precision tracking tasks despite the presence of various system uncertainties and external disturbances. 2) In contrast to the existing works [15]- [19] where the ILC is utilised to replace the inverse dynamics directly, the proposed ILC scheme is designed to work in parallel with the inverse dynamics to fully use the available system information. Hence, it is able to enhance the control performance of the FFC by nullifying the effect of the system uncertainties.
3) The rigorous convergence analysis of the proposed control scheme is conducted by establishing a Lyapunovlike composite energy function (CEF). It has been shown explicitly that the cooperation between the ILC and FBC is capable of reducing the peak error and improving the tracking accuracy significantly as the iteration number increases. 4) The effectiveness of the proposed control scheme is verified through both simulations and experiments. In the later one, the proposed learning-based controller is implemented on a seven-degree-of-freedom (7-DoF) robotic manipulator. In contrast to the demonstrated results in [3], where only three joints of the manipulator are used, we consider all the seven joints of the demonstrated manipulator. The rest of the paper is organised as follows. The robotic system, consisting of the kinematics and dynamics, is presented in Section II. In Section III, the control scheme is proposed with rigorous convergence analysis using a well established CEF. In Section IV, an experiment test on a 7-DoF robotic manipulator is carried out to show the effectiveness and efficiency of the proposed control scheme. Section V concludes the present work and future directions. Throughout the present work, let's denote N and R the set of natural and real numbers, respectively. For a given matrix H, H T denotes the transformation of H, and H ≻ 0 represents that H is positive definite.

A. System Overview
The robotic manipulator utilised in this work is the KUKA LBR IIWA 14 R820 robot (named as the IIWA robot in the rest of the work), as shown in Fig. 1, which is mounted on a mobile platform with both levelling feet and wheels. The IIWA robot consists of n = 7 revolute (R-) joints providing 7-DoF manipulation at the end-effector and in each R-joint, a torque sensor and an angular position encoder are equipped after the corresponding gearbox. The kinematic sketch is illustrated in Fig. 1 and the corresponding Denavit-Hartenberg (D-H) parameters are listed in TABLE I.
In this work, the CoppeliaSim ® is selected for both simulation and real robot control purpose, due to its supporting of simultaneous incorporation with both programming environment (i.e. MATLAB ® in this work) and hardware control.

B. Dynamic Modelling
Consider the following n DoF dynamics of the robotic manipulator (1) where t ∈ R denotes the time instant and k ∈ N + denotes the number of iteration. q k (t),q k (t),q k (t) ∈ R n are the joint position, velocity and acceleration vectors, respectively, at time t of the kth iteration. τ k (t) is the vector of joint torques, which is the system control input. M (q k (t)) ∈ R n×n is the inertia matrix, which is symmetric, bounded and positive definite, C(q k (t),q k (t))q k (t) ∈ R n represents the Coriolis and centrifugal forces, G(q k (t)) ∈ R n represents the gravitational and frictional forces. For the dynamics (1), the following properties always hold: P1 M (q) = M T (q) ≻ 0 for any q ∈ R n , and there exist constants µ 1 > 0 and µ 2 > 0 such that 0 µ 1 I n M (q) µ 2 I n , where I n denotes the identity matrix of dimension n. P2 The matrix ( 1 2Ṁ (q k (t)) − C(q k (t),q k (t))) is skew symmetric and satisfying Assume that q d (t) is the desired trajectory, to facilitate the controller design and subsequent convergence analysis, the following two assumptions are imposed to the system: A1 q d (t) is first and second order continuously differentiable for all t ∈ [0, T ], and are satisfied for all k ∈ N.

III. CONTROLLER DESIGN
In the present work, to improve the control performance of the robotic manipulator, a novel learning-based controller is proposed to realise the high-precision tracking task with the presence of system uncertainties and external disturbances. In the proposed control algorithm, as illustrated in Fig. 2, the torque input τ k to the robotic manipulator consists of three components including the learning controller τ ILC k (t), the inverse-dynamics-based controller τ ID k (t) and the FBC where q d (t) is the desired trajectory,q d (t) andq d (t) represent the desired joint velocity and acceleration, K p ≻ 0 and K d ≻ 0 are the diagonal control gain matrices to be designed, e k (t) := q d (t) − q k (t) represents the tracking error at the kth iteration, γ > 0 ∈ R is the learning gain to be determined. Moreover, without loss of generality, the initial value of the the proposed learning controller is assumed to be τ ILC 0 = 0. Remark 1: Note that the inverse dynamics will give the same result with the same reference signal (i.e. q d (t)). Hence, τ ID k (t) is identical for every k and thus will not affect the performance of the ILC and the FBC. While the coordination of the inverse dynamics and the PD controller can guarantee the control precision of the first iteration. Remark 2: It is worth to note that usually when implementing the FBC and the ILC algorithm together, they would mutually influence each other since the accumulation of the noise caused by derivative action through the iterations, as shown in [20]. In the present work, to reduce the interaction effect, the ILC controller is designed in the structure of (6), where the ILC input is corrected by the FBC signal at the previous iteration.
With the proposed controller (3)-(6), we have the result as shown below.
Theorem 1: Consider the robotic manipulator (1) satisfy the A1 -A2, the control laws (3) -(6) ensure the convergence of the tracking error as the iteration number approaches to infinity, namely, hold for all t ∈ [0, T ]. In the following, the time variable t will be omitted for clarity if no confusion caused. To pave the way for convergence analysis, the following notations are firstly defined: where τ d is the desired control input. Obviously, according to P1 and A1 -A2, it is not difficult to obtain that Therefore, we may focus on the convergence of ξ k and then obtain the convergence of e k . The convergence analysis is based on the Lyapunov-like theory with the well-designed CEF presented as follows Actual position q k (t) where λ ∈ R is a positive constant and

Inverse Dynamics
Proof of Theorem 1. The proof consists of two parts. The first part demonstrates the boundedness and non-increasing property of the J k (t) with respect to k. The second part shows the convergence of ξ k and thus the convergence of the tracking errors.
Part I. From the definition in (5), (6) and (9), we have the difference of the CEF at a time instant t ∈ [0, T ] of two consequent iterations is Differentiating ξ k−1 followed by a multiplication with the inertia matrix M (q k ), results in the following relationship: and Substitute (15) into (14), following several manipulations, we can obtain that From the definition (9) and (19), it is possible to show that there exist two constants c 1 > 0 and c 2 > 0 [21] such that The property P1 shows that there exists a constant µ > 0 ∈ R such that for any bounded q k−1 , Using the inequalities (21) and (22) for (20) gives where h(|ξ k−1 |) = |K d |(γ|K d |+2c 1 +2c 2 |ξ k−1 |). Since there exists a positive constant c 3 > 0 such that 0 < h(|ξ k−1 |) < c 3 , we have It follows that if λ is selected such that λ + 2 − c 3 > 0, then which shows the non-increasing property of the CEF J k . Part II. Computing the cumulative sum for (25) gives that which implies Since J −1 is bounded and J k is positive, we have for t ∈ [0, T ], which implies the convergence (7).

A. Trajectory and Data Acquisition
To validate the proposed control scheme in Section III , a small scale ellipse trajectory in the XY-plane is selected for tracking control, as it requires synchronised actuation of all the 7 joints. The ellipse trajectory contains 9665 samples in total and the duration is 9.8 s. Inverse kinematics of the IIWA robot is utilised to compute the desired joint position, then a low pass filter is used to generate the corresponding joint velocity and acceleration. The joint position, velocity and input torque range responding to the ellipse path are listed in TABLE II.
To implement the proposed control algorithm, the MATLAB ® is deployed to compute the torque input, which is subsequently delivered to the virtual robot in CoppeliaSim ® . Once the simulation results convince of the safety and effectiveness of the controller, the control signal can be then transferred to the real IIWA robot through the UDP socket. Meanwhile, the joint positions of the IIWA robot are acquired through its integrated joint sensors.
In the purpose of demonstrating the effectiveness of the ILC algorithm, comparison in the trajectory tracking between the initial implementation (Case I) and the third iteration (Case II) are considered. Note that at the initial implementation, only inverse dynamics and the FBC are involved in the system, i.e. τ ILC 0 = 0. The repetitive disturbances, i.e. the kinematic uncertainty and initial joint offset, are considered during the experimental test. Furthermore, the environmental disturbance, i.e. platform with wheel configuration, is considered as non-repetitive disturbance.

B. Test results
Based on the aforementioned setup, the trajectory tracking tests are conducted in both simulation and real experiment. The joint positions measured during the the experimental test are presented in Fig. 3 As shown in Fig. 3, it can be noted that the initial offsets of joints 2, 4, and 6 have been set as 10 • , 30 • and 15 • , respectively, which were utilised as system repetitive uncertainty. Moreover, as mentioned in Section II, the mobile platform for the IIWA robot equips with both levelling feet and wheels, and the later option is chosen for providing the nonrepetitive disturbance. As the consequence of the uncertainty and disturbance, oscillations are obviously observed in Case I, which results in a maximum tracking error of 5.78 • in joint 2 and 5.14 • in joint 4, respectively. While in Case II, the maximum tracking errors are reduced to 1.09 • and 1.53 • in joints 2 and 4, respectively.
Furthermore, it is worthy noting that in TABLE III, the maximum tracking error percentage in joint 2 is 21.09%, this may caused by the unwell designed PD gain parameters. While the involvement of the ILC algorithm provides efficient compensation in the control input, which improves the maximum tracking error percentage to 3.99% in only three iterations.
In the case of small joint motion range, e.g. joints 3 and 5, although the ILC algorithm does not provide impressive improvement in the control performance, there is no chattering phenomenon observed neither. This indicates that the proposed structure of the ILC algorithm can work coordinately with the FBC as discussed in Remark 2.

V. CONCLUSIONS
In this work, an ILC algorithm is proposed to enhance the FFC performance for the robotic manipulators by working coordinately with the inverse dynamics module and the FBC module. Through this control scheme, both repetitive and non-repetitive disturbances, as well as various uncertainties can be suppressed. Meanwhile, both the FBC and ILC algorithm contribute to the system convergence as presented in the Lyapunov-like analysis.
To validate the proposed ILC algorithm, an ellipse trajectory tracking experiment has been implemented by a 7-DoF IIWA robot, in which all the 7 joints are actuated synchronously. The resulting maximum tracking error and percentage to the range have been improved from 5.78 • to 1.09 • , and 21.09% to 3.99%, respectively, in only three iterations. It is indicated that the proposed ILC algorithm is capable of compensating the environmental disturbances and system uncertainties, enhancing the system control performance, and working coordinately with the FBC module. Therefore, through learning from the previous system behaviour, the proposed control scheme can improve the control accuracy with successive iterations and finally achieve highprecision tracking performance.
To further explore the effectiveness and efficiency of the proposed learning based control scheme, more complex experiment scenarios and advanced FBC algorithm will be considered in the future work. The system could also be extended in the presence of dynamic payload and physical constraints.