Heterogeneous Unknown Multiagent Systems of Different Relative Degrees: A Distributed Optimal Coordination Design

This study delves into the distributed optimal coordination (DOC) problem, where a network comprises agents with different relative degrees. Each agent is equipped with a private cost function. The goal is to steer these agents towards minimizing the global cost function, which aggregates their individual costs. Existing literature often leans on known agent dynamics, which may not faithfully represent real-world scenarios. To bridge this gap, we delve into the DOC problem within a network of linear time-invariant (LTI) agents, where the system matrices remain entirely unknown. Our proposed solution introduces a novel distributed two-layer control policy: the top layer endeavors to find the minimizer and generates tailored reference signals for each agent, while the bottom layer equips each agent with an adaptive controller to track these references. Key assumptions include strongly convex private cost functions with local Lipschitz gradients. Under these conditions, our control policy guarantees asymptotic consensus on the global minimizer within the network. Moreover, the control policy operates fully distributedly, relying solely on private and neighbor information for execution. Theoretical insights are substantiated through simulations, encompassing both numerical and practical examples involving speed control of a multimotor network, thereby affirming the efficacy of our approach in practical settings.


I. INTRODUCTION
ulti-Agent Systems (MAS) have been utilized recently in a wide variety of different applications such as robotics [1], power systems [2], sensor networks [3], air traffic control [4], and so on.Many different distributed control protocols have been developed so far to ensure agents reach consensus, which is the most fundamental objective in MASs, despite various real-world constraints such as time-varying communication topology, communication delays, limited bandwidth, and saturation in agents' actuators.However, observing some sense of optimality is an essential portion of control goals for most real-world applications.Optimality in MAS is interpreted in several ways, which causes several distinct fields of study to emerge, such as Distributed Optimization Problems (DOP) [5], Cooperative Optimal Control Problems [6], Differential Games [7], and Differential Graphical Games [8].

Distributed Optimal Coordination
In DOP, agents aim to calculate the minimizer of a global cost function, which is the summation of the agents local cost function.Attaining this goal must be done through a distributed algorithm.This means that the algorithm must be designed so that each agent only requires information regarding its local cost function and the information transmitted by its neighbors to implement the algorithm.DOP has been investigated for a long time.Some pioneering works in this field of study can be found in the context of parallel and distributed computation [9], [10].This decade has witnessed a surge in researchers' interest in DOP owing to its growing renewed applications in communication networks, machine learning, and sensor networks, to name a few [11].Thus, several significant papers have been published recently proposing various combinations of consensus-based and gradient-based approaches to deal with DOP [12]- [15].
The emergence of Cyber-Physical Systems (CPSs), where computation/communication (cyber) parts and dynamic/control (physical) parts are integrated to perform desired tasks, has extended the notion of DOP.Many real-world CPSs, such as power systems, a team of robots, UAVs or UGVs, and sensor networks, can be categorized as MASs consisting of physical dynamic agents.Investigating DOP for dynamic MASs is often termed Distributed Optimal Coordination (DOC) by researchers in the control community.There exists a marked difference between DOC and conventional DOP.When it comes to studying DOC, the designer must take into account the agents dynamics, which is not the case in conventional DOPs.Regarding DOC, computing the minimizer is only a portion of an optimization algorithm role and the algorithm must control the state of agents, which are physical variables, in such a way that they achieve consensus on the minimizer.Consequently, the traditional algorithms for DOP [12]- [15] are not viable for DOC.
Generally speaking, existing algorithms for DOC can be categorized into 1) double-layer open-loop and 2) integrated closed-loop classes.The first class takes advantage of a traditional DOP algorithm as its top layer, often called the cyber layer, computing the minimizer and generating a reference trajectory for the bottom layer.In the bottom layer, often referred to as the physical layer, each agent is equipped with a controller enabling the agent to track the trajectory provided by rezanaseri@aut.ac.ir).

M
the top layer and finally reach the optimal point [16].Although each of these two layers has been broadly investigated independently, combining them for DOC is a daunting task because they become dependent on each other in this case.The difficulty stems from the fact that the dynamics of agents only permit them to track particular trajectories, which means the top layer is only allowed to generate tractable reference signals.In contrast, in this class, the bottom layer does not provide the top one with any feedback [17].Despite the first class, in the integrated closed-loop structure agents are not provided with reference trajectories; instead, the control inputs, which are constructed utilizing the MAS states or outputs, explore the optimal point and steer agents toward this point simultaneously.This method has been widely employed in the literature for various dynamic MASs, including but not limited to single/double integrator MASs [18]- [20], high-ordered MAS [21]- [23], Euler-Lagrangian interconnected systems [24]- [26], nonlinear MAS [27], [28].

Motivation and Contribution
In many industrial applications, a network of mixed agents of heterogeneous dynamics must interact with each other to realize global tasks.Furthermore, heterogenous MASs possess more capability to perform complicated missions; thus, these systems might perform more satisfactorily when working in environments including considerable diversities [29].Take the example of a network of UGVs and UAVs cooperating to explore an environment encompassing terrestrial and aerial areas [30].UGVs are often considered second-order systems, and the UAVs are often modeled as higher-order systems [31].
Several remarkable works investigating the DOC problem for heterogenous MASs can be found in the literature [17], [32]- [35].Authors in [34] proposed a solution assuming the gradients of local cost functions follow a specific form.In [17], a more general structure is considered for the gradients of local cost functions.Instead, agent system matrices must feature a set of algebraic conditions.Authors in [32] relaxed these conditions.Furthermore, their DOC algorithm ensures exponential convergence.In [35], the authors presumed the same condition on agents dynamics as [17].They developed a distributed control protocol to address the resource allocation application under inequality constraints.The main contribution of [33] is to remove the restricting conditions on the agents dynamics.To do so, the authors exploited the double-layer open-loop structure (first class) to develop their optimization algorithm in contrast with algorithms presented in [17], [32], [34], [35] utilizing the integrated closed-loop structure (second class).They derived conditions ensuring the reference trajectories generated by the top layer are tractable by the physical agents.
However, the papers mentioned so far do not address the DOC problem when there exist some uncertainties that deplete the effectiveness of their results in practice because a considerable portion of real-world applications is subjected to uncertainties or contains some unknown parameters.Recently, some research has been conducted to address this issue [36]- [39].Authors in [36] offered an adaptive distributed optimization algorithm to deal with the case in which the Lipschitz constants and network connectivity are unknown.In [39], authors considered a DOC problem where the gradients of the local cost functions are unknown.They addressed this problem for a network of agents with Euler-Lagrange dynamics.
Authors in [38] considered a MAS composed of nonlinear agents whose dynamics can be represented by parametric strictfeedback form.They addressed the same problem for this MAS as the one in [39].To do so, they proposed an adaptive backstepping distributed optimization algorithm.However, the agents dynamics are assumed to be known in these three works.The DOC problem for Euler-Lagrange MASs in which the inertial parameters are unknown is studied in [37].In [16], authors considered agents with uncertain high-order nonlinear dynamics.They designed a double-layer DOC algorithm based on the high-gain approach for this network.Their proposed algorithm needs the uncertain part to be constrained by a linear growth condition which greatly restricts the generality of the algorithm.Furthermore, the algorithm can only guarantee that the agents converge to a small vicinity of the global minimizer.This paper considers an LTI heterogeneous MAS composed of a mixture of agents with different relative degrees.We introduce a novel distributed control policy taking advantage of a double-layer open-loop structure.The top layer is a distributed double integrator optimizer.In the bottom layer, each agent is provided with an adaptive controller enabling the agent to track the reference trajectory despite the unknown dynamics of the agent.It should be noted that LaSalle's Invariance Principle, Lyapunov theorem, and Barbalet lemma play vital roles in proving our design's global convergence and stability.
Our contributions to improving the existing results can be listed as follows: • Despite the results in [17], [32]- [35], which consider heterogenous MASs without uncertainties or unknown parameters, this paper takes into account the heterogeneous unknown agents case.
• Different from [36], [38], [39], in which the considered uncertainties are not associated with the agents dynamics, this paper deals with the case where the dynamics of agents are entirely unknown.In [37], agents are Euler-Lagrange, not encompassing LTI systems considered in this work.In addition, in [37], only one unknown parameter exists in the agents models, while in this work, all system matrices are assumed to be unknown.
• The agents dynamics considered in [16] does not encompass general LTI form.Furthermore, the results only provide approximate consensus.However, our presented results can be applied to more general form of LTI MAS and also ensures asymptotically consensus on the minimizer.

Paper Organization
The remainder of this work is arranged as follows.In Section II the preliminaries are presented.Section III includes the formulation of the DOC problem.Section IV offers a novel distributed adaptive control policy and elaborates on its convergence and stability properties.Section V provides and discusses numerical examples to verify the correctness of the theoretical results.Finally, Section VI is devoted to the conclusion.

Basics on Graph Theory
The communication network between agents is described by a graph (, , ) where each of its nodes is associated with an agent. = { 1 ,  2 , … ,   } is the set of the graph's nodes in which  denotes the number of agents in the network.The communication links between nodes are included in the set  ⊆  ×  where (  ,   ) ∈  shows that th agent directly receives information from th node.In this paper, the communication graph  is assumed to be undirected and connected.The former means if (  ,   ) ∈ , then (  ,   ) ∈  and based on the latter, at least one communication path exists between any pairs of agents.The adjacency matrix associated with graph  is denoted by the square matrix  = [  ] ∈ ℝ × in which   = 1 when (  ,   ) ∈ , otherwise   = 0.Moreover, self-loops do not exist in  meaning that   = 0.The associated Laplacian matrix is presented by the matrix  = [  ] ∈ ℝ × .In this matrix,   = −  for  ≠ , and   = ∑    =1 . Since the communication graph is undirected and connected,  has  − 1 eigenvalues with positive real parts and one zero eigenvalue which its associated eigenvector is   satisfying    = 0.
Lemma 2:  * is the unique minimizer for the strictly convex function ℎ() if and only if the following equation holds [21].

III. PROBLEM STATEMENT
A MAS of N agents sharing their information through a communication network is considered as follows.
Where    ∈ ℝ   ×1 is th agent state vector,   ∈  is the control input, and    ∈ ℝ is the output.The system matrix    ∈ ℝ   ×  , input vector    ∈ ℝ   ×1 , and output vector    ∈ ℝ   ×1 describe the th agent state-space model.The th agent's transfer function is given by: The state-space model and the coefficients    ,    , and the highfrequency gain    are unknown for all agents.
Where   :  →  stands for th agent local cost function.
The term "distributed" refers to the case that the output of each agent is only affected by its own information and its neighbors' information in the network through its control protocol.
We make the following assumption on agents' cost functions.

IV. MAIN RESULTS
In this section, a distributed control protocol steering MAS (2) toward output consensus on  * is introduced inspired by [21].Some definitions and lemma must be presented first.
Let us define the following two auxiliary systems.
Now, we are ready to propose our main results.
For agents with  * = 1,   ,   and   is defined as follows: For agents with  * = 2,   ,   and   is introduced as follows: In both cases, Σ  is a positive definite matrix and ̅  is defined as: Where   and   are positive scalers.Furthermore

∎
Proof: The proof includes four main steps.In the first step, it is indicated that the equilibrium point of reference MAS (7), (   ,    ,    ), solve the minimization problem (4), i.e.    =  * , where  * is solution for (4).Next, using LaSalle's Invariance Principle, we prove that the reference MAS system (7) achieves a consensus on its equilibrium point, i.e. lim →∞   =    =  * .
Third, the proof of boundedness of ℰ  =    −   is provided.Finally, taking advantage of Barbalet Lemma we demonstrate that lim →∞ ℰ  = 0 meaning that    converges to  * .
Step 1: The compact form for the reference MAS (7) is Where  = ( According to (14), the following equations hold at the equilibrium point.
We can conclude from ( 15) and ( 15): Based on Assumption 4,  is connected which means    = 0, so the following equation is derived: Which means ∑    =1 remains constant.On the other hand (0) = 0, thus the following equation is achieved.

𝟏 𝑁
= 0 Equation ( 16) along with (18) result in the following equation In other words, ∑   (   )  =1 = 0, which according to Remark 1, can be rewritten as  ∑   (  )  =1 = 0. Hence, one can obtain: Step 2: This section is devoted to proving the asymptotic stability of the equilibrium point of the reference MAS (7).
We define the following state space transformation.
Applying the transformation ( 22) on ( 21) derives the following state space equation.
in other words First, we concentrate on  ̅ and propose the following Lyapunov candidate to investigate the stability properties of  ̅.
Taking time derivate from (25) results in the following equation.
Next, we define the following set.
Thus for  ̅ ∉   , the following inequality holds.
From ( 8), the following result is obtained.
which means  ̅ enters   and remains in this set.
Next, we aim to investigate the stability properties of  ̅.The following Lyapunov candidate is considered to do so.
It is evident that   ≥ 0 and   = 0 if and only if  ̅ = 0 owing to the fact that the communication graph is connected.The time derivative of (32) along its trajectories results in the following equation: We define the following set.
Thus for  ̅ ∉   , the following inequality holds.
Based on (8), the following result is obtained.
According to this result,  ̅ will converge to   and remain in this set, which means  ∆ 2 ( ̅) =  ̅.Subsequently, (24) can be rewritten as follows: By applying the state transformation  =  −1  ̅̅̅ to ( 37), ( 14) can be reformulated as follows.
Moreover, we define ℛ  ≜ {(, , ) ∈   | ̇ = 0}.Using (42) and ∇ 2 () > 0, the following holds Next, we aim to explore ℛ  for its largest invariant set.To do so, we take the derivative of  + ∇() and make it equal to zero, as follows From ( 43) and (44) we derive which means the largest invariant set in ℛ  is as follows: Equations ( 43) and (46) satisfy the conditions under which LaSalle's Invariance Principle can be applied.Using this principle and (42), we can conclude that (  ,   ,   ) is asymptotically stable.Utilizing this conclusion and ( 20), the following result is reached: Step 3: The dynamics of   and   in the reference MAS ( 7) can be rewritten as follows using (12).
Thus the following transfer function is derived: * = 1 case: In this case, ̇ = −    + ̅  from ( 10).Thus, one can define the reference transfer function for th agent with  * = 1 based on (49): And the following equation holds: In other words, (  ℬ     ) is a realization of  ,1 ().Please note that  ,1 () is a SISO transfer function, while the system (51) has 3  − 2 eigenvalues, so 3  − 1 uncontrollable or unobservable eigenvalues exist.However, all of the zeros of MAS ( 2) is assumed to be located on the left-half plane, which means all zeros-pole cancellations are stable, thus, the system (51) is internally stable.Now, let    ,    , and    are unknown, so the control input is expressed as follows: The transfer function from      () to ̅  () is which is SPR.Thus, according to Meyer-Kalman-Yakubovich lemma, matrices   > 0 and   > 0 exist such that: Now, we propose the following Lyapunov candidate to investigate the stability of (58): The time derivative of (60), along with ( 9) and ( 58) is calculated as follows: It follows from ( 59) and (61): It can be easily verified that  ̇() = ̇().Substituting ̇() from ( 9) into (62) obtains the following equation: In other words  ̇() ≤ 0. Thus, the following results are concluded: * = 2 case: In this case,   = ̅  from (11).Consequently, we define the reference transfer function for th agent with  * = 2 as follows: Since  * = 2, according to Bezout Identity, in the case that    ,    , and    were known, the distributed control policy would be found such that the input-output behavior of the agent  can be expressed by    () =   ()  ().Thus, the closed-loop system for th agent can be expressed as: Where And the following equation holds: Now, let    ,    , and    are unknown.Thus, one can formulate the closed-loop system for th agent as follows: Furthermore, because of (  ℬ     ) is a realization of   1 (), in the reference MAS (48),   can be described by the following equation: Next, we define   ≜   −   and ℰ  ≜    −   .Considering (56) and (57), the differential equation for   is achieved as: The transfer function from (  − (  * )    ()) to ̅  () is   () = 1 (+  )(+  ) .compared to   () for  * = 1 case in (50), here   () is not SPR, hence Meyer-Kalman-Yakubovich lemma could not be applied.To resolve this issue, we define  ̅  as follows: Using ( 9), (11), and (72), the following equation is obtained: Equations ( 13) and ( 72 The following Lyapunov candidate is proposed to investigate the stability of (78): The time derivative of (80) along with the trajectories of system (78) is obtained as follows: ̇() = ̂ Since   =   −   * , the following equation is derived using (11)  ̇ = −(   )Σ  (   −   ) ̅  (83) Substituting (83) into (82) results in: It is obvious that  ̇() ≤ 0. Thus, the following conclusion holds: (), ̂(), ℰ  (),   () ∈ ℒ ∞ (85) Step 4: In both  * = 1 and  * = 2 cases ̅  = (    )  + (  +   )  +   .As it can be observed in (7),   is bounded owing to the definition of the saturation function  ∆ (. ).Furthermore, in Step 2, it is shown that lim →∞   () =   , lim →∞   () = 0 assuring that   (),   () ∈ ℒ ∞ .Accordingly, the following result is derived: We would rather separate the remainder of the proof in Step 4 into  * = 1 and  * = 2 cases for the sake of clarity.
Case  * = 1: As it can be observed in (10), ̅  () is the input of a BIBO system whose output is   ().Thus   () is bounded too: Moreover, it is shown in step 3 that   is Hurwitz.This fact and (87) together confirm that   () is a bounded input for an LTI system with Hurwitz eigenvalues.As a result, the following holds: Using ( 64) and (88), the following conclusion is obtained: () includes    ,  1 and  2 , thus: From ( 87) and (90), the following is derived From ( 64), ( 90), (91) and our previous results confirming that the reference MAS converges to its equilibrium point, we come to the conclusion that all signals in unknown SISO heterogeneous MAS (2) under the proposed control policy in (9) are bounded.
Integrating both sides of (63) achieves the following equation: We have already shown that In addition, all signals on the right-hand side of the dynamics for   () provided in (58) are bounded.Thus, the following result is derived.
The results in (64), (93), and (94) are summarized as follows: Consequently, taking advantage of Barbalet's lemma, we can conclude: Case  * = 2: Based on (12)   = ̅  , thus according to the argument provided in the beginning of this step, the following conclusion is achieved In (70)   is Hurwitz.This fact along with the boundedness of   () result in the following: can be expressed as follows owing to the fact that   =   −1 ()  .
According to Assumption 2, Assumption 3, and (6), we can deduce that each element in  ̅  is the output of a proper stable transfer function whose input is   or   .We have already proved that ℰ  () ∈ ℒ ∞ , thus   ∈ ℒ ∞ .Moreover   () ∈ ℒ ∞ from (98).As a result, the following is derived: We have defined ̂ ≜   − ℬ     () ̅  ().  , ̂ and  ̅  are bounded from (85) and (101).Hence, we can conclude: Using ( 99) and (102), the following conclusion is obtained: () includes    ,  1 and  2 , thus: From ( 85), ( 101), (104) and our previous results confirming that the reference MAS converges to its equilibrium point, we come to the conclusion that all signals in unknown SISO heterogeneous MAS (2) under the proposed control policy in (9) are bounded.
By integrating both sides of (84), one can achieve the following equation.

−𝑉
In addition, all signals on the right-hand side of the dynamics for ̂() provided in (78) are bounded.Thus, the following result is derived.
Fig. 1 shows that the outputs of all agents converge to the solution of the global cost function.The boundedness of the closed-loop system must be validated too.According to Fig. 2 all MAS states are bounded.Fig. 3 to Fig. 8 depict that other states, including reference MAS states   ,   and   , auxiliary states  1 and  2 , and the time-variant control gain   are bounded too.Moreover, it can be observed in Fig. 4 that   converges to zero as expected.

Assumption 1 :Assumption 2 : 2 . 3 :
(   ) is known.The relative degree of each agent transfer function i.e.   * =   −   are assumed known and also   * = 1,Assumption The zeros of each agent transfer function are located in the open left-half plane.Assumption 4: The communication graph between agents  is assumed undirected and connected.This paper's main goal is to propose a solution for the following DOC problem.Problem 1: Design a distributed control protocol   ,  ∈ {1,2, … , } for the MAS (2) gaurantees output consensus on the optimal solution of the following optimization problem.min ∑   (   )  =1   ∈ ℝ ..   =     ∀,  ∈ {1,2, … , }

Remark 1 :
According to Lemma 1 and Lemma 2, Assumption 5 ensures  * minimizes the global cost function if and only if the following equation holds.Furthermore,  * whould be unique.∑ ∇  ( * )