An evolutionary algorithm based approach for selection of topologies in hierarchical fuzzy systems

This paper presents an evolutionary algorithm based approach for selection of topologies in hierarchical fuzzy systems (HFS). The different topologies of HFS for a given dynamical system such as inverted pendulum are investigated and analysed to address the problem of how input configuration in multi-layered structure affects the controller performance. The experiments are conducted to test controller performance for different topologies of the hierarchical fuzzy system. The impact of different topologies on control process is discussed. The results from case study (inverted pendulum) can be extended to other dynamical systems, in particular to a robotic manipulator.


I. INTRODUCTION
Increasing the number of input variables or input fuzzy sets results in an exponential increase in complexity of the rule base.The decomposition of the system into hierarchical fuzzy system is intended to reduce the size of the rule base while maintaining an adequate accuracy.Layered fuzzy logic systems utilize the modularity characterising many physical systems and their mathematical models.The output influenced by one closely related group of input variables may be largely independent of the values of other variables.Therefore, a layered fuzzy logic system can decompose the rule base along these lines of weak interdependence and still maintain a high level of accuracy.
In this paper, different topologies of HFS for a given dynamical system (the inverted pendulum system) are investigated and analysed to address the problem of how input configuration in multi-layered structure affects the controller performance.Examination of the topologies gives insight into the workings of the physical system and its control system.For the inverted pendulum system, a single layer, two layers, three layers, and four layers HFS with different input configurations are examined and controllers' performances compared.There are various approaches to building a topology of the HFS and no consistent guidelines how to do it.In most cases it is human intuition or expert knowledge.The presented research contributes to development of practical applications in robotics.The use of hierarchical fuzzy systems makes possible developing a fuzzy control system for real-time industrial applications, for example: a robotic manipulator.The results presented in this paper are extension of the research presented in [1].The new simulations for the 4-layered HFS are presented and analysed.

II. RELATED WORK
Yi and Yubazaki [2] developed fuzzy controller based on the Single Input Rule Modules (SIRM) and Dynamic Importance Degree (DID).The method was tested for a wide range of system parameters (cart and pole mass, pole's length, etc.).Similarly, Khwan-on et al. [3] developed neuro-tabu-fuzzy controller to stabilize a wide range of inverted pendulum systems using the same SIRM technique.
Combination of hierarchical fuzzy control and sliding mode was presented by Lin and Mon [4].Other examples of hierarchical fuzzy control applied to the inverted pendulum system can be found in [5] and [6].Chang et al. [7] used the inverted pendulum system as a case study for the robust fuzzy-model-based sliding mode.Another popular approach to the inverted pendulum problem is by use of Neural Networks (NNs), Evolutionary Algorithms (EAs) and hybrid techniques, [8]- [10].In the paper by Astrom and Furuta [11] the authors discuss strategies based on Lyapunov analysis for swinging up an inverted pendulum and show that the cart-pendulum system critically depends on the ratio of the maximum acceleration of the pivot to the gravity acceleration.Comparison of energy based strategies with minimum time strategies are provided.In the paper, a designed controller is capable of bringing the pendulum to the upright position in one swing, providing that the control force satisfies u > 2g.Instead of controlling position and velocity of the cart-pendulum system, the method relies on the control task being achieved by controlling the energy of the system.Margaliot and Langholz [12] also used classical Lyapunov approach to design a fuzzy controller and one of their case studies was the inverted pendulum.The system variables were restricted to the pole angle and its angular velocity.The fuzzy rules were derived from the Lyapunov function derivative condition (negative definite derivative).The method main advantage is that it required very little knowledge about the system under consideration.The dynamic equations do not need to be known, only the state of the system must be known and proportional dependence of input variable (angular speed) to the system control output.Furthermore, the authors demonstrated that for linear time-invariant plants of arbitrary order, four Mamdani-type fuzzy rules suffice to guarantee local asymptotic stability.
Yu and Sun [13] developed a fuzzy adaptive control for a class of nonlinear systems and verified it on the example of the inverted pendulum.The control algorithm guarantees global stability of the system with the output of the system approaching the origin if there are no disturbances and

An Evolutionary Algorithm Based Approach for Selection of Topologies in Hierarchical Fuzzy Systems
Juliusz Zajaczkowski and Brijesh Verma uncertainties, converging to the neighbourhood of the origin for all realisations of uncertainties and disturbances.Similar approach was presented by Koo [14] using reference model adaptive fuzzy control.Muskinja and Tovornik [15] designed an adaptive fuzzy controller for a real inverted pendulum and compared various control strategies.Their investigation showed advantages of using fuzzy control theory in real-time applications, specifically for the inverted pendulum system with the aim of fast stabilization of the pendulum and the pendulum cart.Research by Castillo [16] examined stability issues in fuzzy control theory on the example of the inverted pendulum problem.In many reallife applications the reliability of the controller is considered more important than stability issues but development of the stability analysis allows the use of model based approach in fuzzy logic control.The authors used Lyapunov theory to develop stable Mamdani Type 2 Fuzzy Logic Controllers.The curse of dimensionality remains an unsolved problem in fuzzy logic control theory [17] The problem is subject of many research papers with some authors focusing on systematic design of fuzzy logic systems [18] There are also research projects with focus on finding automatically fuzzy structure and parameters of the fuzzy system, for example see [19], [20].One of early methods to reduce the size of the rule base by introducing hierarchical fuzzy control was developed by Raju et al. [20], [21].Raju proposed a hierachical structure in which the most influential system variables were input in the first layer, the next most important variables as input in the next layer, and so on.The first layer control output is an approximation of the controller and is modified by the fuzzy rule base of the next layer, until the final layer produces the final control output.

III. INVERTED PENDULUM SYSTEM
The inverted pendulum system consists of the cart and a rigid pole hinged to the top of the cart, see Fig. 1.The cart moves left or right on a straight bounded track and the pole swings in the vertical plane determined by the track.The dynamics of the system is modeled by the following equations: where x 1 is the position of the cart, x 2 is the velocity of the cart, x 3 is the angle of the pole, x 4 is the angular velocity of the pole, u is the control force on the cart, m is the mass of the pole, M is the mass of the cart, L is the length of the pole, and g is gravitational acceleration.The control force is applied to the cart to prevent the pole from falling while keeping the cart within the specified bounds on the track.The system has the following parameters: m = 0.1 kg, M = 1 kg, L = 0.5m, g = 9.81 m , with state variables restricted by: −1.0 ≤ x 1 ≤ 1.0 and −π /6 ≤ x 3 ≤ π/6.
Fuzzy controller task is to stabilise the system about the unstable reference position ⃗ = ⃗⃗ whilst maintaining the system within the target region (TR) defined by the following bounds: The fuzzy controller is to ensure the state variables convergence to the TR and to maintain them within the TR for a prescribed time limit = 20.0.

Fig. 1 Inverted pendulum system
Each domain region for input variables x i is divided into five overlapping intervals covered by membership sets , k = 1,...,5, encoded as integers from 1 to 5. The output variable u is divided into seven regions covered by membership sets B k , k = 1,...,7.All fuzzy membership functions are assumed to be triangular [1].Given a fuzzy rule base with M rules and n antecedent variables, a fuzzy controller as given by ( 1), with Mamdani product inference engine, uses a singleton fuzzifier and centre average defuzzifier to determine output variables: where ̅ are centres of the output sets B l and are membership functions associated with fuzzy sets A i l .

A. Notation
The following notation for 2-layered HFS topologies is used: L2-mn-kl denotes that input variables for layer 1 are x m , x n and for layer 2: x k , x l, where all integers m,n,k,l ∊ [1,4].Similarly, for a three-layers HFS the following notation: L3mn-k-l denotes that input variables for layer 1 are x m , x n for layer 2: x k , and for layer 3: x l , where all integers m,n,k,l ∊ [1,4].For other 2 and 3-layers HFS the same notation is used, for example: L3-3-41-2 means that input in layer 1 is x 3 , in layer 2: x 4 , x 1 , and in layer 3: x 2 .Generally speaking, the number after capital letter L defines the number of layers, followed by groupings of numbers separated by hyphens representing input variables in every layer.

IV. HIERARCHICAL FUZZY SYSTEM
In the context of HFS, topology means both structure (layers) and input configuration of the hierarchical fuzzy system.In inverted pendulum problem there are four possible structures: 1-layer, 2-layers, 3-layers, and 4-layers, with different input configuration (except single layer topology).The number of rules in the hierarchical fuzzy system is a linear function of the number of input variables.Let assume that there are n input variables in a L-layers structure.For every input variable there are m fuzzy sets associated with that variable.Assume further that in the first layer there is n 1 input variables, 2 ≤ n 1 < n, and L, then the total number of rules in the hierarchical fuzzy system is given by: If m ≥ 2 the number of rules M is minimized when c = 2, which means that there are two input variables in every layer [23].This input configuration provides the minimal number of fuzzy rules in the knowledge base but it does not necessarily provide the best configuration from the control system perspective.Often, such systems do not provide sufficient control performance, especially in complex highdimensional systems.The decomposition needs to be performed along the weak interdependencies between input variables.Obviously, it requires certain knowledge of the physical system in the absence of any automated method of HFS decomposition.Note, that only L3-mn-k-l configuration satisfies the above condition.Topologies L2-mn-kl do not provide the minimal size for the knowledge base.There are six different topologies of the 2-layered hierarchical fuzzy system in the L2-mn-kl configuration: two input variables in the first layer and two input variables plus intermediary control variable from layer 1 in layer 2. This decomposition does not exhaust all possibilities as different input configurations can be considered, for example: three input variables in layer 1 and one input variable plus intermediary control from layer 1 in layer 2. Another possibility is to have one input in layer 1 and three input variables plus intermediary control in layer 2 which would result in knowledge base consisting of 880 rules which is larger in size than single layer architecture.The architecture of 2-layered HFS is shown in Fig. 2.

A. Two Layered HFS
For the inverted pendulum system the first knowledge base has the two inputs to produce as output a first approximation of the control u 1 .This u 1 together with another state input x i and x j , i, j ∊ [1,4] are used as input in the second knowledge base to produce the final control output u.In the first layer there are 25 =5 2 rules in the knowledge base.The l th fuzzy rule for the first layer has the form: Then (u 1 is B l ), where , k = 1,2,3,4, are fuzzy sets for input variables x k , k = 1,2,3,4, respectively, and where are fuzzy sets for output variable u 1 .For the second layer there are 175 = 7(5) 2 rules in the knowledge base.The l th fuzzy rule for the second layer has the form: If (u 1 is C l ) and (x i is A i l ) and (x j is A j l ) ) Then (u is B l ), where C l are fuzzy sets for the input control variable u.There are a total of 200 fuzzy rules in this hierarchical structure while there are 625 rules in the single layer rule base: 5 4 = 625.

B. Three Layered HFS
In L3-mn-k-l configuration there are twelve different topologies for the 3-layered hierarchical fuzzy system with two input variables in the first layer, and one input variable plus intermediatory control in layer 2 and 3. Again, this decomposition does not exhaust all possibilities, as different input configurations can be considered but with an increased number of rules in the knowledge base.The architecture of 3-layered HFS is shown in Fig. 3.This input configuration provides the minimal number of fuzzy rules in the knowledge base.For this system the first knowledge base has the two inputs, x i and x j , i, j ∊ [1,4] to produce as output a first approximation of the control u 1 .This u 1 together with x k are used as input in the second knowledge base.Then the second layer produces another approximation of control u 2 which with x l is used as input to the third (and final) layer to produce the final control output u.There are a total of 95 fuzzy rules in this hierarchical structure.

C. Four Layered HFS
There are twenty four different topologies for the 4layered HFS for the inverted pendulum with one input in every layer plus intermediatory control in layer 2, 3, and 4. The architecture of 4-layered HFS is shown in Fig. 4.
In the first layer there are only five rules in the knowledge base.The l th fuzzy rule for the first layer has the form: where k = 1,2,3,4 are fuzzy sets for input variables x k , k = 1,2,3,4, respectively, and where B l are fuzzy sets for output variable u 1 .For all the other layers there are 35 = 7 • 5 rules in their respective knowledge bases.For the second layer the l th fuzzy rule has the form: If ( (u 1 is C l ) and (x k is A k l ) ) Then (u 2 is B l ), where C l are fuzzy sets for the input control variable u 1 .Fuzzy rules for the third and fourth layer has a similar form.There are a total of 110 fuzzy rules in the 4-layered hierarchical structure.

V. ALTERNATIVE TOPOLOGIES
There are other input configurations within two and three layered hierarchical structures that do not have two inputs in the first layer (one of the conditions for the minimal size of the rule base).Two examples of 2-layered alternative input configurations are shown in Fig. 5.In the first configuration there are three input variables in layer 1 and one input variable plus intermediary control in layer 2 which results in 160 rules in the knowledge base.In the second configuration there is one input in layer 1 and three input variables plus intermediary control in layer 2 which results in a knowledge base of 880 rules -more than in a single layer knowledge base.Two 3-layered alternative topologies are shown in Fig. 6.Both configurations have 215 rules in their rule bases.

VI. PROPOSED EVOLUTIONARY ALGORITHM BASED APPROACH
The HFS decomposition is not unique and may give rise to variables with no physical significance.This can cause difficulties in obtaining a complete class of rules from human experts.Therefore, the rules need to be learnt by some learning algorithm, for example an evolutionary algorithm (EA).These fuzzy rules are typically evolved with no previous knowledge other than input-output data, or the physical system model.
In the knowledge base of any layer each fuzzy rule is uniquely defined by the position of the consequent part in the string.The consequent part is identified by a particular output fuzzy set, for example, B k .Such a fuzzy set can be identified by the integer k  [1,7].For example, 2-layered fuzzy rule base can be represented as a linear individual string of M = 25 + 175= 200 consequents, ⃗ (a 1 ,…,a 200 ), where a j is an integer [1,7] for j = 1,...,200.Other hierarchical fuzzy system structures can be represented in a similar fashion.
Full replacement policy (constant population size) is used and tournament selection with size n T = 4.A strong elitism policy is implemented: four copies of the top five individuals are passed to the next generation.To maintain diversity of the population crossover operators of parent strings to form two children in the next generation are used.In examination of different topologies so called random crossover is implemented.The random crossover procedure creates child1 from parent2 by copying it, then randomly selecting m-genes in the parent1 string to copy them in the corresponding positions in the child1 string.The procedure is repeated for the child2 string with parent strings roles reversed.The random crossover operator gives more control over crossover process as the number of genes subject to exchange can be arbitrarily determined.Mutation is undertaken with probability p m whose value is determined by a mutation schedule that decreases from 0.8 to 0.001 over 300 generations.The objective function is evaluated as follows: each string ⃗ is decoded into two or more components defining the fuzzy knowledge base for each layer, then the formula ( 2) is used to evaluate u 1 , u 2 , and u 3 to find the final control to be applied at each value of the state ⃗.The system state equations are integrated by the Runge-Kutta algorithm with step size 0.02.The fitness f k to be minimised, is then calculated based on certain measures of the behaviour of the system: the accumulated sum of normalised absolute deviations of and from zero, the average deviation from vertical, the average deviation from the origin or T -T S where T S , the survival time, is taken to mean the total time before the pole and cart break defined bounds.A penalty is added to the objective if the final state breaks the following bounds: , leaves the designated target region.The objective function is defined as: , where x max = 1.0, θ max = π/6, ṁax = 1.0, ̇max = 3.0.N is the number of iteration steps.Survival time is defined as: T S = 0.02•N, with T = 0.02• N max , where the max.number of iterations N max = 1000.

VII. EXPERIMENTS AND ANALYSIS OF RESULTS
The relatively low number of possible topologies for the inverted pendulum enables their examination one by one and finding the topology with the best controller performance.The best controller is decided arbitrarily by considering its performance: time to reach target region, magnitude of control, and oscillations.For each topology ten simulations are run with randomly generated initial populations.The initial population P(0) = { ⃗ : k = 1,...,M p } is determined by choosing the a j as a random integer in [1,7].Population size is set at M p = 500.The initial state is: ⃗ 0 = (0.5, 0.0, 0.01, 0.0).The EA is terminated after 300 generations.The following fitness function parameters: ω 1 = 3000, ω 2 = 2000, ω 3 = 0, ω 4 = 0, ω 5 = 5000, are selected for all simulations except single layer fuzzy system with: ω 1 = 1000, ω 2 = 0, ω 3 = 1000, ω 4 = 0, ω 5 = 5000.Selecting different weights ω introduces a bias in the EA towards one or a group of input variables.In most simulation, the most difficult was to control the cart's position and therefore ω 1 was assigned a larger value than other weights, except ω 5 that corresponds to the survival time.Controller acts on the system for T = 20.
Comparison between controllers with L2-14-23 topology and L2-23-14 shows that swapping input variables between layer 1 and layer 2 can have a significant effect on the controller performance.This indicates that the HFS topology is a decisive factor in the controller performance.This assertion is confirmed by the 3-layered HFS results presented in the following section.

A. Two Layers Topology Results
The EA converged on average to a satisfactory solution within 100-184 generations.The best result is shown in Fig. 7, where pole angle x 3 and its angular velocity x 4 are input to layer 1, and cart position x 1 and its speed x 2 as input variables to layer 2. Stabilisation is very quick and to 5 decimal place accuracy for all variables.This result gives the first indication as to which input configuration provides the best controller performance.Furthermore a small control effort is required with control values in [−2.0, 1.6].

B. Three Layers Topology Results
For the 3-layers HFS, the evolutionary algorithm converged to a satisfactory solution within 150-200 generations.Controller defined by topology L3-34-1-2 achieves all state variables smooth convergence to the origin without any large oscillations.The magnitude of control is reasonably low, within range of [−2.4,1.4].It is one of the best controllers among the 3-layers HFS.However, in ten simulations the controllers L3-34-2-1 generally outperformed the controllers L3-34-1-2 and therefore should be considered as the best controller amongst the 3-layers HFS.Note, that the state variables convergence to the TR is faster for L2-34-12 than for L3-34-1-2.As can be seen from comparison of L3-13-2-4 and L3-13-4-2 controllers, a seemingly insignificant change in input configuration in layer 2 and layer 3 results in a significant change in the controller performance.A similar effect can be seen by in topologies L3-14-2-3 and L3-14-3-2.Physical properties of the system under consideration require grouping of the input variables along weak state variables interdependence.In case of the inverted pendulum this grouping corresponds to two subsystems: the cart represented by x 1 and x 2 , and the pole represented by x 3 and x 4 .Swapping the input variables between the layers but preserving to some extent abovementioned groupings has little effect on the controller performance.When this grouping principle is broken, the results are often detrimental (depending which variables are more influential in the dynamical system).In case of L3-14-2-3 and L3-14-3-2 it seems that controlling the angle of the pole is more crucial than controlling the cart's velocity as it is reflected in both topologies.
Examination of 4-layered topologies can determine which input variables are most influential in the inverted pendulum system.It was found that topologies L2-34-12 and L3-34-2-1 are the best performing controllers.The simulation results show that the topology L4-4-3-2-1 is the most consistent in producing well performing controllers for ten different initial populations with L4-4-3-1-2 close behind.This clearly indicates that the most influential input variable is the angular speed of the pole x 4 , second -the angle of the pole x 3 , and then cart's speed x 2 and its position x 1 .performance of these controllers they cannot be ruled out on the basis of their large rule bases.The large number of rules does not hamper performance.It is noted that 4-layered topologies are significantly outperformed by lower layers HFS controllers.This indicates that for the inverted pendulum problem this ladderlike structure does not produce well performing controllers.

VIII. CONCLUSIONS
In this investigation the EA based approach for topology was examined.The EA approach enabled comprehensive analysis of simulation results and detailed conclusions presented below.Fine-tuning of the EA parameters allowed achieving better controller performance.
It was observed that the performance of the fuzzy controller is not related to the EA learning speed.It has been shown that it is important to select the correct input variables into the first layer to achieve effective and accurate control.Furthermore, structure of the second and third layer in 3layered HFS plays a significant role as reversed order in input in those layers produced dramatically different results, as shown for example in case of L3-13-2-4 (poor results) and L3-13-4-2 (good results).Similarly, controller L3-14-2-3 shows poor results and L3-14-3-2 good results.Both cases illustrate how intricate interdependencies between input variables can be.Results from the 4-layered HFS simulations established that the most influential variable is x 4 (angular velocity), then x 3 (pole angle) in the first grouping followed by x 2 (cart's velocity) and x 1 (cart's position) in the second grouping.Simulation results obtained for two, three and four layered HFS confirm that it is important to control the inverted pendulum, by examining first its angular speed and angular position then the cart's speed and position.
The controller with topology L1-1234 stabilises the system relatively well, with no preference given to any input variable, and interdependence between input variables (being locked in the fuzzy rules) remains hidden.Only by decomposition of the HFS (by breaking a single knowledge base into a hierarchically structured knowledge base) this interdependence comes into play with dramatic effect.
The topologies analysis shows that the 2-layered HFS provides a slightly better solution to the control problem of the inverted pendulum than 3-layered HFS.This result reflects the physical nature of the inverted pendulum system with pole and cart variables grouped in two separate subsystems that are mirrored in the 2-layered HFS.However, the 3-layered HFS significantly reduces the size of the knowledge bases while providing control system of similar performance.
The investigation into HFS topologies suggests that the size of the knowledge base is not an important factor in controller performance.On one side there is topology L2-3-412 with 880 fuzzy rules or single layer topology L1-1234 with 625 fuzzy rules, and on the other side 3-layered topology with 95 fuzzy rules in its knowledge base, except for 3-layered ‗alternative topologies', with some of them providing similar controller performance.However, the topology of the HFS seems to be the decisive factor in controller performance.
Similarly, the HFS topologies investigation shows that the number of layers is not an important factor in the controller performance (in terms of control magnitude and stabilisation rate of the state variables).This fact allows large single knowledge base to be replaced with the HFS without loss of controller performance.In fact, the HFS produced more efficient controllers (in terms of system stabilisation) than single layer controller except for control magnitude that is lowest for a single layer fuzzy system.
The simulation results indicate that a particular input configuration in the HFS layers is more important than the number of layers as good controller performance was achieved for L2-34-12, L3-34-2-1, and L4-3-4-2-1.This indicates that interdependence of variables plays a crucial role in finding the ‗optimal' HFS for a particular problem.Examining the nature of variables interdependence is a key to an automated determination of the decomposition of the fuzzy model of control.The decomposition of the hierarchical fuzzy structure should be performed along weak interdependency between input variables.However, with more complex dynamical systems there might be multiple weak interdependencies in input configuration.In such cases either expert knowledge is required to resolve the decomposition problem or an automated process that finds optimal or near optimal hierarchical fuzzy topology.

Fig. 15
Fig. 15 State variables convergence L4-4-3-1-2.It is noted that 4-layered topologies are significantly outperformed by lower layers HFS controllers.This indicates that for the inverted pendulum problem this ladderlike structure does not produce well performing controllers.