A constraint consensus memetic algorithm for solving constrained optimization problems

Constraint handling is an important aspect of evolutionary constrained optimization. Currently, the mechanism used for constraint handling with evolutionary algorithms mainly assists the selection process, but not the actual search process. In this article, first a genetic algorithm is combined with a class of search methods, known as constraint consensus methods, that assist infeasible individuals to move towards the feasible region. This approach is also integrated with a memetic algorithm. The proposed algorithm is tested and analysed by solving two sets of standard benchmark problems, and the results are compared with other state-of-the-art algorithms. The comparisons show that the proposed algorithm outperforms other similar algorithms. The algorithm has also been applied to solve a practical economic load dispatch problem, where it also shows superior performance over other algorithms.


Introduction
Solving constrained optimization problems (COPs) Cai 2012a, 2012b) has been an important research topic in the optimization, operation research and computer science domains. In solving COPs, an acceptable solution means that it must be feasible and either optimal or near optimal. An optimal solution in a COP may exist either on the boundary of, or well inside, the feasible space. In this case, if an infeasible solution vector is assisted to move towards a feasible region, it will have a higher chance of quickly reaching the optimality or near optimality in the search process.
Over the past few decades, a variety of population-based algorithms has been introduced for solving COPs, such as the genetic algorithm (GA) (Goldberg 1989), particle swarm optimization (PSO) (Kennedy and Eberhart 1995), differential evolution (DE) (Storn and Price 1995;Wang and Cai 2011), evolutionary strategy (ES) (Rechenberg 1973;Wang and Cai 2011) and evolutionary programming (Fogel, Owens, and Walsh 1966). In these algorithms, different mechanisms have been applied to either reduce or remove the constraint violation of infeasible individuals. In the evolutionary computation domain, these methods are widely known as constraint handling techniques. Some of the popular methods are: feasibility rules (Deb 2000), the ε-constrained method (Takahama and Sakai 2010), stochastic ranking (Runarsson and Xin 2000), multi-objective concepts Cai and Wang 2006;Wang et al. 2007aWang et al. , 2007b, novel penalty functions (Jianhua et al. 2007) and novel special operators (Leguizamon and Coello 2009). These techniques assist the ranking and selection process. However, once the ranking and selection are done, they have no direct influence in the actual search process (that involves crossover, mutation and/or local search method). This means that, after selection of parents, there is no difference between the actual search process in unconstrained and constrained optimization problems.
In the literature on optimization, there are specialized methods (such as projection algorithms) that deal with the feasibility issues as a search process (Censor and Zenios 1997). The projection methods basically guide the infeasible solutions to move towards the feasible space. There are various projection methods, including orthogonal, sequential, simultaneous, component averaging and constraint consensus (CC). The orthogonal projection of an infeasible point, so as to satisfy a constraint, is defined as the closest feasible point (Xiao et al. 2003). In the sequential projection algorithm, the current infeasible point is updated for each violated constraint in turn (Chinneck 2004). In simultaneous projection algorithms, the current infeasible point is updated for all the violated constraints simultaneously, and then the final movement to feasibility is calculated. In gradient projection algorithms, to move from a current infeasible point to a feasible point, the constraint violation and the gradient of that point are required (Chinneck 2004). The component averaging projection algorithms consider the fact that the calculation for the final movement vector should contain only the constraints that contain particular variables, as not all of the constraints contain all of the variables (Censor, Gordon, and Gordon 2001). The projection algorithms have been proved to converge for feasibility problems when the constraints are inequalities with a convex region (Censor and Zenios 1997). The component-averaging projection algorithms have also been proven to converge in cases that simultaneously include linear equality constraints and convexity (Censor, Gordon, and Gordon 2001). CC methods (Censor, Gordon, and Gordon 2001) are a variant of the simultaneous component-averaging gradient-projection algorithms that provide approximate solutions to feasibility problems which include nonlinear and non-convex constraints. The basic CC method is widely used in practice because it can reach a feasible region quickly, it is simple as it does not require any line search or matrix inversions, and the method has been shown to be robust, even in the presence of non-convex constraints in infeasibility. There are several variations of the CC methods, such as the direction-based maximum (DBmax) algorithm and the feasibility distance far (FDfar) algorithm. Empirical testing has shown the superiority of DBmax followed by FDfar (Ibrahim and Chinneck 2008). Note that the constraint satisfaction approaches that are applied to the constraint satisfaction problem (CSP) (Kumar 1992) are based on different principles, and the overlap between CSP and COP application domains is very small. These approaches have also been applied in conjunction with GA for solving CSPs (Brailsford, Potts, and Smith 1999).
In this research, the real-valued constrained optimization problems are considered, so a realvalued encoding is used to represent the variables. The main contribution of this research is the development of a new GA-based algorithm, for constrained optimization, that has the capability to assist the movement of infeasible individuals towards the feasible region, using a mix of three CC methods (the basic CC, DBmax and FDfar). The algorithm is named the constraint consensus memetic algorithm (CCMA). The sequential quadratic programming (SQP) technique has also been applied as the local search method in CCMA.
To analyse the contributions of the CC methods and the local search technique to CCMA, a study has been conducted using a GA in combination with these CC methods. To do this, all the variants of the algorithms start with a randomly generated population. Based on the level of constraint violations, the individuals are then divided into two groups (feasible and infeasible). Next, the CC methods are applied to some of the infeasible individuals until the stopping criteria are met. The resulting individuals are then merged with all of the other individuals and then the GA procedure is applied. For the variant of the mix of CC methods, all three methods are used in one algorithm framework by assigning one method to each individual, and the number of infeasible solutions that will be assigned to each method is adaptively updated. All algorithms have been analysed by solving 24 specialized test problems (Liang et al. 2005). CCMA has also been applied to solve another well-known set of test problems with 36 problem instances (Mallipeddi and Suganthan 2010b). The proposed CCMA has been compared with other state-of-the-art algorithms. The results show that, in most cases, the algorithm can achieve either better or equal solutions with significantly reduced computational time. The algorithm has further been applied for solving an economic load dispatch problem (ELDP) that is an important scheduling application for effective power system operation and management (Happ 1977;Chowdhury and Rahman 1990;Vlachogiannis and Lee 2009). The mathematical representation of this problem forms a complex nonlinear constrained optimization model. The experimental results from different instances of this problem show that the proposed algorithm is superior to other state-of-the-art algorithms.
This article is organized as follows. After the Introduction, Section 2 describes the CC methods used with GA in this article. Section 3 presents the GA with an overview of its parameters. Section 4 demonstrates the design of the proposed methods. The experimental results and the analysis of those results are presented in Section 5. In Section 6, the ELDP is described and its results are then analysed. Finally, the conclusions are given in Section 7.

Constraint consensus methods
A considerable number of projection algorithms has been proposed to solve feasibility problems with a set of convex constraints. However, the CC methods (Censor and Zenios 1997;Chinneck 2008;Ibrahim and Chinneck 2008) use an inexpensive heuristic for solving feasibility problems with nonlinear and non-convex constraints. Here, the main idea is to assist an infeasible point to move towards the feasible region, by making a consensus among the currently violated constraints, through calculation of the direction and distance that are required to achieve feasibility. The movements are updated repeatedly until a stopping condition is met (Ibrahim and Chinneck 2008).
The process of CC starts with the construction of a feasibility vector for each violated constraint at a selected infeasible point. The feasibility vector estimates the move from the current location to the closest feasible point for each violated constraint. The calculation of the feasibility vectors is exact for linear constraints, but it is a linear estimation (known as a linear feasibility vector) for nonlinear constraints. The linear feasibility vector moves the infeasible point in a parallel direction to the gradient of the violated constraint. The step size of the movement is determined by using a first order Taylor series expansion of the constraint function around the current point (Chinneck 2004). The linear feasibility vector is calculated according to the following equation for both linear and nonlinear constraints, in which this equation is derived from the first order Taylor series (Ibrahim and Chinneck 2008;Hamza et al. 2011): T is the transposition of its gradient, and ∇g i (x k ) T is its length; x k is the current infeasible point at the kth generation, and x k+1 is the estimated feasible point for each violated constraint at the (k + 1)th generation. The feasibility vectors for all of the violated constraints are combined into a consensus vector which is then used to update the current position by applying the proposed movement. The CC procedures are repeated until certain stopping conditions are met (Chinneck 2004), such as: (1) the length of every feasibility vector is less than the feasibility distance tolerance (α) (e.g. 10 −6 ); (2) the length of the consensus vector is less than the movement tolerance, β, which means that the consensus vector is too short to make good progress; or (3) the number of generations, μ, exceeds a predefined value. The details of these conditions can be found in Ibrahim and Chinneck (2008). An unsuccessful termination of the algorithm may happen as a result of the last two conditions, and those conditions are generated when the convergence is too slow. The algorithm may also exit unsuccessfully if the length of the consensus vectors across generations is not decreasing quickly enough (Jain and Elmaraghy 1997).
There are several variations of the CC method. The difference among these variations is the way in which the consensus vector is constructed. The basic CC treats all eligible feasibility vectors equally and the proposed movement is created by averaging the non-zero components of the feasibility vectors. DBmax decides the direction of the movement based on the size of the largest proposed movement in each of the positive or negative directions. Then, the largest movement is set as the consensus vector. FDfar sets the consensus vector based on the length of each feasibility vector, in which it selects the longest feasibility vector to be the consensus vector with the goal of generating the most rapid movement towards feasibility. Any components that do not appear in the selected longest feasibility vector are set by averaging, as in the basic CC scheme.
In the highly nonlinear constrained problems, the linear feasibility vector has two drawbacks: it is an approximation and the calculation of gradient is expensive. To overcome these problems, an augmentation technique was proposed as an alternative way to build the feasibility vector (Smith 2011). The calculation of the augmented feasibility vector uses a predictor-corrector approach, to make an adjustment to the length of the previous consensus vector (i.e. predictor) without any changes in the search direction. This calculation is like the Secant method for root finding (Tjalling 1995). The augmentation technique increases the speed and the success rate of the CC methods by increasing the accuracy of the feasibility vector for nonlinear constraints. This technique works as follows: (1) in the predictor step, the feasibility vector is calculated according to Equation (2), and the consensus vector is then constructed to determine the search direction; and (2) in the correction step, for every nonlinear constraint, an augmented feasibility vector (Smith 2011) is used instead of gradient. The step size is determined by using the values of the constraint functions at the current and previous generations. When using the augmentation technique, finding a point with a smaller constraint violation consumes fewer calculations in comparison to when using the linear feasibility vectors. The augmented feasibility vector is calculated according to the following equation: is the previous consensus vector, x k is the initial infeasible point at the kth iteration, x k+1 is the current point at the (k + 1)th iteration, and x k+2 is the new point at the (k + 2)th iteration. After calculating the augmented feasibility vector, the consensus vector is constructed using one of the CC methods. In this article, the augmentation technique is used with each CC method.
The weakness of the basic CC method is that the method of combining individual feasibility vectors can be fooled by special circumstances, especially when a set of feasibility vectors pulling in one direction is counterbalanced by another set pulling in the opposite direction, thereby resulting in a very short consensus vector which leads to a slower convergence (Ibrahim and Chinneck 2008). Also, the consensus vector may be too short if the violated constraints seem to be parallel, so the feasibility vectors will be almost opposites of each other. Moreover, the weakness of FDfar is that it may repair the most violated constraint by cycling between the constraints (Smith 2011). The effect of such problems is much less significant when CC is used with an evolutionary algorithm.

Genetic algorithm
In this article, a genetic algorithm with multi-parent crossover (GA-MPC) has been considered that outperforms all other existing evolutionary algorithms in solving benchmark COPs (Elsayed, Sarker, and Essam 2011a), as well as different real-world applications (Elsayed, Sarker, and Essam 2011b).
In GA-MPC, a population of individuals is randomly generated and the best k individuals are saved into an archive pool (based on their constrained violations and/or fitness functions). According to Elsayed, Sarker, and Essam (2011b), the value k = PS/2 (where PS is the population size) is used. A tournament selection then takes place and the selected vectors are saved in a selection pool. The crossover operator is applied to generate new offspring in which three parents are selected based on a selection rule and then the selected individuals are checked for duplication. However, if any duplication is found, one of them is replaced with a randomly selected individual from the selection pool. For the purpose of the crossover operation, the three selected individuals are ranked from the best (x 1 ) to the worst (x 3 ) based on their constrained violations and/or fitness functions. Three new offspring are then generated from three individuals as follows: The parameterź is a normally distributed random number with a mean value p and a standard deviation σ. After the three individuals are generated, to improve the diversity in the new population, a randomized operator is then applied on each of the generated offspring. This is done by choosing, with a given probability, one of the variables (it is assumed that there are n variables/dimensions) from the considered offspring, and replacing it with the corresponding one of a random individual from the archive pool. This step acts as a mutation process. Finally, the mutated offspring are merged with the individuals in the archive pool, by selecting the best group of individuals for the next generation.

A mix of constraint consensus methods with genetic algorithm
In this article, the above GA is combined with a mix of CC methods. The process for mixing CC methods is described below.

Genetic algorithm with multi-parent crossover with a mix of constraint consensus methods
In the proposed algorithm, an initial population with a size PS is generated. The PS individuals are then divided into two groups: the feasible individuals group and the infeasible individuals group. The infeasible solutions are then sorted, in ascending order, based on their constraint violation. The best P infeasible solutions, i.e. individuals with lower constraint violations, are divided into three groups (group i , i = 1, 2 and 3) of equal size (n i , i = 1, 2 and 3), in which the individuals in group 1 are better than the individuals in group group 2 and group 3 , and the individuals in group 2 are better than the individuals in group 3 . The basic CC method is applied to the individuals in group 1 , FDfar is applied to the individuals in group 2 and DBmax to the last group of individuals. Each method is processed for one generation and new solutions replace old ones. As DBmax has the ability to quickly move the distant individuals to the feasible region, it has been applied to the worst group of infeasible individuals (i.e. those far away from the feasible region). After each generation, and before applying genetic search operators, the size of each group is adaptively updated based on its success. The updating procedure is discussed in Section 4.2. The population is also updated by merging the new individuals with the remaining individuals (both feasible and infeasible). The genetic operators of GA-MPC are then applied, as described in Section 3. The selection process is discussed in Section 4.3. The algorithm is continued until the overall stopping criteria are met. The basic steps of the proposed algorithm (GA-MPC-M) are presented in Algorithm 1.

Algorithm 1. Steps of GA-MPC-M algorithm
Step 1: In generation t = 0, generate an initial random population (PS is the number of individuals in the population). The variables in each individual i are generated as follows: where L j , U j are the lower and upper bound for decision variable x zj and rand is a random number, rand ∈ [0, 1].
Step 2: Divide PS into two groups. The first group contains the feasible individuals, if any, and the second group contains infeasible individuals, if any.
Step 3: At each generation, if there are infeasible individuals, select the best P of them, and divide them into three subgroups (group i , i = 1, 2 and 3) of equal size (n i , i = 1, 2 and 3); else; go to Step 5.
Step 4: For each infeasible individual selected in Step 3, generate a random number (rand) between 0 and 1; if rand < 0.5, then apply the CC methods as follows: (a) basic CC to individuals in group 1 , (b) FDfar to individuals in group 2 , and (c) DBmax to individuals in group 3 .
Step 5: The new P individuals, obtained from Step 4, will replace the older ones, andthen the average improvement for each CC method is calculated (as described in Subsection 4.2) and the size n i is changed adaptively in each generation.
Step 6: Apply the genetic operators such as selection, crossover and mutation as of GA-MPC (as discussed in Section 3).
Step 7: Stop if the termination criterion is met; else set t = t + 1, and go to Step 3.

The success measure
As discussed earlier, three CC methods are considered in this article. Initially, they are applied to equal numbers of infeasible individuals, which means n 1 = n 2 = n 3 . However, these numbers may be changed depending on the success of the CC methods in the current generation. This means that the size of the subgroup can be increased for a well-performing CC method and similarly it can be decreased for a poorly performing CC method. After applying a CC method, if an individual has improved, a success measure is increased by 1. Using this concept, the size of the subgroups (n i ) can be revised as follows: Step 1: Set success S cci = 0, where cci = 1, 2 and 3 are for the CC methods basic CC, FDfar and DBmax, respectively.
Step 2: For each cci, y = 1 : n i (for all individuals in a subgroup n i ) -If the infeasible solution has been improved, then S cci = S cci + 1 Step 3: Calculate the proportion of total success for each cci: PropS cci = S cci /(S cc1 + S cc2 + S cc3 ), and then Step 4: n i = PropS cci /P, where P is the number of the infeasible individuals that should be handled by all the CC methods.

Selection process
In this article, the selection of the individuals (Deb 2000) is considered as follows: (1) between two feasible solutions, the fitter one (according to the fitness function) is better; (2) a feasible solution is always better than an infeasible one; and (3) between two infeasible solutions, the one having the smaller sum of its constraint violation is preferred. The equality constraints are also transformed to inequalities of the following form, where ε is a small value: where E is the number of equality constraints.

Experimental results
In this section, the computational results of the proposed algorithms are presented and analysed, using two sets of specialized benchmark problems: (1)  For the parameter settings, PS is set to 100 individuals, and the number of infeasible individuals that follow the CC methods, P, equals 10% of the infeasible solutions. For GA, the crossover rate is 100%, the probability of the randomized operator is 10% andź (shown in Equations 3-5) is a random Gaussian number with mean value equal to 0.7 and a standard deviation of 0.1. These parameter settings are the same as those used in Essam (2011b, 2011c). The parameters for the stopping criteria of the CC methods are α = 10 −6 , β = 0.0001 and μ = 10.

GA-MPC with a single constraint consensus method
In this subsection, the results of the GA-MPC algorithm are briefly discussed and they are compared with GA-MPC combined with a single CC method, namely the basic CC method (GA-MPC-CC), FDfar (GA-MPC-FDfar) and DBmax (GA-MPC-DBmax). In each version, the initial population is divided into two groups (feasible and infeasible). The CC method is then applied to the best P infeasible individuals. The new individuals are merged with the remaining individuals (not considered by the CC method), and the GA-MPC algorithm is then applied to all the individuals. The detailed results [best, median, mean, worst and standard deviation (St.d)] can be found in the supplementary document (Appendix A) available through the web link given at the end of this article. The results are based on 25 independent runs.
From the literature, it is known that there is no feasible solution for g20, while for g22 it is rare to find a feasible solution. Therefore, these two problems are excluded from the comparisons. With regard to the best values obtained, it is found that GA-MPC-CC, GA-MPC-FDfar and GA-MPC-DBmax are better than GA-MPC only for five, four and four test problems, respectively, To compare the above algorithms, statistical significance testing was performed. A nonparametric test, the Wilcoxon signed rank test (Corder and Foreman 2009) was chosen, which allows the difference between paired scores to be judged when the assumptions required by the paired-samples t test, such as that the population should be normally distributed, cannot be made. As a null hypothesis, it is assumed that there is no significant difference between the best and/or mean values of two samples, whereas the alternative hypothesis is that there is a significant difference in the best and/or mean fitness values of the two samples, with 5% significance level. The sign of the difference between the two independent samples is used to classify cases into one of two samples: differences below zero (negative rank R − ) or above zero (positive rank R + ); the corresponding p value is also calculated. Based on the test results/rankings, one of three signs (+, − or ≈) is assigned for the comparison of any two algorithms (shown in the last column), where the '+' sign means that the first algorithm is significantly better than the second, the '−' sign means that the first algorithm is significantly worse and the '≈' sign means that there is no significant difference between the two algorithms. The test statistics show that there is no significant difference between GA-MPC and each of the three variants discussed above.
As samples, a few convergence plots for these three variants for several problems are presented in Figure 1. These figures reveal that GA-MPC-FDfar is the best during the early generations, while GA-MPC-CC is better later on. Generally speaking, the integration of the CC methods with GA clearly results in an improvement.
During the evolution process, it is important to analyse the percentage of feasible solutions that are obtained by GA-MPC-CC, GA-MPC-FDfar and GA-MPC-DBmax in comparison to GA-MPC only. It is found that GA-MPC-CC increases the percentage of feasible solutions by 1.42%, while GA-MPC-FDfar and GA-MPC-DBmax increase the percentage of feasible solutions by 2.11% and 2.23%, respectively. It is clear that the inclusion of CC methods with GA increases the feasibility rate.

Genetic algorithm with multi-parent crossover with a mix of constraint consensus methods
The detailed experimental results are shown in the supplementary document (Appendix A). With regard to the best values obtained, it is found that the algorithm GA-MPC with a mix of CC methods (GA-MPC-M) was better than GA-MPC for five test problems and was able to obtain the same best solutions for 15 test problems. The algorithm was inferior to GA-MPC for only two test problems. Based on the average results, GA-MPC-M is superior to GA-MPC for seven problems and is equal for 14 test problems. The algorithm is inferior for one test problem only. Considering the Wilcoxon signed rank test, with regard to the best results, there is no significance between the two algorithms, whereas for the average results, GA-MPC-M is clearly superior to GA-MPC. Note that the algorithm GA-MPC-M increases the number of feasible solutions found during the search process by 3.43%, and it also requires 4.040% lower computational time than GA-MPC.

Constraint consensus memetic algorithm
In this section, the final form of the proposed algorithm is discussed. In this algorithm, a local search technique to GA-MPC-M has been applied for further improving the solution quality. Note that the integration of a local search with GA is recognized as being a memetic algorithm. This version of the algorithm is named CCMA. SQP (Powell 1978) is used as the local search technique in CCMA. The idea of SQP is to model a problem, at a given approximate solution, by a quadratic programming subproblem. The solution to this subproblem is then used to construct a better approximation (new point) (Boggs and Tolle 1995). As this method may be viewed as an extension of Newton and quasi-Newton methods to the constrained optimization setting, the SQP methods share many of the characteristics of Newton-like methods, such as that when the iterates are close to the solution, a rapid convergence can be achieved, and when the iterates are far from a solution, a possible eccentric behaviour, which needs to be carefully controlled, can occur (Boggs and Tolle 1995).
CCMA follows the same steps as GA-MPC-M (presented in Algorithm 1), with an additional step. In this additional step, one random individual is selected and the SQP technique is applied to it for up to a defined number of fitness evaluations (FEs) (here it is equal to 500 FEs). To reduce the chance of becoming trapped into a local optimum, SQP is applied periodically, i.e. once every 25 generations. This means that at every 25th generation, after completing Step 6, the local search technique will be executed.
Similarly to the previous few sections, CCMA was used to solve the CEC2006 competition problem set. The results of these problems are presented in the supplementary document (Appendix A). In Appendix A, it is found that CCMA is able to obtain the optimal solution for all CEC2006 test problems, except for one (g14) in which the difference is negligible (=3.0E-8), with a 100% feasibility ratio. If the above difference is considered as negligible, from the summary of comparisons presented in Table 2 In Figure 2, a few convergence patterns for the algorithms are shown. It can be seen that CCMA performs the best, in that it is able to reach the optimal solution very quickly. In particular, a slightly different pattern is shown in Figure 2(d), where although the convergence of CCMA in the early generations is slow, it can reach the optimal solution before any other algorithms.

Comparison to the state-of-the-art algorithms
In this section, the performance of CCMA is first compared, based on CEC2006 test problems, with other state-of-the-art algorithms, including self-adaptive multi-operator differential evolution (SAMODE) (Elsayed, Sarker, and Essam 2011d), ensemble constraint handling techniques based on evolutionary programming (ECHT-EP2) (Mallipeddi and Suganthan 2010a), modified differential evolution (MDE) (Mezura-Montes, Velazquez-Reyes, and Coello Coello 2006), adaptive penalty formulation based on genetic algorithm (APF-GA) (Tessema and Yen 2009), adaptive trade-off model with evolution strategy (ATMES) (Wang et al. 2008) and multimembered evolution strategy (SMES) (Mezura-Montes and Coello 2005). It must be mentioned here that CCMA used the same number of FEs (240 K) as SAMODE, ECHTEP-2, ATMES and SMES, while MDE and APF-GA used 500,000 FEs. The parameter ε was set to 1.0E-04 for all algorithms, except for ATMES and SMES, in which the parameter ε was set to 5.0E-06 and 4.0E-04, respectively. The detailed results for all the above algorithms are shown in the supplementary document (Appendix B), available through the web link given at the end of this article.  The comparisons for the quality of the solutions are summarized in Table 3. It must also be mentioned here that ATMES and SMES solved only the first 13 test problems; therefore, they were compared for those 13 problems only. Considering the best fitness, CCMA obtained either better or equal, but not worse, fitness values compared with SAMODE, ECHT-EP2, MDE, APF-GA, ATMES and SMES. In term of average fitness, CCMA obtained a higher number of better solutions than the others. Based on the statistical test, CCMA is better than APF-GA and SMES, while there is no significant difference between CCMA and SAMODE, ECHT-EP2, MDE and ATMES. Considering the average results, CCMA is superior to SMES, while there is no significant difference between CCMA and SAMODE, ECHT-EP2, MDE APF-GA and ATMES. It is now clear that the overall performance of CCMA is either better than or equal to SAMODE, ECHT-EP2, MDE, APF-GA, ATMES and SMES.

Solving the CEC2010 test problems
In this section, the algorithm was tested on another set of problems, known as the CEC2010 competition problems on constrained optimization. This set contains 18 test problems, each with two instances (one 10D and the other 30D). Each problem was run 25 times with the maximum number of FEs being equal to 200,000 and 600,000 for the 10D and 30D problems, respectively. The details of this problem set can be found in Mallipeddi and Suganthan (2010b). The proposed algorithm has been compared with the winning algorithm of the CEC2010 competition, namely ε DEag (Takahama and Sakai 2010). The detailed results are presented in the supplementary document (Appendix C), available through the web link given at the end of this article. With regard to the feasibility ratio, CCMA obtained a 100% feasibility ratio for both 10D and 30D, while ε DEag reached 100% and 95.11% for 10D and 30D, respectively.
Based on the quality of the solutions obtained, a comparison summary is presented in Table 4. From this table, for the 10D test problems, CCMA is better than ε DEag, with regard to the best values obtained, for five test problems, while it is worse for only one test problem. Considering the average results, for the 10D problems, CCMA is better than ε DEag for eight test problems, while it is worse for three problems. On the other hand, for the 30D test problems, CCMA is able to obtain better best solutions than ε DEag for 17 problems, while ε DEag is not better for any test problems. With regard to the average results, CCMA is superior to ε DEag for 15 problems, while it is worse for three problems.
Furthermore, based on the statistical test, CCMA is superior to ε DEag when considering both the best and average results obtained on the 30D test problems, while there is no significant difference in the 10D test problems.

Static economic load dispatch problems
The purpose of this section is to demonstrate that the proposed algorithm can effectively solve not only the benchmark problems, but also real-world complex constrained optimization problems. For this reason, a popular practical problem, which is known as the static ELDP, has been chosen.
The ELDP is an important issue for any power-generating system, which basically determines the commitment of each generating unit, either at each time interval of the planning horizon, or for a given period of time (Dahal, Aldridge, and Galloway 2007). The current problem involves only thermal power plants, where the aim is to minimize the total fuel cost while allocating power loads to different plants. The mathematical model of such problems contains both equality and inequality constraints. The objective function is usually derived from the relevant production cost equations, which can be approximated as a quadratic function of the active power outputs from the generating units (Chaturvedi, Pandit, and Srivastava 2008;Cheng-Chien 2008;Vlachogiannis and Lee 2009). It can be represented as follows: where f i (P i ) = a i P 2 i + b i P i + c i , i = 1, 2, . . . , N G is the expression for the cost function corresponding to the ith generating unit, and a i , b i and c i are the cost coefficients; P i is the real power output (MW) of the ith generator that corresponds to period t, and N G is the number of on-line generating units to be dispatched. The cost function for a unit with a valve point loading effect is calculated using: where e i and f i are the cost coefficients corresponding to the valve point loading effect. This problem is subject to different constraints depending upon the assumptions and practical implications, as shown below.
(1) Power balance constraints or demand constraints: These constraints are based on the principle of equilibrium between total system generation ( N G i=1 (P i )) and total system loads (P D ) and losses (P L ). That is, where P L is obtained using B-coefficients, given by (2) Generator constraints: The output power of each generating unit has a lower and an upper bound. This constraint is represented by a pair of inequality constraints: where P min i and P max i are the lower and upper bounds, respectively, for the power output of the ith generating unit.
(3) Ramp rate limits: It is commonly assumed that the adjustments of the power output are instantaneous. However, under practical circumstances, for all the on-line units, ramp rate limits restrict the operating range for adjusting the generator operation between two operating periods. Hence, generation may increase or decrease with corresponding upper and downward ramp rate limits. Thus, units are constrained by these ramp rate limits, such as: P i − P t−1 i ≤ UR i , if power generation increases; and P t−1 i − P i ≤ DR i if power generation decreases, where P t−1 i is the power generation of unit i in the previous period, and UR i and DR i are the upper and lower ramp rate limits, respectively. The inclusion of ramp rate limits modifies the generator operation constraints as follows: (4) Prohibited operating zone: The generating units may have certain ranges where the operation is restricted on the grounds of the physical limitations of machine components or instability, e.g. due to steam valves or vibration in shaft bearings. Consequently, discontinuities are produced in the cost curves that correspond to the prohibited operating zones. Hence, there is a desire to avoid operations in these zones in order to economize production. Symbolically, for a generating unit i, where P pz andP pz are the lower and upper limits of a given prohibited zone for unit i. In this research, five cases of the ELDP are considered from the literature (Chaturvedi, Pandit, and Srivastava 2008;Cheng-Chien 2008

Computational study
The proposed algorithm (CCMA) has been applied to solve the above five cases with the same parameters used in earlier sections. The results obtained were compared with two other state-ofthe-art algorithms: differential evolution with multiple strategies (SAMODE) (Elsayed, Sarker, and Essam 2011a) and adaptive differential evolution algorithm (IDE) (Asafuddoula, Ray, and Sarker 2011). All algorithms used 150,000 FEs. The detailed results are shown in Table 5. From Table 5, it is clear that the best fitness values of CCMA are better than the same values for SAMODE and IDE for four and five cases, respectively, while this value is equal to SAMODE for only one case. Based on the average results, CCMA is better than SAMODE and IDE for four and five cases, respectively, but it is inferior to SAMODE for only one case. With regard to the standard deviation, CCMA is better than SAMODE and IDE for three and five cases, respectively, but it is inferior to SAMODE for two cases. As shown in Table 6, considering the Wilcoxon signed rank test with the 10% significance level, CCMA is superior to both the algorithms with regard to the best and average solutions.

Conclusion and future work
During the past few decades, many evolutionary algorithms have been introduced to solve constrained optimization problems. In those algorithms, reaching feasibility or near feasibility quickly is not an easy task, especially when the feasible region is tiny. In this research, different GA variants are introduced that are able to reach the feasible region quickly. They are: (1) the basic CC method with a genetic algorithm with multi-parent crossover (GA-MPC); (2) the DBmax method with GA-MPC; (3) the FDfar method with GA-MPC; and (4) an ensemble of the basic CC, DBmax and FDfar methods with GA-MPC.
In the ensemble version, which is known as GA-MPC-M, each CC method (the basic CC, DBmax and FDfar) is assigned adaptively to certain individuals. With regard to both the quality of solutions and the computational time, this algorithm was clearly better than the GA-MPC algorithm. The GA-MPC-M algorithm was further improved by adding a local search (SQP), which was called CCMA. CCMA was found to be superior to GA-MPC-CC, GA-MPC-DBmax, GA-MPC-FDfar, GA-MPC-M and GA-MPC in terms of both the quality of results and the computational time required. CCMA was found to be competitive, if not better, in comparison to SAMODE, ECHT-EP2, MDE, APF-GA, ATMES and SMES. In addition to this, CCMA was tested on the CEC2010 constrained problems, with both 10 and 30 dimensions. The results showed that it was better than the best algorithm in the CEC2010 competition.
CCMA has also been applied to solve a real-world complex constrained optimization problem. From the comparison of results, CCMA clearly dominated two other well-known algorithms. The statistical testing supports this finding.
For future work, it is intended that CCMA will be applied to other optimization problems, different parameters will be analysed and further attempts will be made to improve the performance of the algorithm.