Search Manager : A Framework for Hybridizing Different Search Strategies

In the last decade, many of the metaheuristic search methods have been proposed for solving tough optimization problems. Each of these algorithms uses its own learn-by-example mechanism in terms of “movement strategy” to evolve the candidate solutions. In this paper, a framework, called Search Manager, is proposed for hybridizing different learn-byexample methods in one algorithm, which is inspired by the organizational management system in which managers change their management method by viewing performance reduction in their managerial organization. The proposed framework is verified using standard benchmark functions and real-world optimization problems. Further, it is compared with some wellknown heuristic search methods. The obtained results indicate not only the optimization capability of the proposed framework, but also its ability to obtain accurate solutions and to achieve higher convergence precision. Keywords—Global optimization; metaheuristic; organization management; hybridizing search methods


INTRODUCTION
In different areas of science such as industry, engineering, and management there are many complex problems, also known as optimization problems, such that there is no exact algorithm to solve them in polynomial time.Due to this fact, to find a relatively optimal solution for these kinds of problems, in the past decades, a significant number of different optimization algorithms have been introduced by researchers.
Optimization algorithms can be divided into two broad groups, deterministic and stochastic.Deterministic algorithms such as the Nelder-Mead search method [1], the tunnelling method [2], and renormalization group methods [3] perform based on gradients and second-order derivatives.A remarkable advantage of deterministic optimization methods is the fast convergence; however, for high dimensional and multimodal functions they may fall into a local optimum.In stochastic algorithms, although the quality of the obtained solution cannot be guaranteed, they are more efficient and flexible than deterministic approaches.Other advantages are the capability of escaping from a local optimum, good performance, and ease of implementation.
Stochastic optimization algorithms generally are population-based algorithms that begin with a set of randomly generated candidate solutions and by applying some specified rules iteratively, gradually evolve initial solutions.The rules are usually inspired by the behaviors of biological and physical systems in nature, culture, society or politics.Based on the source of inspiration, they can be classified into three main groups: (1) Evolution-based, (2) Swarm-based, and (3) Humanbased algorithms.
Evolution-based algorithms mimic natural biological evolution and selection.The Genetic Algorithm (GA) has been the most popular in this class of optimization algorithms.It uses the Darwinian theory of natural selection, crossover, and mutation [4].Other algorithms that can be classified in this group are Evolution Strategy (ES) [5], Genetic Programming (GP) [6], Differential Evolution (DE) [7], Evolutionary Programming (EP) [8], and Biogeography-Based Optimizer (BBO) [9].Swarm-based algorithms get their inspiration from the collaborative conduct of a group of animals, such as ant colonies, honey bees, and bird flocks.They are typically made up of a population of agents (swarm individuals) interacting together to fulfill the main goal of the system.To date, several swarm based optimization algorithms have been proposed in the literature.Particle Swarm Optimization (PSO) is the most popular and significant algorithm that mimics the behavior of a flock of migrating birds heading for an unknown destination [10].So many variants of PSO algorithm such as MLPSO [11], FST-PSO [12], etc. with the aim of improving its performance in different types of problems have been presented so far.Other examples of this class of algorithms are: Ant Colony Optimization [13], Artificial Bee Colony [14], Bacterial Foraging [15], Cat Swarm Optimization [16], Elephant Herding Optimization (EHO) [17], Bat Algorithm (BA) [18], etc.
There are some extra optimization algorithms that cannot be assigned to the above three groups.They imitate some additional rules that exist in the world and universe.For www.ijacsa.thesai.orginstance, Optics Inspired Optimization (OIO) uses the law of reflection [29] or Ray Optimization (RO) was developed based on Snell"s light refraction law [30].Some of the other algorithms are: Water Wave Optimization (WWO) [31], Big-Bang Big-Crunch (BBBC) [32], Charged System Search (CSS) [33], Artificial Chemical Reaction Optimization Algorithm (ACROA) [34], Curved Space Optimization (CSO) [35], Central Force Optimization (CFO) [36].However, we think this kind of optimization algorithms is more complex for understanding than the specified three groups of algorithms.
Regardless of the inspiration"s source, still no universal optimization algorithm has emerged; nor does it appear likely that one ever will [37] and the No Free Lunch Theorem of Optimization (NFLT) supports this view [38].Besides, although many optimization methods have been proposed so far, one or another method presents a better solution on a specific problem.Therefore, finding more efficient algorithms is still in progress and we will see further optimization algorithms with the development of human identity from nature.However, a powerful optimization algorithm can solve many problems.
Considering the aforementioned facts, in this paper we intend to propose a framework, Search Manager, for combining different search abilities of different optimization algorithms in one algorithm to effectively solve the optimization tasks.The proposed framework accomplishes this by imitating the changing management style that is applied by managers in real-world organizations.In any organization when managers cannot obtain better results, they have to change their management style to improve the performance of the organization.
The simplicity and capability of using variable movement strategies are powerful aspects of the proposed framework.
Although in this study we use four simple movement strategies, the others can be proposed or obtained from the other optimization algorithms to get better performance for different optimization problems.
The remainder of this paper is organized as follows: Section II gives a general view of the main behavior of the proposed algorithm.Section III reviews some of the main principles about organizations and their management process.In Section IV, the proposed framework is introduced.Section V presents experimental investigation of the proposed method and its comparative study with other metaheuristic algorithms.Finally, some concluding remarks and future works are presented in Section VI.

II. MOTIVATION
Almost all of the previous population-based optimization methods iteratively evolve and optimize a population of individuals (candidate solutions) according to some criteria to reach a final population which has a near-optimal solution to a problem.In fact, they use learn-by-example mechanisms in terms of "movements" to evolve the candidate solutions, but the main problem of these methods is the lack of alternative movement strategies for different situations.The proposed framework overcomes the aforementioned limitation by combining different movement strategies.If the current strategy does not improve solutions within the population, another one will be selected.For example, consider the situation in Fig. 1 that represents the search space of an objective function, in which there exist a local and global minimum at inflection points.In this situation, the population of six individuals has got stuck in the local minimum.If there is not enough diversity in the population like this situation, an optimization algorithm would get trapped in the local optimums due to the lack of an alternative strategy.In the proposed framework, it is more likely that the population with not enough diversity could escape from local minimum by using different movement strategies.

III. ORGANIZATION MANAGEMENT BACKGROUND
Firm infrastructure, considered as organizations" structure and its management process, is a branch of social science, which includes a lot of relative concepts.However, in this section, we only review a few important concepts that are used in the proposed framework.
In the society around us, there are various organizations such as hospitals, schools, social institutions, etc. to achieve certain objectives in social life.In the words of Leavitt, an organization is "a particular pattern of structure, people, tasks and techniques" [39].In the view of Katz and Kahn, an organization is "a system which is composed of a set of subsystems" [40], put the other way round, an organization is a unified system, whose functionality is divided into different key subsystems.These subsystems are all parts of an organization working together for a common purpose.Each organization in the society needs a system of management.Management is defined in different ways by different people.It may be viewed as a process of leading and controlling the activities of organization employees [41], or as a decision-making process in which the manager is a decisionmaker that leads the organization to achieve the objectives [42].In all of the organizations, employees look up to managers for guidance, thus for making appropriate management decisions and guiding organization members, managers adopt a management style or a form of leadership, primarily by their abilities, personalities, and values [43].However, half of the decisions made by managers within organizations fail [44].Failure in decisions signals the need for new decisions or changing the management style that decisions were taken under it.Failed decisions can be a source of learning and they can help the manager to take effective and operational decisions in the future [45].Management style is a form of leadership that specifies how employees should follow organization"s policy.There are many types of management styles.Managers move in and out of these various styles as the need arises.
Management structure is another aspect of the management process.It is the manner in which the management of a company or organization is organized.It determines the scope and nature of how leadership is disseminated throughout the organization [42], [43].Organizations commonly adapt either a flat or hierarchical structure.In the flat structure, there are a few or no levels of middle management between top managers and employees.But in the hierarchical structure, there are a number of hierarchical levels between top managers and employees.Fig. 2 shows these structures.The proposed framework uses the aforementioned concepts for combining different learn-by-example methods in one algorithm for solving optimization problems.

IV. PROPOSED METHOD
In this section, according to the basic principles of organization management outlined in the previous section, a framework is proposed for combining different search methods.The core of the Search Manager is based on a basic principle in the management of today"s organizations: "managers need to adopt a new management style when organization"s performance goal gets worse".The framework consists of three steps as follows: Step 2 until the stop criterion is satisfied.Fig. 4 shows the flowchart of the Search Manager and details of its steps are represented in the following subsections.

A. Step 1: Initialization
The goal of the optimization task is to select n decision variables x 1 ‚ x 2 ‚…‚x n (known as candidate solution) from a feasible region in such a way as to optimize a given objective function.The values of the decision variables are represented as floating point numbers and the cost of a candidate solution is obtained by evaluating the objective function f at these variables like the following equation.To start the optimization algorithm, a population of size N pop is generated, and then the best individual is selected.The remainder of solutions is divided among m distinct groups.For example, Fig. 3 shows four distinct groups of population, in which the best individual from the population is specified.
Each group will have current and previous average cost.The average cost of a group is defined as Where GC n is the group cost of the nth group of candidate solutions and c i is the cost of the ith solution.Furthermore, each group has an t-dimensional style vector, where t is the number of movement styles.The initial values of these vector components are set to 1 and each component holds the score value for the specified movement style.More explanations on this vector are presented in the following subsection.

B. Step 2: Movement
In the movement step, candidate solutions are evolved for the next generation using movement styles.The framework uses four simple movement styles inspired from other optimization algorithms and their mathematical formulas are represented in the following subsections.
As mentioned in subsection 4-A, each group of the candidate solutions has a vector that holds score values for different movement styles.For each group, a style (movement method) is selected by using the values of style vector and the roulette wheel selection method.The selection probability is calculated as (1).
Where, P i is the probability of being selected ith style from the t styles.It is obvious that the style with a higher score is more probable for selection than the others.www.ijacsa.thesai.orgAt the beginning of the movement step, for each group one of the styles is selected randomly, then all candidate solutions in the group move toward a position calculated by using the selected movement style, and then finally, the group cost (GC n ) is computed and compared with the previous group cost, if an improvement is seen, the selected style is rewarded by increasing its score value in the style vector.Otherwise, the selected style is penalized by dividing its score value by a random number generated between 2 and the number of population.At the same time, the score values of the other styles are increased by a random number between 0 and 1.For example, Fig. 5 shows a sample of style vector for a group.Style3 is more probable to be selected for the next iteration.If Style3 is selected and the group cost is improved, the values of the vector will be changed like Fig. 6.It is observable that the score value of Style3 has been increased.Now, based on the values in Fig. 5, if Style3 does not improve the group cost or make it worse, the values of the vector style will change like Fig. 7.It can be seen that Style3 has been decreased and the others have been increased.
The used movement styles are inspired from other optimization algorithms and their equations are formulated as follows: 1) The First Movement Style By the first movement style, solutions accept some of the variable values from the best individual.This style is applied by using (2).
Where, X i is a selected solution from a group, and Φ is a set of randomly chosen decision variables.

2) The Second Movement Style
In the second movement style, solutions in the group are updated as www.ijacsa.thesai.org Where, {rand} is an n-dimensional random vector and its values are in [0, 1].X r corresponds to the randomly selected solution from the group and X i old corresponds to the ith solution in the group.

3) The Third Movement Style
In the third movement style, solutions in the group are updated as Where, V is the variance of solutions in the group.

4) The Fourth Movement Style
The fourth movement style uses (5) for updating solutions in the group.Some of the selected variables of the best individual are changed in a temporary vector and then the other solutions in the group move towards this vector.

 
Where, {rand} is a generated random vector from the search space.
After applying movement operation, the best individual is selected from the population.

V. COMPARATIVE STUDY
In this section, the ability of the Search Manager is assessed from two perspectives: one is by applying it to optimization of 44 benchmark functions; another is through 15 real-world problems.
In the first perspective, Search Manager is applied on a wide range of nonlinear benchmark functions, 14 functions from the CEC 2005 1 [46] and all of the 30 functions from the CEC 2014 [47].These benchmark suites include a diverse set of problem features such as unimodality, multimodality, separability, non-separability, rotation, scalability, etc. for single objective optimization.They are based on classical benchmark functions, such as Rosenbrock"s, Rastrigin"s, Swefel"s, Griewank"s, and Ackley"s function.It should be noted although these test functions are organized by the community of heuristic algorithms in the framework of a workshop, in this study they are used as standard test functions for comparison.The total functions in CEC 2005 and CEC 2014 can be divided into some groups according to their 1 The first 14 functions are selected from CEC 2005 benchmark set.characteristics.Table I shows these groups.The other properties of these functions have been defined properly in their corresponding references, and hence they are not repeated here.The experimental results on these benchmark functions may reveal how Search Manager performs on various functions as well as can be compared to those obtained by other algorithms.
In the second perspective, Search Manager is applied to 15 real-world problems, which are derived from CEC 2011 [48].
To evaluate the performance of the proposed method, obtained results using Search Manager is compared with the result of some well-known and recently proposed social and nature-inspired optimization algorithms from the computational viewpoint.The selected algorithms are listed below.
 CA: Cultural Algorithm models social evolution and learning in agent-based societies [22].
 ABC: Artificial Bee Colony inspired by foraging behavior of honey bee swarm [49].
 GSA: Gravitational Search Algorithm uses the law of gravity and the notion of mass interactions based on the laws of gravity and motion [50].
 FOA: Forest Optimization Algorithm inspired by few trees in the forests which can survive for several decades, while other trees could live for a limited period [51].
 WOA: Whale Optimization Algorithm mimics the social behavior of humpback whales.The algorithm is inspired by the bubble-net hunting [52].
 DE: Differential Evolution is a type of standard genetic algorithm [7].
Computation code of all above-mentioned algorithms is taken from web pages dedicated to these algorithms.The MATLAB code for all algorithms is available at: https://data.mendeley.com/datasets/f5jvxbw8xb/1(DOI: 10.17632/f5jvxbw8xb.1)

A. Experimental Settings
The experimental environment is a computer of Intel Core i3, 4GB DDR2 memory, and Windows 7 operating system.Tests are performed in 10-, 30-, and 50-dimensional for each test function in the first perspective.All algorithms are executed 30 times for each problem with a total of D*10 4 and 5*10 4 evaluations of the objective function (Max_FES) in the first and second perspective respectively, where D is the dimension of the problem, and the obtained average results are compared with the other algorithms.The population size P = 50 is used for all algorithms.Table II shows the other recommended parameter settings for each algorithm.The values of these parameters are selected based on the recommendation from their original papers or previous related works.Additionally, initial candidate solutions are randomly calculated by uniform distribution between lower and upper limits of benchmark functions.www.ijacsa.thesai.orgIt should be noted that we know better results can be achieved from these algorithms by fine-tuning their control parameters.However, finding the perfect parameter for each problem is expected to be a very time-consuming task.Therefore, fixed parameter settings are adopted for each algorithm and this condition is equal for all algorithms.

B. Results and Discussion on Perspective 1
Experimental results of all algorithms on 14 functions of CEC 2005 and 30 functions of CEC 2014 are presented in Tables III to V and Tables VI to VIII, respectively.In all tables, the mean and standard deviation of 30 runs of the algorithms for each function are adopted to assess the optimization performance of the proposed algorithm.Additionally, a statistical test called Wilcoxon rank-sum test [55], which is a nonparametric statistic test for the independent samples, is conducted on the experimental results at the 5% significance level to judge whether the obtained results from the proposed method are significantly different from the other algorithms and have not occurred by chance [56].The cases are marked with "+/ ≈ /−" when the performance of Search Manager is significantly better than, equal to, and worse than the other test algorithms.For clarity, the best results are marked in bold.Tables IX and X show obtained p-values in the statistical test between the proposed algorithm and each of the remaining algorithms over benchmark functions in 10 and 50 dimensions.Moreover, Fig. 8 and 9 represent the convergence rate of some selected unimodal and multimodal functions for all of the algorithms.
Based on the presented results of CEC 2005 test suite in Tabels III to V, Search Manager provided better results over 10 dimensions.Although its performance decreases in 30 and 50 dimensions, it can be seen that its rank is better than the other algorithms overall dimensions.In addition, the proposed algorithm displayed the best performance over expanded functions in all dimensions.
For CEC 2014 test suite, the experimental results in Tables VI to VIII show that the Search Manager presented good results in terms of average rank, but the DE algorithm outperforms Search Manager in most of the functions over 10 and 30 dimensions.Furthermore, Search Manager could not produce successful results on composition functions as well as unimodal functions as compared to the ABC and DE algorithms.In addition, the GSA showed the best results in 30 and 50 dimensions of hybrid functions.However, Search Manager may get better results by using other types of movement styles.
Based on the convergence curves (Fig. 8 and 9), it can be seen that Search Manager has a good convergence rate and it can reach an optimum solution with less calculation.
To summarize, although Search Manager outperforms most of the compared algorithms in terms of average rank in all benchmark function sets, its performance is lower than the DE and ABC in some complex functions.However, the powerful aspect of the Search Manager is its ability to accept different movement strategies, and hence it may achieve better results for these functions by using other combinations of learn-byexample methods as movement styles.In other words, instead of adjusting parameters to get better results that we have seen in previous optimization algorithms, we can adjust the combination of different learn-by-example methods in Search Manager.

C. Results and Discussion on Perspective 2
Herein, the performance of Search Manager is evaluated through 15 real-world optimization problems.Detailed definitions of these problems can be found in [48].
Considering the experimental results represented in Table XI, Search Manager outperforms all of the compared algorithms in most of the real-world problems.In more details, Search Manager outperforms ABC, FOA, GSA, WOA, CA, and DE on 9, 11, 11, 9, 10, 9, 7 problems respectively.On the contrary, ABC, FOA, GSA, WOA, CA, and DE are better than Search Manager on 2, 2, 2, 2, 2, 1, and 1 problems, respectively.All the compared algorithms obtained similar results in solving the T03BCB and T08TNEP.Therefore, based on the obtained results, it is observable that Search Manager performs better in most of the real-world problems when compared with ABC, FOA, GSA, WOA, CA, and DE.
The conclusion that can be drawn from experiments on real-world problems is that the combination of the used learnby-example equations as movement styles in Section IV is most suitable for this kind of optimization problems.

VI. CONCLUSION AND FUTURE WORKS
In this paper, we have proposed a framework, called Search Manager that imitates an important part of the management process, which is concerned by managers in any organization.The heart of the Search Manager is applying another movement strategy when the average cost of candidate solutions gets worse.Search Manager has one main operator: Movement.The movement operator implements different learn-by-examples methods as movement styles.Extensive analysis is carried out to reveal the ability of Search Manager www.ijacsa.thesai.org to find the best fitness value over the search space not only in the benchmark functions, but also in the real optimization problems as well.Of course, it should be noted that like the other optimization algorithms, Search Manager is suitable for some sort of problems.However, its important property is that it provides a framework for combining different search methods, so it can be made compatible with any optimization problem by modifying its movement styles.
In the basic Search Manager, we have used four movement strategies as movement styles, but more or other types of movement methods can be formulated to improve its performance.
Search Manager can use the learn-by-example strategies of other optimization algorithms.Therefore, it is capable of combining these strategies in one algorithm and in utilizing their search abilities.From this capability, a possible line for future work would be investigating the behavior of the Search Manager with the combination of different learn-by-example strategies.
Other possible avenues of future research on the Search Manager include: using other mechanisms for rating movement styles because the performance of the proposed framework is highly dependent on this mechanism, applying a mechanism for moving solutions between groups, and finally, management science and its application in organizations have a lot of concepts in the real-world, such as how to fire and hire employees, employee promotion mechanisms, etc. that can be brought into the framework via simulation."+", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm.www.ijacsa.thesai.org

Fig. 3 .
Fig. 3.A sample of population structure in Search Manager.    12 , , , n Cost = f Candidate solution = f x x x 

Fig. 5 . 3 Fig. 6 .
Fig. 5.A sample of style vector for holding scores of movement styles.Style 1 Style 2 Style 3 8 5 25.3 Fig. 6.A sample of style vector after punishing and rewarding styles.

Fig. 7 .
Fig. 7.A sample of style vector after punishing and rewarding styles.

TABLE III .
MINIMIZATION RESULTS FOR 14 FUNCTIONS FROM CEC 2005 OVER 30 RUNS AT 10 DIMENSIONS +", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm. "

TABLE IV .
MINIMIZATION RESULTS FOR 14 FUNCTIONS FROM CEC 2005 OVER 30 RUNS AT 30 DIMENSIONS "+", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm.

TABLE V .
MINIMIZATION RESULTS FOR 14 FUNCTIONS FROM CEC 2005 OVER 30 RUNS AT 50 DIMENSIONS +", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm. "

TABLE VI .
MINIMIZATION RESULTS FOR 30 FUNCTIONS FROM CEC 2014 OVER 30 RUNS AT 10 DIMENSIONS "+", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm.

TABLE VII .
MINIMIZATION RESULTS FOR 30 FUNCTIONS FROM CEC 2014 OVER 30 RUNS AT 30 DIMENSIONS +", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm. "

TABLE VIII .
MINIMIZATION RESULTS FOR 30 FUNCTIONS FROM CEC 2014 OVER 30 RUNS AT 50 DIMENSIONS "+", "", "" respectively denote that the performance of Search Manager is better than, similar to, and worse than the corresponding algorithm.

TABLE IX .
P-VALUES OBTAINED WITH WILCOXON"S RANK SUM TEST OVER 14 FUNCTIONS FROM CEC 2005 BENCHMARK IN 10D

TABLE XI .
MINIMIZATION RESULTS FOR REAL-WORLD PROBLEMS