A New Optimization Algorithm For Combinatorial Problems

Combinatorial optimization problems are those problems that have a finite set of possible solutions. The best way to solve a combinatorial optimization problem is to check all the feasible solutions in the search space. However, checking all the feasible solutions is not always possible, especially when the search space is large. Thus, many meta-heuristic algorithms have been devised and modified to solve these problems. The meta-heuristic approaches are not guaranteed to find the optimal solution since they evaluate only a subset of the feasible solutions, but they try to explore different areas in the search space in a smart way to get a near-optimal solution in less cost and time. In this paper, we propose a new meta-heuristic algorithm that can be used for solving combinatorial optimization problems. The method introduced in this paper is named the Global Neighborhood Algorithm (GNA). The algorithm is principally based on a balance between both the global and local search. A set of random solutions are first generated from the global search space, and then the best solution will give the optimal value. After that, the algorithm will iterate, and in each iteration there will be two sets of generated solutions; one from the global search space and the other set of solutions will be generated from the neighborhood of the best solution. Throughout the paper, the algorithm will be delineated with examples. In the final phase of the research, the results of GNA will be discussed and compared with the results of Genetic Algorithm (GA) as an example of another optimization method.


I. INTRODUCTION
Many optimization problems have been encountered in different domains of manufacturing and industry.Usually the optimization problem that needs to be solved is first formulated and all the constraints are given.The optimization problem mainly consists of an objective function and a set of constraints.The objective function can be in mathematical form or combinatorial form.Once the objective function of the optimization problem is formulated and all the constraints are defined, then the main issue is to solve this problem.
The solution is usually the best values of the variables or the best scenarios which can also be called the optimal solution.This optimal solution should give us the best performance or best fitness in terms of the objective function.
In most optimization problems there is more than one local solution.Therefore, it becomes very important to choose a good optimization method that will not be greedy and look only in the neighborhood of the best solution; because this will mislead the search process and leave it stuck at a local solution.However, the optimization algorithm should have a mechanism to balance between local and global search.An example of a two-dimensional function that has more than one local and global solution is shown in Fig. 1 [1].
There are multiple methods used to solve optimization problems of both the mathematical and combinatorial types.In fact, if the optimization problem is simple or if the search space is small, then the optimization problem can be solved using conventional analytical or numerical procedures.However, if the optimization problem is difficult or if the search space is large, it will become difficult to solve the optimization problem by using conventional mathematics or using numerical induction techniques.For this reason, many meta-heuristic optimization methods have been developed to solve such difficult optimization problems.These include Genetic algorithm (GA), simulated annealing (SA), ant colony algorithm (ACA), and particle swarm (PS).Most of these meta-heuristic optimization problems are inspired by nature, biology, or environment.
The term meta-heuristic refers to a specific class of heuristic methods.Fred Glover first used this term and defined it as follows, "A meta-heuristic refers to a master strategy that guides and modifies other heuristics to produce solutions beyond those that are normally generated in a quest for local optimality.
The heuristics guided by such a meta-strategy may be high level procedures or may embody nothing more than a description of available moves for transforming one solution into another, together with an associated evaluation rule."[2].
The meta-heuristic algorithms do not always guarantee an optimal solution.However, in most cases a near optimal solution can be obtained in much less time than the computational methods [3][4].www.ijarai.thesai.orgThe meta-heuristic algorithms can be classified based into different categories [5][6]:

1) Constructive and Improvement:
A constructive heuristic (also known as a greedy approach) usually constructs a solution from scratch based on a certain criteria.Some of the common constructive heuristics are nearest neighbor, multiple fragment and insertion heuristics [7].An improvement or neighborhood search, which is usually known as a local search, attempts to improve the solution by exploring the neighborhood of the current solution [8].The neighborhood of a solution is the set of solutions that are close to the current solution.The local optimal solution is the best solution in each neighborhood and the global optimum is the best solution with respect to the whole search space.An improvement or local search begins the search from a given solution, and then iteratively attempts to improve the solution quality by using move operators, the move operator is usually determined based on the neighborhood structure, and it aims to change (move) the solution to a newer solution in the same neighborhood but with a better fitness.

2) Single Solution and Population based approaches:
In the single based solution, a unique solution is first generated and then based on a certain move criteria, other solutions are generated.Some of the meta-heuristic methods that start with a single solution are: Tabu Search (TS) and Simulated Annealing (SA).Population based algorithms on the other hand start by generating a set of multiple initial solutions.Examples of those methods would be Genetic Algorithm (GA) and Ant Colony Algorithm (ACA).
The computational drawbacks of mathematical techniques and methods (i.e., complex derivatives, sensitivity to initial values, and the large amount of enumeration memory required) have forced researchers to rely on meta-heuristic algorithms based on simulations and some degree of randomness to solve optimization problems [9].Although, these meta-heuristic approaches are not very accurate and they do not always give the optimal solution, in most cases they give a near optimal solution with less effort and time than the mathematical methods [10].
The meta-heuristic algorithms are general purpose stochastic search methods simulating natural selection and biological or natural evolution [11].Different meta-heuristic algorithms have been developed in the last few decades simulating and emulating different processes.Some of these meta-heuristic algorithms were inspired by the biological evolutionary processes; such as the evolutionary strategy (ES) [12], evolutionary programing [13][14][15], and the genetic algorithm (GA) proposed by Holland [16][17].Some meta-heuristic algorithms emulate different animal behaviors; like the tabu search (TS) proposed by Glover [18], the ant colony algorithm (ACA) by Dorigo et al [19], Particle Swarm Optimization (PSO) [20], Harmony Search (HS) algorithm [21], Bee Colony Optimization (BCO) [22].Other meta-heuristic algorithms were inspired by different physical and natural phenomena like the simulated annealing (SA) [23], and the Gravitational Search Algorithm (GSA) [24].
The distribution of publications which applied the metaheuristics methods to solve the optimization problem in the past decade is shown in Fig. 2. [25].In this paper we introduce a new optimization algorithm that can be applied to combinatorial problems.The new optimization problem is named Global Neighborhood Algorithm (GNA), and it is a population based and derivative free algorithm like other evolutionary optimization algorithms including Genetic Algorithm (GA), Ant Colony (ACA) and Evolutionary Strategy (ES).A set of randomly generated solutions from the entire search space are first generated and then the best of these solutions is chosen.After that, the algorithm will iterate, and in each iteration there will be two sets of generated solutions; one from the global search space and the other set of solutions will be generated from the neighborhood of the best solution.This paper starts with a background about optimization problems, then the methodology of the GNA algorithm is explained, and after that results for using this algorithm to solve the well-known Traveling Salesman(TSP) problem are also discussed.

II. METHOLODOLGY
The algorithm proposed in this paper is used to optimize combinatorial problems.The combinatorial problems could have more than one local and global optimal value within the www.ijarai.thesai.orgsearch space values.The proposed methodology will work to find the optimal value among these local optima by switching between exploration and exploitation.Exploration allows for exploring the whole search space.Exploitation allows focusing the search in the neighborhood of the best solution of generated solutions.
In order to explain the methodology of the GNA algorithm, assume we have discrete function that we need to optimize and let us say that we need to minimize this function (without loss of generality).
So the objective function we have is: Where: , are the different combinations of the solution sequence; we can think of these combinations as the city sequence in the TSP problem.
We need to find the optimal combination or solution that will give the optimal (minimum) value for the objective function ( .In general, if each of the variables can be chosen in ways respectively, then if we want to enumerate all the possible solutions this will yield solutions.However, this process could take several hours or days depending on the size of the problem.Thus, using a metaheuristic approach is better even if does not always give the optimal solution, but in most cases it will give a solution that is close to the optimal solution with less computational power.
According to the GNA algorithm, a set of ( random solutions are first randomly generated from the set of all possible solution, where: can be chosen in ways.
The generated solutions will then look like: where The fitness for the above solution will be evaluated and this can be done by substituting them in the objective function ( .The solutions are then sorted according to their fitness obtained from the objective function: is the solution sequence with best fitness.
The best combination ( is then used as a good measure for the local optimal solution and it is also initially set as the best known solution.
In the next iteration, 50% of the ( generated solutions will be generated near the best solution neighborhood by using a suitable move operator.
The other 50% of the ( generated solutions will be still generated from the whole search space, and the reason for that is to allow for the exploration of the search space, because if we just choose the solutions close to the best solution we will only be able to find the local solution around this point, and since the function that need to be optimized could have more than one local optima, which might lead us to get stuck at one of these local optima. Next, the best solutions from the above ( solutions (50%, 50%) is calculated.The new value for the best solution is compared to best known solution and if it was found to be better it will replace it.
The procedure is then repeated until a certain stop criterion is met.This stop criterion can be a pre-specified number of iterations (t), or when there is no further improvement on the final value of the optimal solution we obtained.
The pseudo code for the GNA algorithm is shown in Fig. 3.

III. EXPERIMENTAL RESULTS AND ANALYSIS
The GNA algorithm was used to solve the Traveling Sales man Problem (TSP).The TSP problem consists of a number of cities that need to be visited one time for each, starting from one city and ending at the same city.In order to optimize the TSP problem, the optimal sequence of the different cities that gives the minimum cost (distance) of the tour length has to be found.Thus, the objective (distance) function for the TSP is given by:  Where: : The total distance for a sequence of N cities. : The Euclidean distance between the current city and the next city to be visited.: The Euclidean distance between the last visited city and the first visited city.
To solve the TSP problem, we have to find the optimal sequence (S) that will give the shortest distance.If all the possible solutions are to be checked, then a total number of the combinations will be (N!) for asymmetric TSP or (N!/2) for the symmetric TSP.Obviously, if the number of the cities (N) is small then all the combination can be tried and a deterministic optimal solution can be found.However, if the number of the cities is large, then checking all the possible solution will take very long time and the complexity of the TSP problem will grow exponentially.For this reason, different meta-heuristic algorithms have been widely used to solve TSP problems.
In this paper, the GNA algorithm is used to solve a 29 cities TSP problem.The data were obtained from a real world problem that contained 29 cities in Bavaria, Germany; the source of these data is Zuse Institute Berlin [26].The optimal solution for this problem is known and documented (2020).The GNA algorithm was implemented using MATLAB software, and the total number of solutions (m) generated at each iteration was 50.
At, each iteration 25 feasible solutions were generated from the whole search space and the other 25 solutions were generated from the neighborhood of the fittest solution.The neighborhood move operator that was used in our case is the two-opt swap; where two cities were randomly chosen and swapped.The code was run for different times and at each time the obtained optimal solution and the run time were recorded.The stopping criteria used was 10000 iterations.The results for the GNA algorithm from the MATLAB code are shown in TABLE.1.
As it can be seen from TABLE.1, in the 10 different run times, we obtained a near-optimal solution that is very close to the known optimal solution.The results of the GNA algorithm were also compared to the Genetic algorithm (GA).
The parameters for the genetic algorithm were as the following: Generation size: 50 Crossover probability: 90%   .II shows that the run time for the GA is more than twice the run time for the GNA, and the solution obtained by the GA is not always close to the known optimal solution.MINTAB software was used to conduct a statistical analysis between the means of the two optimal solutions obtained by both GA and GNA.
Statistical Analysis was conducted to test if there is a statistical difference between the average for each algorithm.A two-Sample T-Test was used for this purpose.The output from MINITAB is shown in Fig. 5. MINITAB output for GNA and GA statistical analysis.
A 2-Sample t test showed that there is a significant difference between the optimal solution obtained from both GNA and GA , P-value= 0.000.The 95% CI for difference was (-213.1,-88.1).
Since the difference is always negative as indicated by the confidence interval, This shows us that on average the optimal solution is always higher for the GNA.
The difference in the means between the solutions obtained by the two algorithms is also clear in the Box plot, as shown in Fig. 6.It can be seen the GNA outperformed the GA in terms of obtaining a near optimal solutions, and the run time to get this solution was also less.The reason for that is the selection process in GA is more complicated, and it requires sorting the solutions in each generation, whereas in our GNA, the best solution is always selected.Also, the method by which the solutions evolve in each iteration is much simpler in the GNA, unlike the GA that uses Crossover and mutation at each iteration; which makes it take longer time.In this paper, a new meta-heuristic optimization method was introduced and named Global Neighborhood algorithm (GNA).This optimization method is a population based algorithm; since it starts with generating a set of random solutions from the search space for the optimization problem.The proposed algorithm can be used to solve combinatorial optimization problems.These combinatorial problems are usually more difficult to solve than other continuous optimization problems.The methodology of this algorithm was elaborated and 29-cities TSP optimization problem was solved using the GNA.The TSP optimization problem was also solved using genetic algorithm (GA) and the results were compared to the GNA.Statistical analysis was conducted using MINITAB software, and it was found that the GNA showed better performance, and the results obtained were very close to the known optimal solution.Future studies can include different variants for the basic GNA algorithm to enhance the search power.
Define objective function (g) Initialize the values for all parameters: m,t Generate (m) feasible solutions from the search space Evaluate the fitness from the objective function (g) Optimal solution= the best solution.i=1 Do while i<t,++ Generate 50% × m solutions from the neighborhood of the best solution Generate 50% × m solutions from the search space Find the best solution from the (m) generated solution If best solution is less (better) than optimal

Fig. 3 .
Fig. 3. Psuedo Code For GNA AlgorithmThe flowchart for the GNA algorithm is shown in Fig.4.

TABLE I .
MATLAB CODE OUTPUT FOR USING GNA TO SOLVE THE TSP PROBLEM.

TABLE II .
MATLAB CODE OUTPUT FOR USING GA TO SOLVE THE TSP PROBLEM.