Hybrid Approaches based on Simulated Annealing, Tabu Search and Ant Colony Optimization for Solving the k-Minimum Spanning Tree Problem

In graph theory, the k-minimum spanning tree problem is considered to be one of the well-known NP hard problems to solve. This paper address this problem by proposing several hybrid approximate approaches based on the combination of simulated annealing, tabu search and ant colony optimization algorithms. The performances of the proposed methods are compared to other approaches from the literature using the same well-known library of benchmark instances. Keywords—k-Minimum spanning tree; metaheuristics; simulated annealing; ant colony optimization algorithms; tabu search; approximation algorithms


I. INTRODUCTION
In this work we attempt to provide some approximate methods to solve the well-known combinatorial optimization: The k-minimum spanning tree (k-MST) problem. We want to find a tree in an edge-weighted graph G = (V, E) that have exactly k edges and which minimize the sum of the weights of its edges. The mathematical formulation of this problem has been introduced by [1], It was demonstrated that the k-MST problem is known to be NP-hard [2], it is very difficult to find optimal solutions to large-scale problems that can be formulated as a k-MST within acceptable time.
In literature, several approaches using metaheuristics were proposed to tackle the k-MST problem [3], [4], [6], [5], [7]. In [8], three approaches to deal with the k-MST were presented: an evolutionary computation, ant colony optimization (ACO) and tabu search (TS), in order to show and compare the performance of these methods, the authors built a library named KCTLIB which contains some graph instances for k-MST problems, after that they executed their programs, numerical results showed that ACO algorithm is the best choice to deal with k-MST with small cardinalities, while TS is the best choice for k-MST with large cardinalities. these conclusions had inspired the authors in [14] to hybridize TS with ACO and the authors in [16] to hybridize TS with SA, they have applied their approaches to some graph instances from KCTLIB, and they have presented the results of their methods.
The objective of this paper is to attempt new hybrid approaches by coupling simulated annealing (SA), TS and ACO algorithms. We aim to find new best k-MST solutions, numerical experiments were performed using two graph instances from KCTLIB. Our experiments show that the suggested approaches are able to find new best values for the two graphs and for several cardinalities.
The paper is structured as follows. The problem formulation is presented in section II . In the section III we give the description of the main components of SA, TS, and ACO algorithms. the results of the computational experiments and the discussion are reported in section IV . Finally, in section V we give some conclusions.

II. PROBLEM FORMULATION
Given an undirected edge-weighted graph G = (V, E) on a set V of n vertices , and a positive cost function w(e) on the set of edges E, |V | is the number of vertices of G, |E| is the number of edges. A spanning tree (ST) of G is a connected subgraph that contains all vertices and without any cycle. The k-spanning tree (k ≤ |V | − 1) that we note T k is a tree that contains k edges, if k = |V | − 1 we get a spanning tree. we note by X k the set of possible k-spanning trees, The set of all possible k-spanning trees is denoted by X k . the k-minimum spanning tree (k-MST) problem asks for a k-spanning tree with the minimum sum of weights. The k-MST problem can be formulated as follows: An optimal solution can be easily found in case of small problem size by enumerating all k-spanning trees in a given graph. However, it has been demonstrated that the k-MST problem is a NP-hard problem. Therefore, it is very important to develop new approximate methods using metaheuristics in order to provide solutions to real-world problems in reasonable time.

A. Simulated Annealing
SA is an approximate algorithm inspired from thermodynamics, which was introduced in [9] and it is widely used to address many discrete and continuous optimization problems, such as the travelling salesman, and vehicle routing problem, etc. Researchers are also applying it to multi-criteria  optimization problems [10], [11]. SA is a neighborhood search method. The basic concept of this method is to move at each iteration from a solution x to another one that belongs to its neighborhood N (x). Before describing the main SA components, we present the neighborhood structure used in this method.

B. The Neighborhood
To move around and allow the algorithm to discover the search space, the possible transitions or movements from one solution to another are subject to the chosen neighborhood structure. The neighborhood of a tree T k is the set of k-ST built by deleting one edge from T k and replacing it by an edge of the graph G and which doesn't belong to T k . We choose randomly an edge from the current T k and replace it by another one, which is also is selected randomly from G\T k . To clarify the neighborhood structure, we take the example of a tree composed by six edges in Fig. 1. The 6-ST contains the following edges: (2,3),(3,6), (6,5), (5,4), (6,9), (9,8).
If F < 0 then set the neighborhood element as the current solution Otherwise, the neighborhood element will be the current solution with a probability equal to exp(−F /T ), T is the value of the temperature in the current iteration b) Use the geometric cooling schedule to decrease the current temperature

D. Tabu Search
TS was introduced by F. Glover and Laguna in [12], [17], [13]. This metaheuristic method is used at large scale to find approximate solutions for real-world optimization problems, TS can be applied for example in logistics, resource planning, telecommunications, scheduling, etc.
TS is based on simple ideas inspired from the human memory, it is a local search method which is known to have the tendency to be stuck in local extremums, TS address this problem by prohibiting already visited solutions and avoiding the problems of cycles, in this way, the whole solutions space can have the chance to be visited. For more details about the components description and the complete algorithm of TS algorithm integrated in our proposed hybrid approaches please refer to [14],

E. Ant Colony Optimization
ACO algorithm is a probabilistic technique for tackling computational problems which can be reduced to seeking optimal paths in graphs, it was first introduced by Dorigo, Maniezzo and Colorni [15].
The principle of the first algorithm is inspired from the capability of ants to seek the best path from their colony to source food and vice versa. As they move, they deposit an organic compound on the ground called pheromone, paths (solutions) with a higher pheromone level have a higher probability of being selected by ants (better solutions).
In this paper, the ACO is used in order to diversify the search process, and to explore new regions that may have not been visited in previous iterations by SA algorithm. For more details about the components description and the complete ACO algorithm used in this paper refer to [14].

F. First Hybrid Approach: SA Combined with ACO
In [8], ACO algorithm showed a high diversification ability which allowed it to explore new areas of solutions. To take advantage of this feature, we propose a hybrid approach that combines SA and ACO. The ACO algorithm is used in order to diversify the search process when the SA algorithm can no longer improve the current solution.
In summary: an initial solution will be generated using the Prim method, then the SA will be launched until until that we don't observe any improvement is occurring on the current solution; at that time we call the ACO algorithm to move the search to other regions of solutions space. Next, the SA will resume. Below the outline of this hybrid approach that we note Hybrid SA-ACO : Step 1. Initialization of SA parameters and generation of an initial solution (see Section III-C).

G. Second Hybrid Approach: SA Combined with ACO and TS
In the literature, TS has shown a high intensification potential in finding good solutions in narrow search space for the k-MST problem, the experimentations carried out by [8], have proven this ability, the numerical results showed also that TS is very efficient in case of k-MST with large cardinalities. In [14] TS had had showed another ability which is that it can be a good partner in a hybrid algorithm, the combination with the ACO algorithm had allowed to find new best values for the objective function. In [16] another hybrid approach was proposed by combining TS and SA. These results were obtained even that the neighborhood structures chosen by each approach were different In this hybrid approach, we combine the three metaheuristics seen before, namely, SA, TS and ACO, we will exploit the advantages of each of them to find better solutions. This approach starts with an initial solution generated using the Prim method, then the SA will take the hand to improve this solution under the control of the parameter T IM E T O DIV ERSIF Y ; the ACO will be launched to take the search to another region of solutions not yet explored. When the stopping criterion (T M P LEV EL) is reached, the TS is launched on the best solution found to intensify the search in its neighborhood and get the best one.
The outline of this algorithm, that we note Hybrid SA-ACO-TS, is as follows: Step 1. Initialization of SA parameters and generation of an initial solution (see Section III-C).
Step 2. SA and ACO a) Run the SA algorithm until T IM E T O DIV ERSIF Y b) Launch the ACO algorithm c) Repeat the two previous actions until T M P LEV EL Step 3. Tabu Search a) Run the TS algorithm as described in [14], the best solution found in Step 2 is the starting solution of this step.

IV. EXPERIMENTAL RESULTS
We have performed numerical experiments using two large regular graphs taking from the benchmark instance KCTLIB. The Table I shows the configuration of these graphs. The results obtained by the proposed approaches are compared to those obtained with the following methods: • Two solution algorithms proposed in [8], namely TS algorithm and ACO algorithm, denoted by TSB and ACOB, respectively.
• The simulated annealing algorithm with restart strategy by [7], denoted by SA.
• The hybrid approach that combine simulated annealing and tabu search by [16].
Our algorithms are coded in C programming language and runned on a computer with a CPU Intel(R) Core(TM) i5, 2.5x2.5 GHz, 4GB RAM. It should be stressed that for TS and ACO algorithms, we have used the same parameter settings as in [14], Our algorithms have been executed ten times on each graph instance, after that, we record the best, worst, mean and objective function values, also we record the mean time. Table II presents the SA parameter settings adopted to tackle the graphs 1 and 2. In Tables III-IV results TABLE II. SA PARAMETER SETTINGS ADOPTED TO GRAPHS IN TABLE I.   k  T0  T f  TMP RANGE  TIME TO DIVERSIFY  TMP LEVELS   200  15   0.01   10000   20 s   40  400  15  2000  30  600  15  5000  30  800  10  2000  30  900  10  To easily read and well interpret the Tables III-IV, we transform them into charts, Fig. 3 and 4 represent the order of the performance of each method and for different cardinalities, a higher order value corresponds to better performance. Fig. 3 show that: for k = 200, ACOB is the best; however for other cardinalities we notice that our proposed hybrid approach SA ACO TS is very competitive, because we have improved the best known solutions for k = 400 and k = 800. Fig. 4 show that for k = 400, Hybrid SA ACO TS is the best; however for other cardinalities we notice that Hybrid SA TS approach is very efficient in terms of best and mean values. It should be stressed that we have improved the best known solutions for k = 400 and k = 800. In summary, results reveal that: • SA is not efficient in case of large graphs.
• SA is very efficient when coupled with TS in finding optimal k-MST solutions.
• TS is known for its great ability to intensify the search, the obtained results had confirmed that; it can be coupled with SA and/or ACO algorithm to obtain good performances.
• ACO is very efficient when its coupled with TS.
• We have achieved good results by hybrid approaches only when TS is part of the hybrid approach, this is due to its high intensification ability.
• Our proposed approaches have consumed more time to provide optimal solutions.

V. CONCLUSION
This article suggests new approaches to address the k-MST problem. Hybrid approaches combining SA, TS and ACO were presented. In order to show the performance of the these methods, we compared them with other works from the literature using the same benchmark data KCTLIB. The numerical experiments showed that TS is effective to tackle the k-MST problems when it is combined with SA or ACO or both. In our future works, we will focus on how to improve the computational time of our approaches and then address the same problem in case of multi-objective optimization.