Ant Colony System for Dynamic Vehicle Routing Problem with Overtime

Traditionally, in a VRP the vehicles return to depot before the end of the working time. However, in reality several constraints can occur and prevent the vehicles from being at the depot on time. In the dynamic case, we are supposed to answer the requests the same day of their arrival. Nevertheless, it is not always easy to find a solution, which ensures the service while respecting the normal working time. Therefore, allowing the vehicle to use additional time to complete their service may be very useful especially if we have a large demand with a limited number of vehicles. In this context, this article proposes a mathematical modeling with an Ant Colony System (ACS) based approach to solve the dynamic vehicle routing problem (DVRP) multi-tours with overtime. To test the algorithm, we propose new data sets inspired from literature benchmarks. The competitiveness of the algorithm is proved on the classical DVRP. Keywords—Dynamic vehicle routing problem (DVRP); multitours; mathematical modeling; hybrid; Ant Colony System (ACS); overtime


INTRODUCTION
Being defined more than 50 years ago, Vehicle Routing Problem (VRP) is one of the most classical combinatorial optimization problems.The main objective is to find the optimal path that can visit all nodes in question.For example, in a capacity VRP, these nodes are a set of customers that need to be served from a single depot with limited load capacity vehicles.Another example is the VRP with pickup and delivery, in which, the nodes can be a set of customers who will be served and from which the goods can be collected and a set of depots where vehicles get off.For each of these two examples we can define several sub-variants by adding some specifications to the problem.For example, we can find a problem with the constraint of time windows during which customers must be visited.Another example is the stochastic case where the requested quantity is unknown exactly before visiting requesting customer.Accordingly, the VRP become a very large class of problems that several studies have devoted themselves to review and classify [1], [2].
Among VRP variants, the dynamic VRP (DVRP) is relatively recent.In this variant, the information available at the beginning is incomplete and is subject to random variations over time.In other words, the starting solution is adjustable according to new data.In Fig. 1, we have a simple example of a DVRP where a vehicle has to serve a set of customers.Dynamic customers (E and F) are inserted in the new planned routes taking into account customers not yet visited (in dotted line).This concept, relatively new, has brought several advantages that are potentially beneficial for transportation companies; it can be helpful for companies to increase their competitiveness.As long as it allows companies to serve their customers on the same day of their requests and ensure, there is a better customer satisfaction.Besides, due to its flexibility and adaptability, DVRP is very useful in the emergency context where we need an immediate response to requests.Furthermore, the DVRP can handle dynamic travel time.This case is more present in urban areas where it is more difficult to predict network travel time because of the great congestion especially during peak hours.
All these advantages cannot be implemented without technological tools allowing real-time communication between the dispatcher and the driver.Fortunately, new communication and geographical location tools have allowed dispatchers and drivers to have a real time idea on the state of the network, and provide a real-time response to customers' requests.
That is why, Intelligent Transportation Systems [3] area set of platforms, each one is dedicated to a particular process.Among these platforms the Advanced Fleet Management Systems (AFMS) which are very useful in the DVRP case.They are specifically designed for dynamic or static business management of fleet while considering possible variations in the travel time on networks links.
As a result, since its first introduction by Psarftis [4] many variants of DVRP have been introduced and studied and literature studies have reviewed and classified them [5]- [8].
Despite all this number of variants treated, the concept of overtime still very little studied in the DVRP literature.This concept, which is widely used by transportation companies in Current route New customer www.ijacsa.thesai.orggeneral, is more needed in dynamic cases.Dynamically reoptimizing the current planning to insert new queries usually provides a solution that consumes more time by comparing it with a solution that considers all customers from the beginning.To verify this hypothesis we can just compare the results of the total traveled distance of the dynamic case in Kilby et al. [9] instances and the static case in those of Taillard et al. [10].Even so, in this model, we must answer the maximum of requests the day of their arrival.If we have a limited number of vehicles, we will need more overtime especially in case of high demand.In this article, we introduce a new variant of DVRP with the concept of overtime.To this end, we will consider the case of a transportation company that has a homogeneous fleet of trucks and which responds dynamically to customers' request.Each truck can perform several tours during the day.A limited number of trucks are available to satisfy all customers 'orders.Consequently, it is not always easy to find a solution that ensures the service while respecting the normal working time.If necessary, the trucks are allowed to use overtime on the condition that it shouldn't exceed the maximum legal overtime.So, we have two objectives in this problem; Minimize the maximum overtime performed by trucks and minimize the total traveled distance.Thus, we propose a multiobjective mathematical model.To solve this problem we propose a hybrid Ant Colony System (ACS) algorithm The rest of this paper is organized as follows: the second section presents a brief DVRP literature review.The third section presents the mathematical model.The methodology of resolution will be presented in the fourth section.Before concluding, we will present the numerical results in the fifth section.

II. DVRP: LITERATURE REVIEW
Going around DVRP literature we find all types of optimization methods known up to now, from the exact methods up to the metaheuristics.Otherwise, the literature addresses the DVRP according to four main perspectives: deterministic, stochastic, continuous and periodic.In the deterministic case we consider only known requests and deterministically respond to the dynamic ones.In other words, the current plan, takes into account the data actually known by the dispatcher.While in the stochastic case, we can consider stochastic data, such demand forecasts while elaborating a preliminary routing plan.In continuous optimization, the reoptimization starts at the arrival of each new request to get a new plan.While in the periodic one, the planning period is decomposed into time intervals.In this way, during each time interval, incoming requests are collected and inserted all at the end of the current interval or at the beginning of the next interval.
In order to have a global view of this research field, we will list some research works for the four main perspectives, and then we will exam those are related to the DVRP with overtime.
Let's start by works that have adopted deterministic continuous optimization.It started with Psaraftis [4], who is the first to introduce the dynamic DVRP.In this work, he used dynamic programming to deal with the dynamic dial-a-ride problem.The aim is to find the best route at each new demand.
In the same perspective, Gendreau et al. [11] applied the Tabu search (TS) to solve the DVRP.As soon as a new request arrives, the algorithm saves its former results in the adaptive memory to insert the locations of the new requests.The TS was also applied, in a deterministic continuous way, by Chang et al. [12] on the DVRP with pickup, delivery and time windows.
As for the work done for the DVRP in a deterministic periodic manner, the first one we mention is the work of Chen and Xu [13] who have proposed an approach based on linear programming and dynamic column generation for the DVRP with time windows and infinite fleet.On their part, Hanshar et al. [14] decompose the working day into time intervals.Then they launch the optimization program at the beginning of the time interval.They propose a solution based on Genetic Algorithm (GA) for capacitated DVRP.The main contribution in their work was the way they represent a chromosome in dynamic optimizations.In the same perspective and using the Neighborhood Search Algorithm (NSA), Hong [15] solves the DVRP with hard time windows.It applies the withdrawal reinsertion mechanism of this algorithm to insert new queries in the already planned routes.
Another adaptation of the NSA to DVRP was proposed by Khouadjia et al. [16].To conclude with the deterministic periodic manner, we will quote a very interesting work with ACS.It is the first application of the ACS on DVRP by Montemanni et al. [17].In this masterpiece, the authors decomposed the planning period into time intervals.At the beginning of each time interval, the optimization program is launched to insert the incoming requests during the previous period in the planning of the rest of the day.The static problem is solved using an ordinary version of ACS.As for the other time intervals, the article applies the same algorithm with a modification of the initial rate of pheromone on the arcs of the network.As each sub-problem is potentially similar to its successor, a pheromone conservation mechanism is applied to put the weight on the arcs belonging to the previous solutions and thus reduces the execution time.
In a stochastic periodic manner, Hvattum et al. [18] have developed a heuristic approach whose principle is to divide the planning horizon into time intervals and to assign a set of promising queries to the vehicles at the beginning of each interval depending on their frequencies of occurrence in the possible stochastic scenarios.The algorithm then uses the Branch-and-Regret method to merge them in order to have a single optimal solution.Another example of the stochastic perspective, but this time in a continuous way, is that of Hemert and Poutré [19] who have used GA to solve a problem of collecting charges from customers and delivering them to a single central depot.The authors have introduced the notion of fruitful regions, where there are more probable potential customers.In the same perspectives, a more recent study of Schyns [20], aimed at optimizing the routing of refueling trucks in an airport, proposed an adaptation of the ACS to take into account the lack of visibility of the planning period and the hard time windows.
Our article will deal with the case of a deterministic periodic DVRP with overtime.After checking articles published in this area, we have found only one article that is www.ijacsa.thesai.org the article of Gendreau et al. [21] which deals with a DVRP with pickup, delivery and time window by allowing the use of the overtime and without considering the capacity constraint.To solve this multi-objectives problem, the authors proposed NSA algorithm while adopting a deterministic continuous perspective.

III. MATHEMATICAL MODEL
The objective of this article is to solve a capacitated DVRP multi-tours that tolerate overtime.We have a homogeneous fleet of vehicles with limited capacity and a single depot.The vehicles leave the depot at the beginning of the day with a full capacity and return there to restock and start a new tour or to close their working day.In our case, we adopt the Montemmani et al. [17] approach, which divides the planning period into time intervals.Thereby, we collect all the incoming requests during a time interval, to insert them in the planning of the following period.In this way, the re-optimization is launched at the beginning of each interval and the found solution covers all the rest of the planning period.
We set time limit to accept customers' requests.After this time, the incoming requests will be reported to be recorded in the next planning period.Thus, at the beginning of the planning period, we have a CVRP with a homogeneous fleet and a single depot.However, at the beginning of other time intervals, a vehicle that has already served one or more customers will have a less load as it will have a starting point other than the central depot.We will call this starting point fictitious depot.Therefore, we have a CVRP with heterogeneous fleet and several fictitious depots.In both cases, trucks are allowed to perform several tours.At the end of a time interval, if a truck is serving a customer, this later is considered as fictitious depots in the problem of the next time interval.Else, it will be on the road to a destination customer.In this case, this later is considered as fictitious depots in the problem of the next time interval.We present the mathematical model of one time interval.Thus, we put: We have a single decision variable: if the customer j was visited after the customer i during the tour r else To simplify the problem formulation, we assume that each vehicle can carry a maximum of n tours during the remainder of the day.Considering that we have K trucks in service, the maximum number of tours that can be achieved is .Excluding already served customers, we consider that a tour starts from the fictitious depot where the customer exists at the beginning of the current time interval.
Therefore, for every truck , we choose to reserve indices of tours that will be eventually assigned to it k k k n-.Tours r with indices ∑ ∑ , refers to tours that will not actually be performed by the trucks.So we define the set of possible tours served by the truck .
If a truck k actually makes n successive rounds, these rounds will have respectively the index k, k + K..., k +(n-1)K.The tour with the index k is the first to be performed, then k+K and finally the owner of the index k + (n-1)K.In this way, if the truck k is initially parked in a fictitious depot different from the central depot, the tour of index r = k must imperatively start from this fictitious depot.Any other tour starts from the central depot.So, if r * + then, the tour r=k, starts from depot f>0.Else, r starts from the central depot.Now, we define the available capacity for the tour r as: The travel time of a trunk is the necessary time to serve all its tours.However the overtime for each vehicle is the additional time over the planning horizon whose vehicle needs to ensure its service.If the vehicle respects the constraint of time, the overtime is zero.It can be calculated for a trunk k as follow: www.ijacsa.thesai.orgk max The solution overtime is the maximum overtime of all vehicles t's related to the last truck that returns definitively to the depot: During each period of re-optimization, the mathematical model consists of minimizing the maximum overtime and the routing cost.If the problem has a solution that respects the constraints without needing overtime, then the objective in this case is to minimize the total traveled distance.Otherwise the objective will be to minimize the total overtime of the solution.The objectives of the problem can be formulated as follows: Under the following constraints: The first constraint (1) is made to comply with the remaining capacity of trucks in each tour.The second equation (2) restricts the overtime to a permitted maximum value.The third constraint (3) ensures that each customer is visited once and only once.The flow conservation constraint at the customer and the central depot level is carried out by (4); each truck that visits a customer must leave him after his delivery request and every truck that leaves the central depot must come back at the end of the working period.While the constraint ( 5) is made to ensure that trucks initially parked in a fictitious depot other than the central one must perform at least one tour.
In this way we will be sure that these trucks return to the central depot at the end of the day.The sixth constraint (6) expresses that a fictitious depot can't be a destination.The constraint (7) prohibits the creation of sub-tours.Finally, the integrity constraints associated with decision variables are included in (8).
A study of Hassein and Rubinstein [22] has previously proved that an ordinary VRP is considered NP-hard when the set of customers contains more than 3 customers, so impossible to solve by an exact method.A meta-heuristic can therefore be used to solve this problem in order to find a good solution in a reduced time.The mathematical model just described is done to better describe the specificities of the problem and the objective functions.

A. Static Problem
To solve the problem that has just been described, we propose an approach based on the ACS.Firstly, we solve the problem of the beginning of the day.This problem, characterized by a single central depot and a homogeneous fleet, contains yesterday's requests arriving after the time .The steps of our algorithm are described in Fig. 2.
First, we try to find a realizable solution that respect the normal time by minimizing only the first objective of the problem.If such solution isn't found we try to minimize the second objective without exceeding the maximal permitted overtime.Thus, the first step is affecting a positive value of to each arc.Then, we place an ant on each customer.This allows for more diversification in the solutions found.The ants build, then, their tours.To move from one customer to another, the ant must select the clients that can visit without violating the constraints of the problem (remaining truck capacity and maximum return time).If no client responds to these constraints, the ant returns to the depot to begin a new tour.Otherwise, the ant chose the next customer to visit according to the following rule: j { argmax u i k [ iu (t) iu ] if q q if q q Where q: A random variable uniformly distributed on q : A parameter in the interval .It defines the balance diversification / intensification; iu d iu : The visibility of the arc i u .It corresponds to the inverse of the distance d iu between and .iu (t) The pheromone rate on arc i u at the instant t.
he relative influence of the visibility.i k : The set of nodes that can be visited just after the position i by the ant.i k : A randomly selected customer with probability: www.ijacsa.thesai.org Steps of the algorithm.
After building a solution by an ant k, we allocate its tours to the vehicles in order to calculate the total overtime.The procedure of vehicle allocation is described in Fig. 4.Then, a local update of pheromone is done on arcs it visited according to the following formula: k Where : The evaporation factor set to a positive value less than 1.This factor is intended to avoid the unlimited accumulation of the pheromone traces on the edges of the graph.k : The overtime performed by the ant k.Before applying the global update, the best solutions of the iteration undergoes a phase of optimization by a local search intra and inter tour.The global update is then carried out as follow: ij t where, the arcs ( )belong to the best solution S of the iteration.ij s and s : is the total time of S.
At the end of each iteration, the solution of the best ant undergoes an optimization phase by local search algorithm that combines an intra and inter tour optimization (Fig. 3).This procedure is the same used by Ayadi and Benadada [23].In this phase we look to optimize, only, the total traveled distance.In a first step, the crossovers are eliminated and the possible permutations, that decreases traveled distance within each tour, are sought.Then, the optimal permutations carried out.Afterward, a reinsertion operator comes to see if there is any better location for a customer in its tour.The optimal reinsertion is carried out.These three operations (elimination of crossovers, permutation and reinsertion of customers) form the intra tour local search.Subsequently, we look for the possibility of exchanging two customers of two different tours.Indeed, the optimal exchange is carried.Next, we try to find the possibility of moving a customer from one tour to another without violating problem constraints.The optimal displacement is chosen to be performed.These last two operations form the local search inter tour.Finally, the intra tour search is performed once, while the inter tour search is performed twice.

B. Dynamic Problem
In the dynamic case, we have the same steps described in the static case.However, the details are different since heterogeneous fleets and many fictitious depots, beside the central one, characterize the dynamic problem.The initialization of pheromone rate is done according to the pheromone rate retention mechanism proposed by Montemanni et al. [  Once pheromone traces are initialized, we place an ant on each fictitious depot.Each ant creates its tours by visiting customers one after one until the capacity or the remaining time don't allow to insert a new customer, in this case it returns to the central depot.
The ant starts the new tour from the nearest unvisited fictitious depot or from the central depot if all fictitious depots are visited and there are still unvisited customers.The move, from the central depot to the fictitious depot is done without deposing pheromone.
The transition rule depends on a parameter q , which defines the balance diversification / intensification.In order to choose the next customer j, the ant k located in the customer i use the following rule: With q : A parameter in [0,1] that determine the balance diversification/ intensification q: A random variable uniformly distributed on , -

iu (t):
The pheromone rate on the arc ( ) at the instant t. he relative influence of the visibility.i k : The set of nodes that can be visited just after the position i by the ant k. j i k : A randomly selected customer with probability: The update of pheromone is divided into two levels: a local update and a global one.The first is done after building a solution by an ant k.We modify the pheromone of arcs visited by this ant according to the following formula: The overtime performed by the ant k after affecting vehicles to its tours.The procedure of vehicle affectation is detailed in Fig. 5.
At the end of each iteration and before applying the global update, the best solution found in this iteration undergoes a phase of optimization by a local search intra and inter tour.To this end, we use the local search algorithm of the static problem.Finally, we must note that; if in a time interval of the problem we cannot find a solution without overtime, we will have one goal (minimize overtime) in the following time interval.In other words, we always try to minimize the total distance but once the algorithm ceases to find feasible solutions, the objective becomes the minimization of the overtime for all the rest of the planning period.

V. COMPUTATIONAL RESULTS
We have parameterized our algorithm according to the results found by Gambardella et al. [10] on the classical VRP.Thus, we set:q Gambardella et al. [24] gave the value of n ost where ost is the cost of a solution found by a greedy heuristic; In our case we fixed it in the static case to .This value was chosen by comparing its results with those of other values.In the dynamic case, we set n d ost p where is the number of clients of the current dynamic problem and ost p is the cost of the previous problem; In this way, we reduce the execution time consumed by the greedy heuristic, especially since the current problem is similar to the previous one.
For the number of time intervals and the co we adopt the same parameter of Montemmanni et al. [17] ( co and 25 time intervals).The algorithm stops after 200 iterations for all instances.It is coded in Java and executed on a machine with Intel Core i5 processor, 2.6 GHz, 8GB of RAM and Windows 8 as the operating system.

A. Static Case
The results of the static algorithm are already detailed in [25].In this paragraph we give a reminder of these results.

Assign the tours starting from fictitious depot to corresponding truck
Rank the remaining tours from the longest to the shortest; While (there is "not assigned tour") Rank the trucks according to the total length of the assigned tours in an ascending order; Assign the Firsttour to the Firsttruck Delete Firsttour from "not assigned tours" end www.ijacsa.thesai.orgTo avoid a zero in the denominator of ij , we have grouped customers who have the same coordinates in a single customer with a demand equal to the sum of the requests.Therefore, in F134, we have 132 customers and in CMT5, we have 199 customers.Table I presented our results referred to as OBM16, compared with those of Ayadi and Benadada [23] referred to, as AB13.While Table II presented the average of execution time of compared works, knowing that the AB13 time is normalized to our processor according to Geekbench benchmarks.
To qualify the solutions obtained, we use the Longest Trip Rate (LTR).It compares the time of the longest trip, which corresponds to the time of the truck that return the last to the depot, on the normal time horizon T: With t(k) the time taken by the vehicle to make his tours.

B. Dynamic Casewithout Overtime
These results are already detailed in [26].In this paragraph we give a reminder of these results.
In order to compare our algorithm with other works from the literature, we test it, first, on the benchmark of Kilby et al. [9].To this end, we fix a single objective for the problem that is minimizing the total traveled distance.We consider five runs of the algorithm for each instance.Table III   The hybrid ACS outperforms the ACS Montemmani 2005 on 16 instances.The execution time is also lower than that of Montemanni 2005 with a percentage of 35% knowing that the Montemmani time presented in Table III is normalized to our processor according to Geekbench benchmarks.

C. Dynamic Case with Overtime
To test our algorithm, we have made some modifications on Kliby et al. [9] benchmark data set.Since, they are characterized by a big number of available trucks that is 50 for each instance, we can't have overtime with these instances.www.ijacsa.thesai.orgWe propose a data set for the DVRP with overtime using the twenty one basic Kilby problems; seven problems of Christofides et al. [27] (C), two of Fisher (F) [28] and twelve of Rochat and Taillard [29] (Thai).We use the same demands and truck capacities of the basic problems.Instances are generated by proposing several values of m (the number of available vehicle) and a restricted values of time horizon z , with z is the value of the best solution found by Rochat and Thaillard [29] for the VRP problem.The arrival time of customer requests is proportional to the arrival time set by Kilby et al. [9].We set the maximum allowed overtime for each instance to one quarter of the normal time horizon.Time and distance are considered equivalent.The algorithm stops after 200 iterations for all instances.Table IV presents the value of T for m=1 and the size of each instance.For each instance, we tested m between 1 and 5.In Table V we write, just, the instances to which we found at least one feasible solution (which respects the maximum time plus the maximum permitted overtime).For each instance three runs of hybrid ACS are considered.T is the normal maximum time while m represents the number of used vehicles.We consider the average of the total distance traveled and the overtime as well as the minimum of these two values obtained during the three executions.Time represent the average of the execution time in minute.A feasible solution is any solution that does not exceed the normal time plus the maximum overtime allowed.Instances denoted by * are instances in which only one feasible solution has been found, while those denoted by ** are instances in which two feasible solutions have been found.For both CMT4 and CMT5, we don't find any feasible solution.Therefore, they are not noted on the table.For the other instance, we found three feasible solutions.The graph of Fig. 6 shows the minimum and maximum value of the LTR according to m.
A note to make is that the more the m increases, the more the minimal LTR increases.However, the maximum value is not significant compared to m.This can be justified by the fact that the maximum value of the LTR for m = 1 was obtained for the CMT11 instance which has no feasible solution for the other value of m.If we exclude this instance, the maximum value becomes 1.20 (Thai100d).Adopting this remark we can conclude that the LTR increases with m.This is also justifiable by the fact that we could not have a feasible solution for the big values of m.
Highlighted results are results where the minimum value of the total distance traveled and the minimum value of the overtime do not match the same solution.We have five instances where there is no such correspondence on 62instances.This gives a percentage of 3,1%.Thus in 96,9% a solution that minimizes the overtime minimizes also the distance.This being stated, we can conclude that the two objectives of our problem are proportional on 96,9%

D. Practice Use
For an industrialist what counts from all what is said is having an optimal or near optimal solution in a practical time.Our algorithm was able to give a near optimal solution for the DVRP with overtime in an execution time that does not exceed 6.50 (min, second) on average.To take advantage of these results, we are working on a software project that runs the same algorithm but with adaptable and comfortable interfaces for managers and industrialists while allowing to have results for a static problem in case of need.

VI. CONCLUSION
To conclude, this article introduces a new variant of the DVRP that is the multi-tours DVRP with overtime.The article gives a mathematical model of the problem with a hybrid ACS resolution.The results of the static and dynamic algorithm are competitive, comparatively to other works from literature.To test our algorithm on the dynamic case with overtime, we have proposed new benchmarks inspired from the very famous ones.Results have shown that the two objectives of the problem are proportional on 96,9%.
Several avenues exist for future works; a future goal is to work with another metaheuristics on the same problem to compare its results with those given by the hybrid ACS.Another research direction includes introducing new constraints such customer time windows, as well as considering stochastic data while providing the first planning of the day.

::
Set of depots : Set of Customers to be served (fictitious depots are not included) : Set of depots without central depot * + Set of Customers and central depot * + Set of trucks f: Index of depot (including fictitious depots and the central one) i: Index of customers k: Index of trucks n: Maximum number of tours for a truck : Index of the central depot, d ij : Cost (distance) of movement between i and j t ij : Travel time between i and j k : Remaining capacity of the truck k : The initial capacity of trucks l : Normal driving time remaining for the period l Length of working period : Maximal legal overtime q i : Quantity requested by the customer i f k { if the truck k is initially stationned in fictitious depot f else
ij old : The final value of the pheromone level on the arc (i,j) of the previous problem.r : A positive factor less than 1, which regulates the conservation of pheromone level.: The initial constant of pheromone.It initializes arcs corresponding to the new clients.

:
The visibility of the arc ( ).It corresponds to the inverse of the distance between and .

Fig. 6 .
Fig. 6.Max and min of the LTR value depending on m.

TABLE I
[17]ents our results and those of Montemanni et al.[17]

TABLE III .
NUMERICAL RESULTS OBTAINED BY OUR HYBRID ACS COMPARED TO THE ACS OF MONTEMANNI 2005

TABLE V .
NUMERICAL RESULTS OF DYNAMIC CASE WITH OVERTIME