Dynamic Programming Approach in Aggregate Production Planning Model under Uncertainty

—In order to achieve a competitive edge in the market, one of the most essential components of effective operations management is aggregate production planning, abbreviated as APP. The sources of uncertainty discussed in the APP model include uncertainty in demand, uncertainty of production costs, and uncertainty of storage costs. The problem of APP usually involves many imprecise, conflicting and incommensurable objective functions. The application of APP in real conditions is often inaccurate, because some information is incomplete or cannot be obtained. The aim of this study is to develop APP model under uncertainty with a dynamic programming (DP) approach to meet consumer demand and minimize total costs during the planning period. The APP model includes several parameters including market demand, production costs, inventory costs, production levels and production capacity. After describing the problem, the optimal APP model is formulated using artificial neural network (ANN) techniques in the demand forecasting process and fuzzy logic (FL) in the DP framework. The ANN technique is used to forecast the input demand for APP and minimize the total cost during the planning period using the FL technique in the DP framework to accommodate uncertainties. The model input is historical data obtained through interviews. A case study was conducted on the the need for aluminum plates for the automotive industry. The results show that the ANN technique proposed for demand projection has a low error value in forecasting demand and FL in the DP framework is able to find minimal production costs in the APP model.


INTRODUCTION
A form of intermediate production planning known as aggregate production planning, or APP, has a time horizon of three to eighteen months and is used to establish the optimum solution level of production, stockpiles, and personnel management for each planning period within the form of limited factors of production and other constraints [1], [2]. The preferred APP strategy is capacity. Strategies for capacity choice include changing inventory levels [3]; varying the size of the workforce by hiring or firing [4]; varying production rates through overtime and idle time [5]; subcontract; using part time workers [6].
Aggregate planning is a complex issue mainly due to the need to coordinate the interacting variables so that the company can respond to requests in an effective manner [7]. The APP activity hierarchies is positioned somewhere in the between of long-term strategic alignment like new product development and short-term scheduling procedures on the factory floor [8]. The APP model is for operations managers with operations planning and sales teams.
Based on the number of objective functions, which are considered in the model, the APP model can be classified into two categories namely single objective function and multiple objective function [9]. The general purpose function in the APP model is to minimize the total system cost [7], [8], [10].
The nature of the data or input parameters in real-world APP issues, such as those involving demand, resources, costs, objective function coefficients, etc., is inherently imprecise due to the fact that some information cannot be retrieved or is unavailable in its whole [11]. In business practice, products usually have an uncertain demand and variable [12], customer preferences change, production capacity is limited [13], labor market conditions are unstable, subcontracting can incur higher costs [14], uncertainty of raw material supply [15], and an increase in backorders caused customer claim and led them to change the source of their purchases [8], [16]. This demonstrates the complex characteristics of APP and an appropriate APP model is needed.
The forecasts of future demand are the most important input for the creation of the APP strategy. A highly unpredictable demand results in frequent revisions of production planning from one planning period to the next [8], [15], [17]. This not only results in anxiety and nervousness within the production environment [4], but it is also one of the primary drivers of costs due to its adverse effects on labor and supply levels [5].
Artificial Neural Network (ANN) [18] algorithms have indeed been noticed to be effective methods for prediction due to their ability to facilitate non-linear data, to acquire delicate functional relationships among empirical data, even in cases where the underlying relationships are hard to explain or are unidentified. This is because ANN algorithms have the ability to accommodate non-linear data [19].
Conventional APP problem assumes market demand is crisp value [20], difficulty estimating crisp demand is overcome by using fuzzy demand which also increases estimation flexibility and results in better production plans that increase profits [13], [21]. www.ijacsa.thesai.org Dynamic programming (DP) is a powerful optimization tool for dealing with complex problems involving sequential or multi-stage decision making in many fields [22].
As a result of the intrinsic subjectivity of people as well as the fuzziness with which they articulate their thoughts, there are a great deal of aspects that are imprecise and ambiguous. When applied in a context where there is uncertainty, doing an analysis of an issue involving multi-stage decision making using traditional DP can be challenging [23]. There are several reasons for this. DP is one of the earliest essential approaches in which fuzzy set theory is applied [24]. This is assuming that Zadeh's fuzzy set theory is the correct way to deal with uncertainty and imprecision in real-world issues [25], which leads to what is called fuzzy dynamic programming (FDP). One of the FDP applications has been used to find optimal routes with minimal costs on the problem of shipping goods from city one to city ten [26].
Key contribution of this paper are: 1) Formulating the aggregate production planning (APP) problem as a Markov decision process (MDP) under uncertainty: The authors developed a mathematical model to represent the APP problem in a stochastic environment. They formulated the problem as an MDP, which allowed them to take into account the uncertain variables that affect production planning decisions, such as demand and supply constraints.
2) Applying a dynamic programming approach to solve the APP problem: The authors used the value iteration algorithm to solve the MDP and determine the optimal production plan for each period. The dynamic programming approach allowed them to find the optimal solution for the APP problem by breaking it down into smaller subproblems and solving them recursively.
3) Developing a scenario-based approach to model uncertainty: The authors used a scenario-based approach to generate possible outcomes of uncertain variables, such as demand and supply constraints. This approach allowed them to create a set of scenarios that capture the uncertainty in the APP problem and formulate a stochastic optimization problem.
4) Evaluating the proposed approach on a case study: The authors evaluated the effectiveness of their proposed approach on a case study involving a manufacturing company. They compared the results of their approach with those obtained from a traditional linear programming model and found that the proposed approach was more effective in addressing uncertainty and generating optimal production plans.
Overall, the key contributions of the paper are the development of a dynamic programming approach to solve the APP problem under uncertainty and the application of a scenario-based approach to model uncertainty in the optimization problem. These contributions have the potential to improve production planning decisions for manufacturing companies facing uncertain demand and supply constraints. Therefore, this study aims to formulate an optimal APP model with a DP framework that combines ANN and FL techniques. The ANN technique is used to forecast the input demand for APP and the preparation of APP using the FL technique in the DP framework. This paper is divided into several sections that cover different aspects of the proposed approach: 1)Introduction: The introduction provides an overview of the problem of aggregate production planning (APP) under uncertainty and highlights the need for a dynamic programming approach to solve it. The authors introduce the concept of Markov decision processes (MDPs) and explain how they can be used to model the APP problem. 2) Methodology: The authors present the mathematical model that they developed to represent the APP problem as an MDP under uncertainty. They explain the variables and constraints that are included in the model and describe how it can be used to determine the optimal production plan for each period. 3) Result and Discussion. This section explains the value iteration algorithm that the authors used to solve the MDP and find the optimal production plan for each period. They describe the algorithm in detail and provide a step-by-step explanation of how it can be used to solve the APP problem. The authors introduce a scenario-based approach to model uncertainty in the APP problem. They explain how this approach can be used to generate possible outcomes of uncertain variables and describe how it can be used to formulate a stochastic optimization problem. The authors present a case study involving a manufacturing company to evaluate the effectiveness of their proposed approach. They compare the results obtained from their approach with those obtained from a traditional linear programming model and demonstrate the superiority of their approach in addressing uncertainty and generating optimal production plans. 4) Conclusion: The conclusion summarizes the key contributions of the paper and highlights the potential benefits of the proposed approach for manufacturing companies facing uncertain demand and supply constraints. The authors also suggest areas for future research and development in this field.

A. Research Framework
The work step of this research is to first identify the affected factors in the APP system. Then build a demand forecasting model using the ANN technique which will be used as input to build the APP model. Finally, build the APP model using FL techniques within the DP framework. The research framework can be seen in Fig. 1.

B. Artificial Neural Networks (ANN)
The notion that underpins ANN is that the input, also known as the dependent variable, is passed through one or more hidden layers, each of which is composed of hidden units, or nodes, before it reaches the variable that is being measured as the output [27]. For the purposes of series data modeling and forecasting, the type of neural network model that sees the most widespread use is the single hidden layer feed -forward neural network [28]. www.ijacsa.thesai.org According to the standard concept, the connection between outputs (y t ) and inputs (y t-1 ,...,y t-p ) is as follows: Where: .., q) and w j (j = 0,1,2, ..., q) are the parameter that represents the model weight; p represents the number of input nodes, and q represents the number of hidden nodes.
In other words, the recurrent neural network receives the values y left as input, and it also has a hidden layer that is comprised of size q nodes [29]. In point of fact, then, the model executes a nonlinear functional mapping from historical data to those of the future:

( )
This considers that w is a vector of all parameters and f () is a function defined by the network structure and the weights for the connection. The ANN applied in this study is the default single hidden layer model using the feed-forward backpropagation algorithm [30], where the extension of the nodes number in one layer is equal to the input nodes number plus 1 (Fig. 2). The quantity of grids that were put in place, each having a starting weight that was chosen at random, and then averaged as estimates are calculated [31]. ANN technique is used in the demand forecasting process using Matlab R2017b software.

C. Dynamic Programming
Dynamic Programming is a strong formal instrument that may be utilized for the purpose of addressing a wide variety of multi-stage decision-making issues [32]. Since its origin in the middle of the 1950s by Bellman 1957, DP has developed into a common tool in a variety of fields [26], including but not limited to operations research, systems analysis, engineering, data analysis, control, and computer science, amongst others [22], [2], [33]. The fact that one only needs to solve a little fraction of each subproblem in order to complete DP successfully is one of its strengths [34]. This is because of Bellman's concept of optimality, which explains the situation. According to this, regardless of the decisions that were made in the stage before it, if the decisions that are going to be made in stage n are going to be a part of the overall optimal solution, then the decisions that are going to be made at stage n have to be optimal for all of the stages that come after it [34].
At each stage, n in the DP there are state variables, x n , and optimum decision variables, d n .
In stage n, there is a value returned by the function for each of the values xn and dn, r n (x n ,d n ). The result of the procedure once it has reached step n is x n-1 , the status variable for the stage n-1. The stage transformation function is responsible for calculating this result, x n-1 = x n + d n -D n which means inventory plus production minus demand (D n ). The optimum value function, denoted as fn(xn), is the combined total return beginning at step n in the state xn and proceeding to stage 1 in accordance with the best possible strategy.
In general, the most effective way to tackle problems with dynamic programming is to begin at the end of the process and work your way backwards to the beginning. This is referred to as recursion in reverse. The recursive connection that is presented here may be utilized in order to implement the concept of optimal solutions in the context of achieving the lowest possible total cost: Where ( ) is the total production and storage costs for the stage / month n. Production costs are the cost of production per unit multiplied by the number of units produced (d n ). Storage cost is the ending inventory for the month multiplied by the unit cost of storage. Mathematically, it is written as follows: Since there is no provision for backordering, it must fulfill the needs of the customers. That is, for the month n: Because there is a storage capacity limit of Wn at each and every stage n, the overall inventory at the conclusion of any given month cannot be greater than Wn. So that for every month it must be: It is imperative that the quantity generated in any given month does not go over the capability of production for that month, or: Beginning at stage 0 with the boundary conditions f 0 (x 0 ) = 0, a problem can be solved by working backwards through the stages until reaching the final stage, n. It is assumed that there are no products stored in inventory at the beginning and end of the planning period.

D. Fuzzy Dynamic Programming
A fuzzy set may be identified by its one-of-a-kind membership function, which is responsible for mapping each component of the X discourse environment to the interval [0,1].

̃ {( ̃( )) }
A function related to the degree of membership A determined by ̃( ) , -. In a fuzzy set ̃, ̃( ) is the degree of membership .
A fuzzy set that is defined on the uniform real numbers is denoted by the symbol A, is said to be a Triangular Fuzzy Number (TFN) if there are three arguments with the value ̃ (a 1 , a 2 , a 3 ) determined by the following membership functions: TFN may also be expressed in the form of an interval: [d1,d2], where ( ) and ( ) [35]. Fuzzy numbers can be processed mathematically fuzzy according to the method of representation. Representation in the interval for example ̃ , and ̃ , be a representation of two TFN numbers, the arithmetic operation is as follows: Fuzzy aggregate production planning problems are related to the uncertainty of demand, production costs and storage costs with the aim of producing minimal total costs during the planning period. Suppose an industry must produce goods from one period to another with different production costs and storage costs in each period. So a plan is needed to determine the amount of production per period with the lowest cost that takes into account the demand for each period.
In order to address fuzzy APP issues utilizing the fuzzy dynamic programming approach, one must first follow the methods that are mentioned below.
Step 1: Determine that there is an issue with hazy choice variables and then to state that the fuzzy objective function is going to be optimized using definite bounds.
Step 2: The problem that has to be addressed is then broken down into smaller subproblems or stages. Classify the fuzzy condition variables at each level, and then create the transformation function such that it is a function of both the fuzzy condition variables at the previous stage and the fuzzy decision variables at the stage after that.
Step 3: Then using generalized fuzzy recursive relationships we get the optimal decision for the problem. ̃( ) is the lowest possible sum of money spent on the previous n stages. ̃( ) is the optimal value (minimum cost) when the product is in the state x n with n stage again to reach the final stage. The optimal value equation of ̃ on condition x n can be obtained by selecting the appropriate decision on the decision variable d n that is: Range d n determined by x n , but x n defined by the events that took place in the stage before it. The return function will then assume its final shape in the subsequent stage: So that the fuzzy recursive equation is obtained as follows: Step 4: Create a suitable table to show the importance of the return function at each stage.
Step 5: Determine the overall optimal decision and its value.
The case study reported in this research is aluminum industry which processes aluminum plates for the automotive industry.

A. Demand Forecasting
Demand forecasting is done using the backpropagation feed-forward algorithm in the Matlab R2017b software. The input data used consisted of data on sales results, selling prices, total stock of goods, and prices for complementary products. The activation function in the hidden layer uses sigmoid and in the output layer uses linear (Fig. 3). The learning process uses a scaled conjugate gradient (trainscg) with parameters epochs 5000, sigma 5e-05, lambda 5e-07, goal 0.001 and the rest is default. From the training results obtained the mean squared error (MSE) of 0.00014347 at epoch 18 (Fig. 4) and the overall R value is 0.99074 (Fig. 5).
Demand forecasting stage with the backpropagation algorithm on ANN with input data obtained from historical data from 2016 to 2019 consisting of sales data (X1), selling price (X2), product stock (X3), and complementary product www.ijacsa.thesai.org prices (X4) . Data normalization was carried out using the X/Xmax formula (Table I).  After the training and data testing were carried out, a simulation was carried out. This demand forecasting uses ANN backpropagation with a two-layer architecture consisting of one hidden layer with five neurons and one output layer, the sigmoid activation function (logsig) on the hidden layer and linear (purelin) on the output layer. The comparison of simulation results with actual data can be seen in Fig. 6. The comparison of simulation results and actual data shows that the simulation results by ANN backpropagation are close to the actual data, there is only a slight difference which is not too significant.

B. APP Model under Uncertainty
The simulation results of demand forecasting are used as input for the APP model. This paper sets out for a period of six months, from July to December which is completed by beginning at the end of the process and working one's way back to the beginning (backwards recursion). Stage 1 is December, stage 2 is November and beyond. Units are in tonnes and costs are in million rupiah. Production capacity (P n ) each month the same, namely 600 tons and storage capacity (W n ) each month is 900 tons. Demand data, production costs and storage costs in the TFN are shown in Table II and the fuzzy representation data in the confidence interval (α) is 0.5 in Table III.    APP problem solving is divided into six stage according to a planning period of six months. Its objective function is to minimize total costs which include production costs and inventory storage costs during the planning period. The minimal costs at each stage are solved by equations:   The results of calculations using fuzzy dynamic programming obtained a minimum total cost in the last stage ̃ ( ) is [11142.5; 14517.5] with a mean of 12830 or twelve billion eight hundred and thirty million rupiah. Details of the amount of production and inventory per month can be seen in Table X.

IV. CONCLUSION
This paper presents an application of the use of backpropagation artificial neural network (ANN) to predict demand as input to an Aggregate Production Planning model that is compiled using a fuzzy dynamic programming (FDP) framework. Demand uncertainty, production costs, and storage costs are accommodated in the FDP framework. The prediction result of the number of requests using ANN backpropagation produces a prediction with an MSE value of 0.00014347, which means that the prediction generated by the ANN model is very close to the actual value.
The minimum total cost during the six months of the planning period calculated using the FDP framework is [11142.5; 14517.5] with a middle value of 12830 or twelve billion eight hundred and thirty million rupiah. The lowest production was in September, October and November with the amount between 300 and 400 tonnes. In July the production is between 350 and 450 tonnes. August and December with the same amount of production between 450 to 550 tons. Total product held in inventory was 50 tonnes each at the end of August, September and October.