Improving the Effectiveness of e-Learning Processes through Dynamic Programming: A Survey

E-learning has been widely adopted as an important tool for distance education, especially in these days of pandemic Covid-19. However, several problems/challenges have been reported in different processes of e-learning that need to be addressed for effective use of e-learning. These problems/challenges include development of student focused contents, giving learner partial control, addressing different learning styles, etc. Recently, several efforts have been made to solve e-learning process problems using dynamic programming techniques. Dynamic programming techniques divide a problem situation into several subproblems and dynamically solves each sub-problem based on student needs. Thus it allows student focused customization at each step and provides adaptive e-learning to support students with different capabilities. The objective of this study is to review different e-learning problems and challenges and how those can be addressed using dynamic programming techniques. We conclude by highlighting the importance of different dynamic programming techniques for different processes of e-learning. Keywords—e-learning; e-learning challenge; dynamic programming


I. INTRODUCTION
Developments in information and communication technology (ICT) have introduced E-learning programs which can be employed efficiently as an alternative to allow users to learn and teach new technologies and courses. E-learning refers to knowledge that uses electronic technology beyond the traditional classroom to access curricula. It can be applied to a course, program, or diploma that is entirely online [1]. E-Learning online courses are available 24/7, regardless of the time of day, accessible and reliable educational resources to students in geographically spread regions and time zones. This type of delivery enables the portability of training (i.e., tablet/iPad, laptop, cell phone for files or video assessments and links to webinars, etc.), allowing for easy learning on the go, particularly with the growth of networks, mobile use, and learning via computers or laptops [2]. There are numerous elearning systems and methods (known as learning management systems) that can be used to conduct online courses. Given the right tools, various processes can be automated, such as scoring or viewing reading materials. E-learning is often the most efficient way to adapt the learning to people's busy lifestyles, allowing individuals to gain new skills and develop their careers. In recent days, the importance of e-learning has been recognized, as the COVID-19 pandemic crisis has made e-learning mandatory in all institutions of learning such as schools, colleges and universities throughout the world [3]. In fact, The problems of e-learning can be summarized in five major issues such as: struggle for adaptability, technical problems, computer literacy, time management, and self-motivation. Therefore, Many methods have been used to improve the e-learning process that has helped to improve this area and increase its level of effectiveness; one of these methods is Dynamic Programming (DP). Dynamic programming techniques which are ideal for embedded systems are increasingly comparable to adaptive control techniques for dynamic systems. A growing number of researchers are inspecting and studying dynamic programming (DP) algorithm-based learning systems to solve and fix stochastic ideal control problems, arguing that DP provides the proper basis for combining planning results into real-time reactive dominance techniques, as well as for learning these techniques when the managed system is incompletely understood. This paper reviews how the DP is contributing to solving some of the problems in the field of e-learning through various studies. This paper's structure is as follows: as an introduction, we first present the definition of the DP and then provide a description of the DP. Next, the role of the DP in improving learning is discussed. The fifth section is a comparative analysis that concludes the studies presented in this paper, and the final section concludes the article.

II. DYNAMIC PROGRAMMING
Dynamic programming allows to overcome an optimization problem (e.g. optimizing, decreasing, or finding the total number of ways to do something). It is defined as a quantitative technical analysis that has been used for fundamental and sophisticated problems that require a series of decisions to be made [4]. In [4], the authors defined DP as one of the algorithms used to find the ideal value of a problem. This problem's solution is decomposed into several steps (subproblems) until the solution of a required problem can be considered a succession of interdependent decisions.

A. Steps of Dynamic Programming
DP problems require four phases [4]: • Break the basic problem into subproblems known as phases.
• Accomplish the last problem stage for all potential states.
• Returning from the last phase, resolve every middle phase. This is completed by defining ideal policies from that point to the terminus (final stage) of the problem.
• Get the ideal solution to the original problem by sequentially solving all the stages.
Using the DP, sub-problem results are stored. Therefore there is no need to recalculate these results the next time they are requested. This is an alternative to simple recursion, which requires repeating the solution operation each time the subproblem is encountered. In (Programming n.d.) address these phases as following: description,definition, computation and construction. knabsack problems and shortest paths are two problems that predominate in studies and research on computer algorithms [5]. The issue of knapsacks goes back more than a century, i.e., we need to calculate the amount of each item to be included in a collection for a set of objects, each with a weight and a value whereby the overall weight should be smaller or equal to a specific limit. The overall value is set as high as possible [5]. The shortest path's problem consists of obtaining a path between two vertices in a diagram or such that the total of the weights of its edges decreases, described in [6]. In this paper also we introduce the longest common subsequence problem and illustrate how to enhance learning area.

III. THE ROLE OF DYNAMIC PROGRAMMING IN E-LEARNING
DP is a useful technique for optimizing problems, which pursue the maximum or minimum solution since it allows to browse all possible sub-problems without defining a specific answer. It ensures that the methods used to solve or approximate algorithms are accurate and efficient. Futhermore, it provides a general means of modeling and solving such consecutive learning problems. This formulation offers value in three respects. First, the formulation of the DP allows the optimal solution to be calculated. This is usually only true for small problems [7]. Second, the formulation of the DP sometimes allows structural results to be shown in a theoretical way, which either provides insight into the problem and the behavior of the optimal policy or provides a characterization of the optimal system that can be directly exploited to find an optimal approach, as in sequential hypothesis testing. Third and finally, the DP sometimes provides heuristics that are beneficial for large-scale and complex learning problems. DP approved his role to enhance and optimize problems in learning during the following studies:

A. Shortest Path
Shortest path is the problem of getting the lowest rout in a graph from one vertex to other. In e-learning systems, a course is considered as a graph where each node represents a unit of knowledge. The shortest path's challenge is to find a routing between the peaks of the graph mentioned above so that the overall total of the edge weights is minimal. The Bellman-Ford algorithm (BFS) is one of the algorithms that may be used to solve this problem.
In [8] a DP approach is employed to choose the shortest and the optimale path in virtual learning environments (VLEs). This environment catches a large number of students to allow them to learn and study everywhere. The intention is that the location of the student is considered to be no longer a band [9]. In [8], the author applied an analytical hierarchy process (AHP) method for transforming parameters from qualitative to quantitative ones. The system of VLE is suggested on the basis of an in-depth personal learner profile. The objective is to select the best route compatible with a learner's attention and qualitative properties like attitude, motivational capacity, learning style, and knowledge level. The user in the system, is confronted with service providers who prop VLEs. Every provider of service provides learning subject and internships that are educated by different teachers. The layers are presented in Fig. 1 In the above figure, a network has been presented in which a dynamic process is implemented to choose the shortest route for each learner. The selection of the user depends on the qualitative criteria of every route. These paths must have a quantitative value to implement the DP. To achieve this, the qualitative criteria are converted into numerical values according to the user's preferences (the numerical values are shown in the table in the study paper). Then, the path preferences are related with numeral values. Three matrices are constructed in this study to calculate the ratio of the numerical values of pathways, which are estimated by the learning styles, attitudes, abilities, motivations, and knowledge levels of users based on the numbers of preferences. In this manner, the weights are calculated, and the ideal pathway for learners will be specified.
In [10], the author notes that they aim to estimate the prosperity and/or validation of education systems with a specific pipeline, unlike successful e-learning architectures. He suggests an evaluation procedure that consists of applying the DP process to model the user profile's ideal path problem and applying reliability to measurement the interconnections between users in a network of e-learning. This study has endorsed the effectiveness of DP through a comprehensive example. It is identical to the previous study, resulting from this example presented in Fig. 2 below [10]. In below Fig. 2, we can see the service providers number are five, the number of presented course are five and there are five teachers According to the calculation of numeric values, the ideal path as presented in Fig. 2 is 0-3-9-12-16 that means the group included of the 3rd provider of service, the 4th learning object, and the 2nd educator is identified to become the shortest and the optimal. DP contributes to solving Markov decision process to enhance the learning path of learns in [11]. The atmosphere for adaptive learning is random. Therefore, to establish an effective learning path, the Markov Decision Process (MDP) model is suitable. Besides, the best activity in a sample space [12] is observed. Therefore, the MDP is recommended to provide a structured format to describe a multi-stage decisionmaking process in a probabilistic context. DP approach utilized to resolve the MDP. The issue in this study, selecting the bestead learning path with the highest reward, this problem an optimal substructure it can be split into subproblems, for that DP is the most proper way of solving it [13]. It is almost similar to that of a traveling salesman problem who wants to find the shortest way to increase his bonus. The learning objects it will consider as nodes, and learners' learning style as routes between nodes (in this study, used the Felder-Silverman Learning Style). According to Markov's process, the selected path will offer the highest cumulative remuneration [14]. After constructing the DP code by MATLAB program, the shortest path among two learning materials (learning objects) are computed for all successive LO. Fig. 3 illustrates the result of this study [10].

B. 0/1 Knapsack
Knapsack problem is considered as a combinatorial improvement problem in which each item is related with a profit and a weight. This problem aims at maximizing the number of items in a knapsack by matching capacity of knapsack. The 0/1 knapsack problem can be declared as: offered a limited number of items and a knapsack, find the total revenue under the constraint that the overall weight of all selected items at maximum the weight boundary. However, present learning material is uniform and constant during each grade without concentrating on individual trait's or group differences. This problem makes materials of educational not attractive to some students. Establishing a progressive direction in compiling student books is a playful way of meeting students' demands and enhancing education efficiency. The study [15] In 2012 discussed the problem when the learner meets with a vast amount of content through the process of learning while he/she has a fixed time, and for this, he/she may sense disappointed and frustrated. The study proposed a novel procedure to aid learners get learning content more personalized to their characteristics and background in a playful manner; it used the 0/1 knapsack problem to choose the most suitable adaptive learning materials in a limited amount of time specified by the learner. There are various content categories in learners' books, like texts, exercises, examples, etc. These are known as elements. For each of these items or components, a specific part of the time allotted for teaching a book is set aside.
Students take the regular exam first, and their learning rate is evaluated. Then, the outcome of this test is converted into a coefficient. The time required for each of the upper elements to be taught determined by multiplying the original time allocated by each element's coefficient. For all the features, we can also measure time [10]. To provide the most appropriate learning objects to the learner in a restricted period, the 0/1 Knapsack problem has been applied. This problem is one of the most known combinatorial improvement problems. It is known as a classical NP-hard, which has an excellent search space.
The 0/1 knapsack problem (KP) is appropriately known as follows: We are given n items and a knapsack. Item i weights wi, and the knapsack has a weight boundary C. If object i is put into the knapsack, we will gain a profit p. The issue is to increase the overall profit under the constraint p that the sum weights of all selected objects are at most C. For that, the knapsack problem can be formulated as: Where p denot to the profit ,w refer to the weight and x is a binary variable indicating whether object i is selected or not, in the above Equation, p the constraint that requires to be convinced and realizes the profit of a feasible n-tuple, the following Fig. 4 illustrates this problem [10]. Each student has his/her account and personal profile. When coming to the application for the first time, many questions were faced by each learner to reach the personalities of learner, characteristics, and learning styles. This application, which is established in the suggested way, showed a significant effect on adaptive learning.
Dynamic programming has proven its role in universities through scholarship programs. Bursaries serve as an important motivation and play a vital role in inspiring students to work hard at the universities, exemplify, and strive in their academic career in higher education [16]. It is often viewed from the teaching perspective as an essential factor. Many research findings have shown that bursaries positively impact college performance and inspire students to pursue further education [17]. Various scholarships are sponsored by multiple donors, such as entrepreneurs, companies, citizens, etc. responsible for deciding the number of awarded persons and the cost allocated to any individual beneficiary. The university is involved for awarding the scholarships to applicants by the rate on departmental results. Yet university administrators are not allowed to split the current scholarships further into new scholarships with lower rewards.
In [16], they proposed a challenging DP approach to change the conventional process of decision-making for awarding scholarships. The aim is to figure out the ideal scholarship allocation with the greatest justice while taking into account practical channels and the requirement for fairness. This study explores a department's scholarship allocation process. A variety of scholarships are available to graduate students from a department who have done exceptional work in the course of their university studies. In two ways, multiple types of scholarships are different the number of the offered ones and the money allocated for these scholarships. For various scholarships, students can apply free of charge. The outcomes of the scholarship assessment depend on the scores received by the students in the previous year. A student's total score consists a score named basic, which is calculated based on coursework and some additional study findings provided by the academic accomplishments, such as journal papers. The score mentioned above is calculated in a scale to simplify student work assessment, whereas various forms of achievements are estimated. Also, to turn a particular work into standard ratings, a standardization procedure is implemented. For example, the score of a published paper can be measured in two components: Rating the impact factors of journals and the order of the authors. The study assumed that the department was responsible for the allocation of scholarships and the scholarship was distributed according to the rank of the applicants' final scores. To prevent unfairness in distributing scholarships, we add the limitation that at least no less scholarship should be given to students with a higher score.
Otherwise, the research takes into account other realistic constraints. For example, multiple students cannot share a  single scholarship; they can only be rewarded no more than once for a specific form of scholarship, various grants; students should be assigned all the grants. The ideal assignment is the method that guarantees that the sum of a student's scholarship is corresponding to the scale of the overall score.
The scholarship assignment problem was viewed as a join of two sub-problems. One is seeking the available assignments schemes that pass the constraints and the primary justice demand together [16]. The second, lies on the quantification of each sensible scheme's justice, then, choose the ideal strategy with the top justice. To handle the first subproblem, a way formed of knapsack sub-problems is advanced to allocate in sequence students to scholarships. In the knapsack sub-problem, every student's overall mark is considered the knapsack, and the valent mark of every scholarship is considered as the element placed in the knapsack [16]. The Gini coefficient to calculate unfairness of each feasible solution is added to resolve the second sub-problem. The equity of welfare distribution can be quantified by the Gini coefficient (variation among the real and ideal sum of scholarship earned by every single student) [16]. This study explained the proposed method through a small case: three scholarship kinds and five candidates as an example, as we see in the below Tables I, II  and III. Then, we can calculate the coefficient of Gini of the individual plans, as mentioned in Table IV which is indicated in the schedule IV: As the Scheme 1 Gini coefficient is the smallest, Schema 1 is the one with the highest equity as shown in the Table V. In this numerical example, scheme 1 is therefore selected as the last scheme of scholarship assignment. Dynamic programming helps universities in its strategic management for whole university or departments   inside it through multiple knapsacks. It popularizes the basic knapsack problem (KP) from a singular knapsack to many knapsacks with (probably) various capacities. The goal is to allocate each element to at most one of the knapsacks so that none of the capacity restrictions are assaulted and to maximize the overall profit of the elements placed in the backpacks [18]. When the university develops her strategic administration to plan and apply optimization procedures, this enables the university to be ranked in the education list [19]. In 2019, Dorota et al. in [19] proposed a model which is quantitative for university managing strategy as an entire or university section. This study's methodology is gathering qualitative methods utilized in the administration strategy and the quantitative multiple knapsacks.
This study covers only the building model and decisionmaking operation related to administration in small Polish universities but not yet implemented. This gives us the attraction standards of the department chosen by students.

C. The Longest Common Subsequence (LCS)
Finding the longest subsequence in the same order in the two sequences given,is consider the goal of LCS. This means determining the most extended sequence obtained by removing some items from the first original sequence and removing other items from the second original sequence. This Problem also has a different application in text editing and data compression [20].
Among the main tasks of e-learning is the customization of learning. As mentioned earlier, the interest of numerous researchers has been drawn to the customization of learning objects (LOs) due to the immense significance of the LO in the learning process [21]. LCS contributed to the adaption of learning through personalizes LOs for learners. Mahdi and Fattaneh in [21] see Students and educational resources are the two key learning entities, so educational programs should give both these two entities high priority. They have developed a robust algorithm which is genetic for educational slides, thus adapting as a particular material in the e-learning resources. This algorithm potentially implemented in each type of LO. Initially, a default LO series will be assigned to the learners. This series is diagram of LOs as node or vertex and relationship among them as an edge. This hypothetical sequence is defined with the aid of a skilled and then required from the learners to rearrange these materials, and reasons will be presented for each change.
Below, the action is taken to determine the rearranging of the sequence of LOs and create a new series of them.
• Insert a new LO or deleting from the LOs sequence default (stretched to add or remove a sequence of LOs).
• To interchange the two LOs with each other.  This study introduced a diversity of relationships among LOs in the following.
First Case: (independent relationship): Since the LCS considers a sequence in the default sequence that is not necessarily sequential; the sequences suggested by students can be placed in groups with a specific order. For instance, as we see in we assume that the sequences are as follows for two sets of students (Table VI): As shown in Table VI, a general pattern is found in both groups with a hypothetical LO sequence, and the variance is on the independents (LO 4 and LO 5); therefore, it is possible to examine the reactions of the groups on their LO sequences, and after the educational experts, the endorsement chooses a better sequence. It might alternatively be conveyed that both sequences are right, and any of the sequences can be used in developing the preferred method. For instance, the best sequence may be chosen (a more similar group) after evaluating the new individual's similarity with each group. Then it sequence suggested to the person. Any calculation employed to determine the sum of similarity between entities and the groups can be a measure of similarity [21].
Second Case (Precedence relationship): Let's assume the above example and that one group has determined the prevalent sequence as follows (Table VII): Fig. 5 shows that LO 2 is a precondition for the LO3, but the group in Table VII does not reveal this prerequisite relationship. This may be because the group has been unable to find the group has established the prerequisite relationship among LOs or the fake prerequisite relationship among LOs [21].
Third Case (Container relationship): if there are some LOs, everyone has much smaller LOs. The largest LOs have the required connection, but the interior ones are independent. In Fig. 6, For instance, the First and last LOs are selected. As shown in Fig. 6, the lesson's three primary topics are (Part 1, Part 2, and Part 3). Many sequences of LOs in each subject are not linked to each other, but the following part is expected to be articulated before starting. Both classes did not Fig. 6. The Relationship to the Container is Indicated by the Grey Circle [21].
completely recognize the sequences in the following table. We were unable to find any LCS from 21, 22, 23, and 24 in the first category, for example, due to the fact that its members are suggested sequences which are sufficiently different [21]. The proposed sequence groups are shown in Table VIII and the  final result of case 3 displayed in Table IX. Case 4 (relationship unknown): The effect of dissimilar granularity levels of LOs is investigated. Most popular strategies teachers apply to prepare instructional material is to choose many sections from distinct marital learning such as leaflets, handbooks, or books. Because each of these various tools explores the topic from different viewpoints, there are often no standard sections between selected parts; it is a crucial and time-consuming job to design a series of these instructional materials [21] as shown in Fig.  7. To overcome this problem, we suggest to the student to be able to offer or suggest a sequence for these sections (chapters), next we created the sets based on this proposition. For each set, we need another sequence for each chapter's parts to be suggested by its member (such as case 3), and novel groups are created in ones that already in the past stage [21].
In this study, LCS helps define the fitness function; this function for each set is equal to how much the organ is similar to others in that set. This propinquity gains from computing the LCS of student suggested sequence for LOs with other groups' organs. The findings suggest that this algorithm is efficient, and its findings could be used in different learning activities [21].
Another application of the dynamic programming using the LCS is to detect the plagiarism from a specific text or document, this method was proposed by Mahalakshmi S. et al. [22] in which they presented a system based on a symbolic execution next to the reasoning of the weakest requirement in order to find the distinction of paths and to grasp the semantics of execution paths. Indeed, the proposed method uses the LCS to provide a line-by-line comparison between the text-based search and the string by passing the file that is the object of this comparison through the following steps, starting with reading and converting the file, and then finding the shortest path between the data in the file and the data in the repository. Finally, extract the similarity report using the LCS algorithm based on recursivity. The proposed method proved to be very efficient compared to the existing methods. Alexey Sorokin in [23] has applied the LCS method differently to predict the word forms. In fact, LCS is used to extract summary paradigms from input pairs, inflected word forms and suffix/prefix characteristics to automatically predict the paradigm. The proposed algorithm is based on a mixture of affix characteristics and ngram character models, which significantly improves performance, especially for language processing. In [23], A. Sorokin has employed several tasks to perform his method starting by the encoding stage which is consist in encode the input sequence with some label and trying to predict these labels using machine learning algorithms to convert the input into paradigm abstract. Then, a finite automata is used to extract the LCS. Furthermore, they perform the automatic inflection since the proposed method includes in the feature set the lemma suffixes and the prefixes. Finally, they use the ngram score enhancement the model score.
The LSC was also used to identify the sentence parallelism in student essays, W.Song et al. [24] proposed a robust method to automatically detect and identify the sentence parallelism using machine learning algorithms with the help of LSC. In their work, they build an annotated essay dataset. Furthermore, they construct a feature vector by using the strategies of the combination of generalized word alignment and the alignment measures between word sequences. W.Song et al. [24] started by collecting a dataset of 500 essays made by students, then they annotated the dataset to make the classification and to generate the classification report. The features extracted from the annotated dataset will be fed to a machine-learning algorithm to generate a prediction model. They made a study on various alignment measures, because the parallelism is quite similar to the alignment, to assess the alignment of sentences. They employed a several strategies to generalize the word alignment based on two features which are the semantic and the syntactic properties. Interactions of alignment metrics with word alignment strategies produce the features to show the alignment between sentences. Furthermore, The LCS is used to solve the problem of the parallel sentences by the use of dynamic programming. To assess the efficiency of the suggested method, they tested several algorithms on the basis of dynamic programming, and they reported that they achieve an f-score up to 82% in the best cases.
An interesting application of the LCS was proposed in [25] by A.Flores-Mendez et al. [25], they proposed a new method to verify the dynamic signature by the use of LCS and genetic algorithm. In their approach, they based in two major steps : learning and verification by the use of the LCS. The learning process is made by solving a polynomial equation and by measuring the distance between the current signature and a set of data to verify if they are similar or not. In the other hand, the verification process was about calculating the shortest path to be assigned with a very similar signature based on the learning stage. Furthermore, the genetic algorithm was used to build a knowledge model to make the process inspired by the natural mechanism. The authors reported that the accuracy of the proposed method was outstanding, and it achieves more than 90%.
Multiobjective Dynamic Programming (MODP), which based basically on traditional DP, is advanced as a mechanism for resolving issues (problems) that involve incompatible goals that obey to features of DP. Significant progress in multiobjective dynamic optimization has been made in the last two decades. MODP considers One of the most provocative topics, is A normal expansion of DP is its utilized in conjunction with sets of fuzzy [26]. one of the applications of mode is resource allocation, this also can help e-learning systems to allocate its resource [26].

IV. COMPARATIVE ANALYSIS
DP is considered one of the methods that contribute to improve e-learning through different problems. The DP technics are used in many fields to solve several issues in different case of studies. In the present paper, we discuss three major issues (shortest path, knapsack and LCS) that help improve the learning sector. Based on the studies that we have discussed before, we find that the LCS is used generally to perform a classification process. Indeed, the LCS technique allows finding the nearest samples by finding the shortest paths based on the semantic, which is different from the shortest path, which is based only on the distance. In the other way, the knapsack is used to optimize the choice of the sample; thus, it employed always to maximize the gain. Table X summarizes the studies introduced above.

V. CONCLUSION
Dynamic programming is a method of optimization depends on the precept of optimality realize by Richard Bellman. It is considered one of the solutions and methods that have contributed to solving many problems in various fields, especially in e-learning. This survey explained how DP contributed to supporting education through three types of problems (shortest path, knapsack, and long common sequence), where during the shortest path problems DP supports adaptive e-learning through generating the suitable path of different learners with different learning style and provide the appropriate learning materials as we shown in first three studies. knapsack problems contribute in supporting higher education through the scholarship assignments and assist to select the suitable learners to gain scholarship in a fair way as introduced in fourth study and knapsack problems aid universities in strategic plan and administration like in fifth study. Last studies illustrates the role of LSC problem to detect the most suitable sequence of Learning Objects from the ones introduced by the learners to personalized the learning materials in effective way and this enhancing adaptivity in e-learning. Also, LCS contribute in some application that help learners such as plagiarism detection and verifying the dynamic signature. All studies have proven how DP has contributed to improving and supporting e-learning.