A Review of Recommender Systems for Choosing Elective Courses

In higher education, students face challenges when choosing elective courses in their study programmes. Most higher education institutions employ advisors to assist with this task. Recommender systems have their origins in commerce and are used in other sectors such as education. Recommender systems offer an alternative to the use of human advisors. This paper aims to examine the scope of recommender systems that assist students in choosing elective courses. To achieve this, a systematic literature review (SLR) on recommender systems corpus for choosing elective courses published from 2010–2019 was conducted. Of the 16 981 research articles initially identified, only 24 addressed recommender systems for choosing elective courses and were included in the final analysis. These articles show that several recommender systems approaches and data mining algorithms are used to achieve the task of recommending elective courses. This study identified gaps in current research on the use of recommender systems for choosing elective courses. Further work in several unexplored areas could be examined to enhance the effectiveness of recommender systems for elective courses. This study contributes to the body of literature on recommender systems, in particular those applied for assisting students in choosing elective courses within higher education. Keywords—Recommender systems; elective courses; data mining algorithms; systematic literature review; higher education


I. INTRODUCTION
Looking through the current lens, in and post COVID-19, it is clear that higher education institutions (HEIs) have to change the way they engage with students from the traditional methods to an online or a blended approach. Popenici and Kerr [1] propose that it is time for HEIs to reimagine their function and pedagogical models in a new paradigm with technology at the centre. This calls for increased application and adaptation of artificial intelligence, machine learning and data mining tools to equip the education sector [2].
Many degree programmes offer elective courses in addition to compulsory ones. The courses that students fail to complete include both compulsory and elective courses. Students chose by a student elective courses based on their interests. Predicting student grades in the courses, they will enroll for is useful for guiding students and allowing them to make informed choices regarding compulsory, and elective courses [3].
In higher education, students are faced with difficulties when choosing elective courses. A survey of first-year students at the University College Dublin showed that almost half of the students selected elective courses outside their major because they perceived the courses to be exciting. Some of the difficulties emanate from the limited capacity in some elective courses as well as timetable clashes with compulsory courses which make students choose other elective courses [4].
Finding the most suitable elective course from the available ones can be achieved by using recommender systems [5]. By analysing data on the courses that students completed, it is possible to categorize a student's interests. The ability to predict student enrolment patterns for courses provides an opportunity for HEIs to be effective in allocating resources and providing a high-quality learning experience [6]. Predicting student grades in future courses before they take them is an essential tool that can be used to assist students with choosing elective courses [3].
The purpose of recommender systems is to recommend a product to a user that would possibly interest them based on the user profile [7]. A typical recommender system uses three elements: a user, item and rating. The recommender system attempts to predict a rating that a particular user would provide for unrated items [8]. Recommender systems use different types of input data which are placed in a matrix with one dimension representing users and the other one items of interest [9].
The rest of this paper is organised as follows: Section 2 provides a brief literature review; Section 3 discusses the methodology that was followed for the study; Section 4 presents the findings of the study and proposes work that needs to be considered in the field of recommender systems for choosing elective courses; Section 5 discusses the implications of the findings and suggests new trends in the field that can enhance recommender systems and Section 6 summarises the paper.

II. LITERATURE REVIEW
Laghari [10] warns that poor course selection can cause delays in completing a qualification because students have not completed prerequisite courses, or they have missed the minimum credit requirements for the qualification. Selecting the right elective course is vital for the student to complete their degree programme [11]. Choosing an elective course is influenced by several factors such as the student's personal and academic interest as well as institutional regulations that govern when a particular elective course can be enrolled for [6]. www.ijacsa.thesai.org O'Mahony and Smyth [4] identified the following factors that influence students' choices of elective courses: interest and academic goals, career goals, course pre-requisites and corequisites, ability to progress with their study, difficulty and format, of course, awareness options, availability of places, and timetable clashes. Other factors that influence the choice of elective courses include the number of compulsory and elective courses, the number of credits in a course and the maximum number of students that can be enrolled in a given course [6].
Machine learning tools and techniques have caused disruptive innovation in the way that most industries operate and education has not been spared. Artificial intelligence is defined as "computing systems that are able to engage in human-like processes such as learning, adapting, synthesizing, self-correction and the use of data for complex processing tasks" [1]. Machine learning is considered as a subfield of artificial intelligence. Machine learning involves the development of algorithms used to automatically make sense of data to adapt and learn from experience [12].
According to Gollapudi [7], there are many ways of grouping machine learning algorithms. One such method is the use of model-based grouping. In model-based grouping, machine learning algorithms can be classified into one of the following classes: association rule-based, Bayesian methods based, clustering methods based, deep learning-based, decision tree-based, dimensionality reduction based, ensemble method based, instance-based, kernel methods based and regression analysis based.
Data mining is an umbrella term for two separate processes: knowledge discovery and prediction. Knowledge discovery entails providing information in a form that can be understood by end-users and prediction allows the foretelling of future events [13]. Machine learning and data mining are different in that machine learning focus on using general knowledge, while data mining focuses on discovering new knowledge [7]. A sub-field of data mining with a focus on applying data mining tools and techniques is educational data mining (EDM). EDM can be defined as a process of applying computerised methods to identify patterns in educational data that are hard to detect because of the volume of data [14].
Recommender system techniques use different classifications based on the data that are used as input for the recommendation. There are four broad classifications of recommender systems: collaborative filtering (CF), contentbased, knowledge-based and hybrid approaches [8] [12] [15].

A. Collaborative Filtering
CF is the most widely used recommendation technique because of its power and simplicity [12]. CF uses data about users and items. A recommendation is made by analysing relationships between users and interdependencies among items to identify new user-item associations [16]. CF techniques can be classified as memory-based and modelbased. Memory-based refers to the use of user-based algorithms and item-based algorithms. User-based algorithms produce predictions for a given user by first identifying users with similar choices to the given user and then calculating the most frequently rated items that the given user has not seen [17]. In model-based algorithms, models are used to predict the ratings of unrated items by learning intricate patterns based on training data and using these patterns to make predictions [12].

B. Content-based Filtering
Content-based filtering techniques are based on the idea that users will prefer items that are similar to items that previously offered them enjoyment [12]. Content-based filtering depends on data about users and categories that have been assigned to the available item descriptions [16]. Contentbased filtering allows for the creation of a profile for each user to characterize its nature. This enables an association between a user and matching categories to be made by calculating a set of items that are most similar to items already known by the current user [9] [16]. Content-based filtering has an architecture that consists of components, such as item representations, user-profiles and the ability to learn a user model [12].

C. Knowledge-based Filtering
Knowledge-based filtering use domain knowledge to generate recommendations. This knowledge is made up of rules, metrics and items. Depending on the given user requirements, rules are used to describe the best approach to use to make a recommendation [16].

D. Hybrid Techniques
Hybrid techniques combine the above-discussed approaches to create a unified model that possesses characteristics of all approaches. The use of the unified model helps to mitigate certain limitations of the above approaches [12]. Hybrid techniques are a common feature because they provide opportunities to achieve better accuracy than the techniques mentioned above [16].

E. Matrix Factorisation
Matrix factorisation (MF) techniques can overcome the problem of data sparsity by employing dimensionality reduction to improve the model's ability to generalise [18]. MF can be used within CF recommender systems to achieve better levels of accuracy than those achieved by nearest neighbor techniques [9]. There are a variety of matrix factorization models and combinations in use today. These include singular value decomposition, PMF, non-negative MF, probabilistic sparse MF, Bayesian probabilistic MF and general probabilistic MF [18].
This paper aims to examine the scope of recommender systems for choosing elective courses. The study seeks to survey the landscape and determine the state of recommender systems for elective courses in higher education and to identify emerging technologies that could be explored to enhance recommender systems. To achieve this objective, a review of relevant literature on recommender systems for recommending elective courses was conducted.

III. METHODOLOGY
Kitchenham and Charters [19] define an SLR as "a means of evaluating and interpreting all available research relevant to www.ijacsa.thesai.org a particular research question, topic area, or phenomenon of interest. Systematic reviews aim to present a fair evaluation of a research topic by using a trustworthy, rigorous, and auditable methodology". An SLR involves analysing relevant primary research studies by identifying, evaluating and interpreting corpus.
For this study, the SLR method proposed by guidelines of Kitchenham and Charters [19] is used together with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) principles for the selection of the articles [20]. This section outlines the three steps, followed in performing the SLR study. In the first step, the need for performing the SLR study is identified. Then, a review protocol for conducting the SLR study is developed. In the third step, the process followed in conducting the SLR is described. The reporting of the findings of the study is described in Section 4: Results.

A. The Need for a Systematic Literature Review
Recommender systems are gaining prominence in the education sector. The search of the literature revealed that researchers had conducted systematic literature reviews (SLR) on recommender systems. Iatrellis, Kameas and Fitsilis [21] conducted an SLR study on academic advising systems and its impact on education. This study covered work published between 2008 and 2017. The study found that academic advising systems were used for choosing programs/majors, selecting courses and long-term academic planning.
Rivera, Tapia-Leon and Lujan-Mora [22] conducted a literature review on recommender systems in education. The study revealed that recommender systems are used to address different challenges in education with the majority of studies focusing on academic choice. During the review, no SLR studies of recommender systems for elective courses were found. Thus the motivation for this study is to identify, evaluate and analyze relevant literature on recommender systems that recommend elective courses in higher education.

B. Development of a Review Protocol
Kitchenham [23] states that a review protocol is essential as it defines the method that will is to undertake the study. The following steps describe the development process for the review protocol for the study at hand.

1) Identify the research goals and research questions:
The goal of this study was to conduct an SLR in selecting elective courses in higher education. The research sought to answer the following questions. a) Research question 1: What is the state of recommender systems for elective courses? b) Research question 2: What emerging trends in data mining should be explored to enhance recommender systems for elective courses?
2) Identify keywords: The literature search terms comprised of the following words and combinations: "recommender systems", and "recommendation systems". Kitchenham and Charters [19] proposed the use of Boolean operators such as "AND" and "OR" for refining the keyword search string. For this study, the logical operator 'OR' is used to join the identified keywords, and the 'AND' operator is used to combine the keywords in the phrase. The study used the following search string: [("recommender systems" OR "recommendation systems" AND "elective courses")].
3) Identify the sources: Specific online databases and search engines for were searched for research articles related to recommender systems in higher education with a focus on assisting students in selecting elective courses. These included IEEE Xplore, ACM Digital Library, Science Direct, Emerald Insight, EBSCOhost and Google Scholar. The authors used these online databases and search engines because they assumed that these were the main sources for collecting relevant literature on recommender systems.

4) Identify the inclusion criteria:
The inclusion criteria for this study are as follows: a) Articles that satisfied the keyword conditions. b) Articles that are written in English. c) Articles published between 2010 and 2019.
d) The articles focused on selecting elective courses. 5) Study quality assessment: Kitchenham [23] asserts that it is critical to assess the quality of primary articles. The following questions were used to measure the quality of the articles to be included in the final list.
a) Is the research article focussed on recommender systems for recommending elective courses? b) Is the research article a primary study? 6) Identify the data extraction strategy: The data extraction strategy involved extracting the following information from each research article: author and year, name of journal or conference proceeding, the objective of the study, the size of dataset used, the recommender system approach employed, the data mining algorithm used and the results of the study.

C. Conducting the Review
In the first phase, the search string was applied to the online databases and search engines. The search string was applied on all metadata and obtained 16 981 research articles, as shown in Fig. 1.
Next, refined the search was refined to only the article titles, which yielded to 3 021 papers. Filters were then applied on the online databases and search engines to exclude articles not written in English, non-peer-reviewed articles, articles not published in journals and conference proceedings and articles that are not full access. A total of 2 897 articles were excluded leaving 124 articles that were analyzed for the subject matter, leading to further exclusion of 28 articles. www.ijacsa.thesai.org The references and abstracts for the remaining 96 articles were then uploaded to Rayyan (https://rayyan.qcri.org), a free web and mobile app for screening articles when conducting an SLR study. According to Ouzzani, Hammady, Fedorowicz and Elmagarmid [24], Rayyan is used to speed up the initial abstract/title screening of articles and also allows researchers to collaborate when performing SLR studies. It is easy to pick up duplicates in articles referenced in more than one database when using Rayyan. The titles and abstracts of the 96 articles were then analyzed on Rayyan and excluded 72 articles leaving 24 articles for analysis. These articles passed the quality assessment by having yes as the answer for both the quality assessment questions.
The 24 articles included for the final analysis were then uploaded into an online SLR software -SysRev (https://sysrev.com) to extract the necessary data. The following labels were created to extract data from each article the year of publication, name of journal or conference, country, size of data set used for testing, recommender system technique used and data mining method used. Information about the objective of the article and the results obtained were also extracted. Table I contains the quality assessment results for the 24 reviewed articles. Based on the performance of one course, the study was able to recommend the most appropriate elective courses. [34]

Cybernetics and Information Technologies
To predict student performance in courses using a recommender-based approach and a regressionbased approach.
1268 student records MF ANN, decision tree, SVM, logistic regression The regression-based approach performed better than the recommender-based approach. [

35] Procedia Computer Science
To recommend elective courses based on previous grades.
1 000 student records Not provided ARM The system was tested on 100 students and achieved an efficiency of 90%. [5] Proceeding of 14th International Conference (Lecture Notes in Computer Science) To broaden the range of elective courses that students are aware of by adding diversity to the recommendation process.

student records
Hybrid approach Vector Space Model Content-based The system improves recommendation diversity than content-based and hierarchical taxonomy systems as module descriptions make recommendations more meaningful. [36] Proceeding ARM can be used to recommend elective courses for students. The system did not provide recommendations when the matching rule was less than 50%. [45]

Journal of Theoretical and Applied Information Technology
To use a context-aware recommender system which recommends undergraduate programs to students based on academic performance.
3 421 student records CF Naïve Bayes, J48 The use of a rating matrix is useful when using contextual information. The effectiveness of the recommender system was 98%.

IV. RESULTS
In this study, an SLR is performed to ascertain the state of recommender systems for choosing elective courses and to identify emerging technologies for recommender systems. After the screening process using Rayyan, 24 primary studies were selected. Fig. 2 shows the topics and themes covered in the 24 articles reviewed. The size and the frequency of the topic or theme show its prevalence in the articles. The results of this SLR are structured according to the two research questions.  Table I. The steady increase in the number of articles in the years 2016-2019 shows that there is more interest in recommender systems for recommending elective courses from researchers. Fig. 3 shows the number of articles published each year.

A. The State of Recommender Systems for Elective Courses
2) Publications type: All 24 reviewed articles were published in different journals and conference proceedings. 67% of these articles were published in journals with the remainder being published in conference proceedings, as shown in Fig. 4. Interestingly, all the 24 reviewed articles were published in different conference proceedings, with only two articles being published by the same journal [31] [35]. The fact that research on recommender systems on elective courses is published in both journals and conference proceedings highlights the interest this subject has among researchers.
3) Objectives of the study: The analysis of the articles shows that there are a variety of ways of performing the task of recommending elective courses. These include making recommendations based on the student's background and marks, broadening the range of elective courses available to students, and providing descriptions of the elective courses other than just the name of the course. The variety of ways of performing the task of recommending elective courses has increased over the years showing that this field is growing.

4) Dataset used:
In terms of the datasets used, studies that used datasets that consisted of students and courses accounting for 37.5%. Another 37.5% of the articles used a dataset consisting of students and course data. Articles that used a dataset consisting, of course data without providing student details accounted for 8.3%, and one article (4.2%) used 16 features which were not specified to be either relating to students or courses. Lastly, three articles, representing 12.5%, did not specify the size of the dataset used.
The analysis of the articles revealed that recommending elective courses can be done by using recommender system techniques, data mining techniques or both. 70.8% of the articles reviewed used a combination of recommender system techniques and data mining techniques. 16.7% used data mining techniques to make a recommendation, and 12.5% used recommender system techniques to make a recommendation. It was also interesting to note that all the reviewed articles published in 2019 utilised both recommender system techniques and data mining techniques.

5) Recommender system approaches:
The analysis of the results shows that the most widely used recommender system technique is CF. of the 20 papers that utilised recommender systems, 45% used CF, 10% used the hybrid techniques, and 10% used MF. The rest of the articles used, ARM, fuzzy clustering, knowledge-based, rule-based and multi-armed bandits (contextual). Also, the results show that different and new techniques are being applied to recommender systems. These techniques include ontology modelling, context-aware filtering and goal-based.
6) Data mining algorithms: This review revealed that several data mining approaches were applied in the reviewed articles. These approaches include k-nearest neighbors (k-NN), k-means algorithm, MF, ARM, recurrent neural networks, multilayer perceptron, item and user-based CF, and ANN. Interestingly, some articles used custom algorithms.
The first research question was to show the state of recommender systems for recommending elective courses. This section has highlighted the state of recommender systems for recommending elective courses using these sections: publications per year, publications type, objectives of the study, size of dataset used, recommender system approaches and data mining algorithms used. The analysis shows that recommender systems are useful for recommending elective courses. In the research articles where empirical evidence is given, the efficiency, precision or accuracy rates are greater than 90%, indicating the effectiveness of these recommender systems.

B. Emerging Technologies for Recommender Systems
The second research question for the study sought to establish the emerging trends in data mining that can be explored to enhance recommender systems for elective courses. The following paragraphs discuss the emerging trends based on the analysis of the reviewed articles.

1)
Most studies on recommending elective courses suffer from limitations related to the use of structured data. This SLR study has shown that several unexplored areas could enhance the effectiveness of recommender systems for elective courses.
2) Several studies reported on the fact that some students did not take the recommended elective courses. This could be to do with acceptance or lack-of for recommendations drawn from recommender systems. Researchers could attempt to incorporate models to improve the acceptance of recommender systems by students, which may result in them choosing the recommended elective courses.
3) Another critical area that needs to be considered is the selection of the right programme or discipline before recommending the correct elective course. Recommender systems should start with guiding students on the qualification path based on their interests. 4) Schnabel, Bennett and Joachims [46] suggest that user feedback can increase the learning accuracy of recommender systems. Using the information foraging theory, the authors prove that foraging interventions are complementary to improving algorithms and result in more effective recommender systems. There is a need to consider allowing students to provide feedback on the recommendations made by the recommender systems. Such feedback could be solicited at the beginning of the semester and retrospectively at the end of the semester.

V. DISCUSSION
This study reports an SLR regarding recommender systems for elective courses. This study aimed to ascertain the state of recommender systems for elective courses. It also sought to establish the emerging trends in data mining that can be explored to enhance recommender systems that assist students in choosing elective courses. This study reviewed 24 articles on the corpus and reported the results using different themes. The results showed that research on recommender systems has been increasing with the majority of the articles published in journals. Also, recommender systems are used to address a myriad of challenges faced by HEIs. This finding is not surprising given the impact recommender systems are having in other fields such as commerce, medicine and entertainment. Currently, HEIs have at their disposal vast amounts of data, both structured data and unstructured data obtained from social media [12]. There are possibilities for the task of recommending elective courses to incorporate structured data and unstructured data. Another challenge that most recommender systems reviewed face is the assumption that past student performance is a determinant of future performance. This is not always the case as social factors could have influenced past performance. It could be useful if recommender systems could incorporate data from social media.
The results showed that a variety of datasets were useddatasets on courses, students and a combination of both. The size of the datasets ranges from small to large, with data sourced from institutional repositories and other data sourced from survey questionnaires. The analysis of these articles shows that there are papers focused on recommending elective courses and others focus on recommending courses and predicting grades that the student would likely obtain should they choose the recommended elective course. Thus, this development is considered vital as it addresses one of the challenges grappling higher education-student's poor performance.

Journals
Conference proceedings (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 9, 2020 294 | P a g e www.ijacsa.thesai.org CF was the most widely used recommender system technique. Furthermore, several data mining algorithms have been employed to address the challenge of recommending elective courses. The variety of data mining algorithms may, in part, be attributable to the breakthroughs in terms of big data, data analysis and data science. This is in line with the findings of Jembere, Rawatlal and Pillay [47] that EDM provides numerous prediction tools that can be used to guide students on choosing courses.
Recommender systems must not only focus on broad outcomes such as courses but also on recommending learning resources and activities that will assist students in passing the recommended course. Such recommendations can take into consideration the student's needs, interests, preferences and past activities [14]. Fourthly, none of the reviewed articles touched on the aspect of Big Data. It is crucial for recommender systems to incorporate Big Data techniques as HEIs are producing vast amounts of Big Data. Higher education is changing post-COVID-19, and part of the changes will include online learning, which will produce more Big Data for analysis. This is in line with predictions by Popenici and Kerr [1] that HEIs need to reimagine their function and pedagogical models in a new paradigm with technology at the centre.
The analysis of the recommendations of the articles highlighted the direction that recommender systems for elective courses are taking. Only a handful of the reviewed articles utilised ensemble methods. Ensemble methods combine many models that are built independently to use the combined models to make predictions. The individual models that are combined are referred to as weaker models because the results of these models cannot make the required task on their own [7]. Two standard ensemble methods are bagging and boosting. Bagging (bootstrap aggregating) as a concept was introduced by Breiman [48] to reduce the variance of a predictor. Bagging is a simple ensemble method in which many independent models are built and combined using some model averaging techniques. Bagging is akin to improving existing methods by adding a loop in front that selects the bootstrap sample [48]. Boosting is an ensemble method in which the models are not made independently, but sequentially [12]. Ensemble methods provide an opportunity to build algorithms to recommend elective courses and algorithms to predict the grades that a student is likely to achieve for the recommended course.
Deep learning is defined as "a class of machine learning techniques that exploit many layers of nonlinear information processing for supervised or unsupervised feature extraction and transformation, and for pattern analysis and classification" [12]. Deep learning is an area of machine learning that deals with building more complex neural networks to solve problems classified under semi-supervised learning and operates on datasets that have little labelled data. Some of the widely used deep learning techniques include convolutional networks, restricted Boltzmann machine, deep belief networks and stacked auto-encoders [7]. Deep learning can be employed in a bid to create more accurate user profiles. These user profiles are central to the problem of recommending elective courses.

VI. CONCLUSION
Recommender systems employed to recommend elective courses to students are gaining traction. This growth can be attributed to the rise in the effectiveness of recommender systems that recommend products and services in sectors such as commerce and entertainment. In this paper, 24 articles on recommender systems aimed at recommending elective courses to students in higher education are reviewed. This review offers some insight into the state of recommender systems in this domain. Through this SLR, the recommender systems techniques and the data mining methods used in these papers to make recommendations were identified. The review revealed that several recommender systems approach and data mining algorithms are used to achieve the task of recommending elective courses. More importantly, this study has suggested emerging trends in the field that need to be explored by recommender systems to improve their effectiveness. These include the incorporation of acceptance models to increase the acceptance of recommendations, the effectiveness of user feedback. There is also a need to consider recommendation systems that begin with recommending the qualification path. This review is useful as it summarises current trends and makes suggestions on the future of this field of recommender systems for recommending elective models.
ACKNOWLEDGMENT MM conceived the concept, performed the literature review and drafted the original article. WD performed critical revision and approval of the submitted version. BP performed critical revision and approval of the submitted version. All authors read and approved the final manuscript.