Usability Evaluation of Open Source Learning Management Systems

Advancements in Information and Communications Technology has enabled learning to be conducted online frequently through Learning Management Systems (LMS). The use of Learning Management Systems (LMS) as tools for learning in the present Internet age is seen as an important solution to remedy major problems particularly faced by higher education instructors, students and universities. However, any quality and usability related information regarding such widely used learning management systems are rarely encountered in the literature. The main objective of this study is to evaluate the system quality of the top five widely used open source learning management systems through the external characteristics of ISO/IEC 9126 quality standards evaluation model for Moodle, ATutor, Eliademy, Forma LMS and Dokeos with two experts. ISO/IEC 9126 quality model is adequate for evaluating important system quality metrics. Results highlighted in detail a set of usability and quality issues that are associated to external characteristics for each open LMS which require further attention of developers, educators and researchers to improve the quality of learning. Keywords—E-learning; ISO/IEC 9126; learning management systems; quality model; usability evaluation


I. INTRODUCTION
Advancements in Information and Communications Technology (ICT) has enabled learning to be conducted online usually through Learning Management Systems (LMS). The integration of ICT in learning and its processes has resulted in increased improvements in the quality of learning generally by using several learning techniques all implemented when developing Learning Management Systems [1]. Learning Management Systems refer to different software packages designed to assist in the delivery and management of learning resources, materials and contents to students, usually via online/web platforms [2]. Basically, Learning Management Systems provide educators to create, deliver, supervise and monitor the participation of students as well as assess their performances [3]. They provide a platform for learning and obtaining knowledge at any time regardless of the geographic location of the users [4]. There are many Learning Management Systems that provide education instructors with variety of different options to select from [5]. Other Learning Management Systems provide features for assessing the learning progress of students, student registration, tracking and delivering of educational resources, materials and contents [2]. The use of Learning Management Systems as tools for learning in the present Internet age is seen as an important solution to remedy some of the problems faced by instructors, students and educational institutions in general [1]. Presently, different Learning Management Systems are implemented using different Internet technologies. The most common Internet technologies used are; (i) Open source, (ii) Cloud computing and (iii) Mobile based.
However, the most widely used technology when it comes to Learning Management Systems is the Open Source [6]. This is because open source software are tools whose source codes are available, can be modified to suit the requirements of the user and can easily be made available to the general public at [7]. This enables every institution that wishes to use any open source Learning Management System to simply download the source codes and modify them, which will enable the LMS to function according to the institute's requirements or preferences. With focus on open source Learning Management Systems, this study aims to conduct an evaluation study to compare the system quality of the top five most widely used open source Learning Management Systems available using the ISO/IEC 9126 quality evaluation model.
The top five open source Learning Management Systems subject to this study are: (i) Moodle, (ii) ATutor (iii) Eliademy, (iv) Forma LMS, and (v) Dokeos [8]. The details of the aforementioned Learning Management Systems are introduced below: Moodle: is a learning management system which is open source and free online learning platform for K12, higher education and for workplace that enables collaboration and engagement ATutor: is another open source based LMS which has variability and extended functionality in module features. The content could be easily managed, packed and modified in standard web environment.
Eliademy: is web based learning platform where content could easily and flexibly be created, communicated and maintained. It is free as compared to its commercial competitive rivals such as Moodle and Blackboard. DOKEOS: is an e-learning management system for managing content rich, flexible and effective learning and teaching experiences. DOKEOS works not only on desktop computers but also on mobile environments where no installation is required with always up to date.
The main aim of this study is to conduct an evaluation study to compare the usability of the top five open source Learning Management Systems available using the ISO/IEC 9126 quality evaluation model. To achieve this aim, a detailed study needs to carried out, which will take into consideration all the system quality and usability characteristics for each of the selected open source Learning Management Systems and then compared them with the aim of determining which amongst them has the finest system quality characteristics.
There are many open source Learning Management Systems that are available for use presently in the world. They offer very similar functionalities and have similar features as well, which are often difficult to distinguish. Some of the features enable instructors to interact and provide educational materials and resources to students during their learning process. There is vast increase in the number of Learning Management Systems especially open source LMS because they are easy to modify and customize to suit different preferences. However, the users usually do not know which open source Learning Management System is best in terms of system quality amongst the top rated open source LMSs that are available. Therefore, this paper aims to solve that problem by carrying out an evaluation study to help determining which open source LMS has the best system quality among the Top 5 open source Learning Management Systems.
The outcomes of this study will enlighten particularly instructors to determine which open source Learning Management Systems has a higher system quality in terms of external characteristics. In information technology as a whole, the system quality determines the level of acceptance and the usability quality and user satisfaction of any IT system that is used. Therefore, in order to determine which open source Learning Management System is the best, this paper carries out an evaluation of the top 5 rated open source Learning Management Systems. In addition, to authors knowledge such assessment of open source LMS has not been identified in the literature so far.
The rest of the paper will investigate the system quality of the top five most widely used open source Learning Management Systems using the ISO/IEC 9126 external quality metrics performed by two experts.

A. Learning Management Systems
Learning Management Systems refer to a platform which supports faculty, administration, learning experiences, instructor and learner/student services over the Internet [9]. There are two aspects to which Learning Management Systems can effectively be reviewed; as a social entity and as a technical entity [10]. With Learning Management Systems, quality of instructor functionality is a very important feature in LMS [11]. Due to how important instructors are to Learning Management Systems, instructors should have enough features and time to enable them interact and provide educational materials to students during the learning process [12].Another important feature of Learning Management Systems is its effectiveness to the learner/students [13], as the satisfaction of the learner/student while using the Learning Management System is an important factor when determining its success and overall usability [14], [15]. As the learning management systems provides as triadic bridge among instructors, students and administration, it is inevitable to gain information regarding the quality and usability of such tools for learning in order to improve quality.

B. ISO/IEC 9126 Quality Model
In order to evaluate the quality of a product/software, there are a set of quality characteristics that describes the system and they form the basis and the foundation for the evaluation [16]. The set of characteristics that form the basis for evaluation are called the quality model [17]. The standard of the ISO/IEC 9126 quality model is widely accepted in different countries around the world, with 1,129,446 certified companies worldwide and 485,554 companies certified in Europe alone as at 2013 11 . This study focused on only the external characteristics of the ISO/IEC 9126 quality model, which are functionality, reliability, usability and efficiency as proposed in 22 and was adopted from the model in Fig. 1 and was summarized in Table I.

1) Functionality:
Learning Management Systems should have the capacity to carried out the functionalities that meets the needs of functional LMS under specified conditions [18]. As part of the functionality, Learning Management System should provide feature that are required to improve the learning experience of the learner. These include interoperability, accuracy, compliance and security [1]. Interoperability means the LMS's ability to function and interact with other applications. Compliance means developing the Learning Management Systems according to certain established specifications and guidelines. Privacy refers to the ability to protect information of the users, that is, both instructors and learners/students [18]. Functionality can easily be improved by implementing messaging services, that is, where both instructors and learners/students can send messages to each other [19].
2) Reliability: With Learning Management Systems, reliability refers to the consistency of the system and how it performs its intended functions without crashing/failure [20].
Learning Management Systems should be highly reliable, highly robust and should perform accurate well without being affected by high number of users, time of use, place of use or data access and connectivity issues [4]. One factor that determines reliability is the ability of learners/students to access the learning management systems at any given time, even under difficult situations of network connectivity [21].  3) Usability: Usability in terms of Learning Management Systems refers to the characteristics the define the quality, that is, it deals with how Learning Management Systems can be used by both the instructors and learners/students to achieve certain goals in the most efficient and effective manner possible in any given situation [22]. The guidelines that govern usability deals with mechanisms used to measure, monitor and improve the system processes of the Learning Management Systems [23]. It is very important for a Learning Management Systems to have a very clear and user-friendly user interface [24]. Therefore, in order to have a high usability level for Learning Management Systems, there is a need for the software developers to ensure that the LMS systems are adaptive and sensitive to different environments [25]. 4) Efficiency: Efficiency refers to the performance level, the response time and how the over performance satisfies the needs of the user. In order to be efficient, the performance and response time must be fast so as to fully satisfy the needs of the users. Applications and systems should be able to grant the user fast access to vital information with good network speed available. This is because the main issue that affects the efficiency of Learning Management Systems are low bandwidth, lower security and interference among others [26]. C. System Quality There are different guidelines for measuring the quality of a system and this outlines the characteristics of a software application, one of such guidelines is the ISO/IEC 9126 quality model 3 3 . Due to the many factors involved in the process of software development, there is a need to focus strictly on the characteristics of Learning Management Systems. The LMS characteristics are selected by studying and selecting the most important characteristics needed for a Learning Management Systems [1]. The quality of Learning Management Systems is further influenced by the quality of content and how the instructors can successful manipulate educational materials and effectively deliver them to the learners/students [28]. Developers of Learning Management Systems usually focus on enhancing the characteristics that are considered to be the most important, thereby improving the overall system quality [29].
Here, system quality refers to the needed and desired characteristics of open source Learning Management Systems, which are: • Functionality states to attaining the user's anticipated necessities [30], [31].
• Reliability talks about acceptance and duration. Acceptance states whether any guidance available for the system in use. Duration refers to the duration of the system in the potential market. Fault tolerance deals with the support related issues regarding possible faults in the system.
• Usability refers to how easy it is to learn, accessibility, user interface and operability. Learnability addresses to the ease of control and understand the system without referring to the user manual. Operability is associated to the ease of operating of the system. Accessibility refers to ease of access to the system without requiring any other software or plug in.
• Efficiency refers to maximum performance, be easy to install, configure and operate within a short time. Efficiency is the most important feature when it comes to open source Learning Management Systems.

III. METHODOLOGY
Learning Management Systems have become an important part of higher education. Due to this reason, this study aims to look at the system quality and features of the top 5 highly rated open source Learning Management Systems using the ISO/IEC 9126 quality model and compare the results. Learning Management Systems provide certain usability features that enable education instructors to interact and provide educational materials and resources to students during the learning process. There is vast increase in the number of Learning Management Systems especially open source LMS because they are easy to modify and customize to suit different preferences. Due to this, the purpose of this study is to evaluate and compare the best 5 open source Learning Management Systems to find out which among them has a higher system quality and usability. Initial background research was conducted on 35 open source Learning Management Systems.
Then, the best 5 amongst them were selected and compared using the external characteristics of ISO/IEC 9126 quality evaluation model, according to the system quality characteristics as proposed by [19] and [7]. This is based on the overall system quality of the Learning Management Systems. The importance of system quality in every application and in Learning Management Systems in particular cannot be over emphasized because system quality automatically translates to usability, that is, the higher the system quality, the higher the usability vice versa. Due to the increasing number and usage of Learning Management Systems over the years, it has become necessary to carry out an evaluation studies on the top 5 open source LMSs based on the following system quality characteristics; Functionality, Reliability, Usability and Efficiency. The Learning Management Systems selected for this study are Moodle, ATutor, Eliademy, Forma lms and Dokeos.
Two experts were asked to conduct a research on both the websites and while using the selected top 5 best open source Learning Management Systems in order to get the relevant information on the latest versions of their systems. The background of the experts are; expertise on educational technology and Information Systems and in computer engineering. They are well experienced in the processes of designing and developing various types of management information systems. After the experts have gotten access and obtained required information, they evaluated the selected open source Learning Management Systems using the software quality characteristics of Learning Management Systems were identified using the model proposed by [27]. Table I shows the corresponding questions considered by the experts during evaluation process. The flowchart in Fig. 2 represents the evaluation procedure of the study. 403 | P a g e www.ijacsa.thesai.org

IV. RESULTS
Two experts were required to use the top five open source Learning Management Systems and allocate scores to them based on their how they function effectively. The Ideal Value, which is 1 represents the highest level of functionality for each usability sub-criteria as shown below in the tables. Therefore, in order to get the result of sub-criteria, the score is divided over the Ideal Value.
In order to arrive at the final result, the score of each subcriteria as allocated by both experts is added up and their average is obtained, then the final score (i.e. the average from the score from both experts) is compared against the Ideal Value. If the score is equal to the Ideal Value, then the sub-criteria is set to be complete (i.e. fully functional) but if the score is not equal to the Ideal Value, then the sub-criteria is not complete (i.e. not fully functional). The evaluation was done on the external characteristics of each of the selected open source Learning Management Systems. After both the experts had carried out their evaluations separately, the values of their results for the characteristics of each open source Learning Management System will be compared and their average will be taken in order to determine the final score of each characteristic. Tables II to VI represent       ATutor rather has a very good system quality. It seems to have complete usability, which means it is fully usable. It is moderately efficient as well, indicating that it has an average performance. However, its functionality is high and reliability is average, which indicates that improvement is needed  Eliademy has an average reliability. The functionality and usability are also average, which suggests a need for improvement is mandatory However, it has below average usability, which indicates that there is a possibility that users find it hard to use the system partly due to interface deficiency and other usability factors. Only the external characteristics of ISO/IEC 9126 are used in this evaluation study. This is because they are the only characteristics where evaluators have access to as the others; Maintainability and Portability are internal characteristics, which means only the developers and admin has access to [32]. Since the scores allocated to the characteristics of each open source Learning Management Systems was as a result of the information derived.
The Learning Management Systems mentioned above all possess different levels of system quality characteristics. From the results obtained, Moodle seems to have a higher system quality among the Learning Management Systems selected for this study. This is why it is available in 78 different languages and is being used in 216 countries compared to the others that are available in far lesser languages and countries (ATutor: 20 languages, 58 countries; Eliademy: 26 languages, 53 countries; Forma LMS: 15 languages, 17 countries; and DOKEOS: 11 languages, 18 countries).
The completeness of the reliability of Moodle, which means it can be used effectively in small and large-scale environments, which is difficult for the other Learning Management Systems to achieve. This also, is another reason why Moodle has a larger penetration and used in more countries. Also, Moodle has a complete efficiency as well, which show that its response time to user requests, speed and simplicity of installation are performing at maximum. With regards to efficiency, Forma LMS and DOKEOS both have relatively low efficiency due to their inability to be utilized in large environments, profiling and management competencies and installation deficiencies.
Moodle and ATutor both have complete usability, which indicates the level of user satisfaction while using the systems. It also shows the level of usable the customization features, adaptability and accessibility they both possess. Eliademy has a low usability due to low integration and interoperability. The functionality of Moodle, Forma LMS and DOKEOS shows high functionality features due to the availability of plug-ins, add-ons and core functionality features reside in the core of the systems. These features gave them a high functionality 408 | P a g e www.ijacsa.thesai.org while using the open source LMSs, the internal characteristics could not be accessed and studied.
There exists fuzzy decision making techniques that uses at least one expert in the evaluation process in a similar fashion with this study [33].

VI. CONCLUSION
The aim of this study was to evaluate and help identify the best open source Learning Management Systems from the top 5 current available for use. The study gained access to previous works as reference points and used the external characteristics of ISO/IEC 9126 standard quality evaluation model as the guideline for the evaluation study. Focusing on the system quality of the open source Learning Management Systems alone, the study realized that Moodle is the best open source Learning Management Systems that is presently in use, due to its availability and accessibility in many languages and countries as well as in the availability and plug-ins and add-ons, which greatly improves its functionalities among other features.

VII. SUGGESTIONS
This study evaluates most widely used open source learning systems using external quality characteristics of ISO/IEC 9126 quality standards by two experts. In the future, further research can be conducted using different approaches or models to see whether the same result can be achieved or not. Increasing the number of experts, using heuristic evaluation,-and applying fuzzy decision making processes using same ISO/IEC 9126 quality models were discussed in earlier studies [34], [35] that could also be used in the evaluation of open source learning management systems and risk assessments could also be beneficial for further improvements of such open source learning systems.