A Multi-Criteria Decision Making to Rank Android based Mobile Applications for Mathematics

Exponential growth in the amount of mobile applications for Mathematics has led users to confusion and difficulty in selecting proper application manually which suits to their needs. Therefore, there exists an imperative need for automated and efficient selection of mobile applications for Mathematics where users still heavily trust either application store ratings or the content rated by the application developer. In this study, fuzzy scale weights together with ELECTRE I (ELimination and Choice Expressing REality) were used to solve a typical multi-criteria decision making problem on ranking selected mobile applications for Mathematics with respect to given set of criteria. The alternatives are mobile applications for Mathematics and were chosen from Google Play Store through considering top five highest user ratings and high usage frequencies. Ten sets of criteria on technical and pedagogical aspects specific to mobile applications and five alternatives were used in the ranking process. Findings suggest that ELECTRE I with fuzzy scale weights are remarkably practical for outranking and selection processes. Particularly in the case of unclear and imprecise ratings, this method could offer substantial solution. Keywords—ELECTRE; mobile applications for mathematics; multi-criteria decision making; pedagogical requirements; technical requirements


I. INTRODUCTION
Mobile devices have become very dominant in our lives.The use of mobile devices has been extended merely from making calls and sending text messages to improved ability to execute various applications in demand.This took forward the usage and now smart phones have capability to support mobile learning [1].The shifting and integration of mobile technologies in education milieu has caused users to use their own mobile devices for teaching and learning practices.Mobile devices are considered to be more affordable than PCs and laptops [2].Authors in [3], reported that mobile phones are already be the part of the higher education for teaching and learning online courses.In addition, the researcher in [4] remarked upon benefiting from mobile phones for educational practices.He mentioned that it is possible to learn "anything, if developers designed it right".As time goes by several mobile applications for learning certain subjects have been developed to make learning easy but prior to when these applications are been made available to end users some tests should be performed to ensure it is of satisfying quality, reliable and it meets the specific criteria or requirements.
The authors in [5] defined mobile applications for learning as mobile applications that make it possible for users to exercise learning in a changeable position.These mobile applications could establish anytime, anywhere learning environment [6], [7].This technique of learning provides more flexibility and freedom to the learner which as a result fosters higher adoption rates by many individuals and educational institutions.
Numerous mobile applications for learning were introduced which assist in learning Mathematics at various sub disciplines of Mathematics and other fields as well [8].Particularly mobile applications for Mathematics allow users to evaluate mathematical functions, giving graphical abilities and provide some sorts of mobile calculators.Mobile technologies that provide support to learning Mathematics via using mobile devices have likewise been expanding in the course of the most recent decade [9].However, the researcher in [10] reported that there are over 4000 mobile applications specific to Mathematics to select from which have paved the ways for myriads of options to make selection on which mobile learning application to adopt.This scenario has led many individuals making a premature selection of mobile applications for mathematics because making an efficient selection from more than 4,000 applications seems tedious and time consuming, thus making the proper selection is crucial to enhance applications' continuity in usage and enhancements in development.However, with the help of automated decision making techniques such as multi-criteria decision making, the burden on the decision makers will be minimized considerably.
Hence, to address this problem an evaluation framework with automated selection process model was proposed to provide a roadmap for making a reliable selection of mobile applications for Mathematics.So far, there is only one study to focus on evaluating and selecting suitable applications for mathematics by using multi-criteria decision making approach through considering technical and non-technical aspects such as user satisfaction [11].
Numerous frameworks to evaluate a software in general from technical point of view exist where some of the wellknown models are; ISO/IEC 9126, ISO/IEC 25010, FURPS, [12]- [14], etc.It was stated that ISO 9126 software quality characteristics could be beneficial for evaluating mobile www.ijacsa.thesai.orgapplications in general [15].But till now, no such evaluation is available involving technical and pedagogical aspects together particularly for the evaluation of mobile applications.This study aims to adopt a framework from two viewpoints; technical and pedagogical aspects.
There are numerous multi-criteria decision making (MCDM) techniques which are utilized for the purpose of decision-making such as ANP, AHP, FAHP, SMART, ELECTRE, PROMETHEE, TOPSIS, etc.The researcher in [16] summarized some of the most frequently used MCDM methods in evaluating digital learning objects as; ELECTRE, Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), multiplicative exponential weighting (MEW), simple additive weighting (SAW) and Fuzzy Analytic Hierarchy Process (FAHP).These MCDM methods can be compared according to trustworthiness, perceived simplicity, quality and robustness.Among these, ELECTRE method is used for choosing the most suitable action from a given set of actions.ELECTRE is among most widely used MCDM methods that can be applied to many practical activities.ELECTRE method works with an input of criteria based ratings of alternatives by decision maker(s) which is named as decision matrix and preference information of the criteria expressed as weights and thresholds [17].The ELECTRE I technique for picking the most suitable activity from a given arrangement of activities was contrived in 1965.The ELECTRE is for 'Disposal ET ELimination Et Choix Traduisant la REalité (Elimination and Choice Expressing the Reality).ELECTRE is an outstanding MCDM strategy that has a background marked by fruitful genuine applications.ELECTRE I requires the contribution of criteria assessments for the options, called choice network and inclination data, which are communicated as weights, limits, and different parameters [17].
This study aims to apply fuzzy scale weights with ELECTRE I to obtain the outranking of alternatives through adopted technical and pedagogical criteria.

II. RELATED WORK
It was predicted in an internet report that in 2017, 268,692 millions of total free and paid-for downloads of mobile applications was available 1 .In 2018, it is normal that this number will increment to 254 billion downloads with 48% increase in download rate in 2017 as compared to 2013 1 .These statistics reveal that mobile devices are heavily taking part in our lives day by day: at home, at work, in the public and in teaching and learning as well.Particularly speaking, there exist several mobile applications to deal with numerous endeavors and instructive applications for practicing Mathematics.The consistent utilization of multipurpose innovations empowers the variability in mobile applications for learning.The researcher in [10] remarked upon that there are more than 4000 mobile applications to choose from.The same study also identified that "Despite the rapid expansion of the use of mobile applications in the educational domain, there is a lack of empirical studies as to their effectiveness in supporting learning, particularly in relation to Mathematics".This absence of available research likewise reaches out to the employments of applications by instructors.This scenario has led many users into making a premature selection of mobile applications for mathematics because making an efficient selection from over 4000 applications seems tedious and time consuming.Thus, making right selection is crucial to enhance its continuous usage and developments.Hence, to address this problem a software quality model to provide a roadmap for making a reliable selection of mobile learning application for mathematics from different but conflicting options are inevitable.Particularly the evaluation of mobile applications for Mathematics by employing multi criteria approach have been neglected by the literature where Mathematics is the fundamental field which constitutes the basis for science and engineering.Therefore evaluating any mathematics learning related mobile applications is indispensable and will be quite beneficial for users.
Regardless of the abundance and expanding usage of mobile applications only one study was located in the extant literature to evaluate mobile applications particularly for Mathematics using MCDM methods [11].There exist no other studies either for selecting or evaluating mobile applications in general by applying any of the MCDM methods.In only located relevant study, the authors in [11] proposed an adopted model defining both quality and user satisfaction used to evaluate mobile applications for Mathematics by utilizing hybrid Fuzzy AHP and TOPSIS approaches together.11 criteria used were based on the technical and non-technical aspects specific to the mobile applications for Mathematics.The technical aspects were adopted from the ISO 9126 model of while non-technical aspects were considered as user satisfaction.The weight of each criterion derived was determined through using Fuzzy AHP approach while the alternatives as mobile applications for mathematics were ranked by applying TOPSIS.
The rest of the studies mentioned below have applied ELECTRE method into different disciplines from supplier selection to personnel, network, environmental impact, mcommerce candidate partner and project selection etc.Most of these studies employ FAHP and ELECTRE together.The researchers in [18] applied Fuzzy AHP and ELECTRE to cover the issue of a network selection where the network alternatives were ranked, utilized fuzzy numbers since the importance of criteria cannot be exactly defined to integrate subjective judgment in decision-making.The authors in [19] proposed a methodology that was carried out on a hybrid approach; fuzzy AHP-ELECTRE approach.The criteria weights were computed with the FAHP method, eight criteria were used in this study.In addition, fuzzy ELECTRE I was utilized to the alternatives.The study ended with an aggregate matrix to rank the alternatives finally.The researcher in [20] proposed an Mcommerce partner selection method that uses a hybrid MCDM approach, AHP and ELECTRE I with a set of 13 criteria and 5 m-commerce candidate partners where AHP determined the weight of the 13 criteria and ELECTRE I ranks the candidate partners.The researchers in [21] applied ELECTRE I to select proper supplier with 4 different suppliers for computer www.ijacsa.thesai.orghardware and 13 criteria.The researchers in [22] used ELECTRE I to select most suitable personnel.7 criteria and 5 decision makers involved in this study for selecting most suitable from five personnel using ELECTRE method and were again ranked by using AHP by considering 5 decision makers to rank 5 projects with respect to 5 criteria as financial, solution delivery, strategic contribution, risk management and environmental factors.Researchers in an earlier study employed ELECTRE I to rank 5 projects through using ranking tool [23].The same study also highlighted the dominance of ELECTRE method over other MCDM methods through the inclusion of thresholds and outranking [23].
In general, evaluation by using MCDM methods is based on some set of criteria.Few studies were located in the literature for the evaluation of the quality of mobile applications despite their exponentially growing usage rates [24], [25].The authors in [25] stated the difficulty of finding quality evaluation models specific to mobile learning applications.This study encompasses adopted technical and pedagogical aspects together as the selection criteria from located studies.Therefore, the main technical requirements subject to this study are; user interface (usability, navigation and orientation), reliability and maintainability (error free, easiness of installation, easiness of upgrade), efficiency and performance (energy consumption, responsiveness) and the pedagogical requirements are; content quality, content presentation and content organization.The selected requirements are crucial in expanding users' engagement, inspiration, learning, capability, and capacities.Despite growing usage rates in mobile applications in general and abundantly available mobile applications particularly for Mathematics learning, there should be less time demanding and easier automated ways for users to select proper application to their use.To remedy this problem multicriteria decision making methods can be applied to rank or evaluate the quality of the mobile applications for Mathematics which is a fundamental field of study.So far, technical and non-technical aspects were considered but pedagogical aspects were understated by the current literature.Also studies on mobile learning applications that are specific to quality evaluation frameworks are seldom.ELECTRE I method which is quite practical in addressing particularly ranking problems are frequently seen in the studies used along with another method namely FAHP in existing studies.Therefore, in the light of above, this study adopts 10 technical and pedagogical aspects as criteria to rank 5 top rated mobile applications for Mathematics by using fuzzy scale weights with ELECTRE I method.

A. Alternatives: Mobile Applications for Mathematics
There are several mobile applications for Mathematics and they come in different forms depending on features like design, functionalities, purpose, limitations and target audience.This study targeted Android applications only because they are open source and have most populous mobile application store.These applications are distributed digitally via official Google Play store on the Android OS platform, which is either available freely or at some price.The Google Play store host millions of Android applications of different categories, such as social, games, education, security, etc. Users based on their experience rate these applications.Five mobile applications of Mathematics for adults were selected as alternatives for the evaluation based on their respective Google App Store user rating of at least 4.0 out of 5 and download rates greater than 1000 users.In addition, similarity in the features of the applications was also considered.Table I shows selected applications and user ratings with download numbers.As mobile applications continue to grow rapidly and gain popularity, different platforms have been developed to create and allow users to download these applications.3.4 million mobile applications in October 2017 are available for download and the application store gives users the ability to express opinions through reviews and ratings 2 .By looking at the extant literature the most crucial criteria involving technical and pedagogical aspects were involved in the adopted evaluation framework.

B. ELECTRE I Method
ELECTRE is frequently employed to find most suitable alternative with several set of criteria.Experts can select the most suitable choice through outranking alternatives via pairwise comparisons using concordance and discordance matrices.This MCDM approach has a special capacity to point out the exact motives of a decision-maker suggest an appropriate result through its ranking.
The ELECTRE I method steps were described as follows: If a problem has a number of alternatives 1,2,3,…, and b number of criteria 1,2,3,…,b .Each alternative is rated to b criteria.
Step 1: K number of decision makers (DM) denoted as 1, 2, D3… .DMs rate weights verbally.Verbal variables are transformed into (l, m, u) which is a fuzzy number.=1, 2…  and =1, 2…  and the aggregated fuzzy significance weights can be; Finally, the normalized aggregated fuzzy significance weight matrix is Step 2: A selection matrix with the aid of = ()  is fashioned for each criterion: Step 3: The normalized decision matrix = ()  by calculating , which shows the normalized criteria.
Step 4: Seeing that each criterion has an exclusive weight, the weighted normalized decision matrix is constructed by taking product of significance weights of criteria and the values inside the normalized fuzzy selection matrix. . = ()  for =1,2,…, and =1,2,…, where Step 5: weighted normalized fuzzy choice matrix was used to calculate concordance indices and sets are calculated with the use of the and pairwise assessment most of the options, respectively.If p and q are two options, the concordance index  represents the pairwise contrast between p and q (⟶). is the gathering of attributes where  is higher than or equal to .
Where j + are attributes covered inside the concordance set .
Step 6: The discordance indices mean the variances in judgment among alternatives p and q (⟶). represents that  is worse than or equal to .The discordance indices are calculated as; Where, j + are attributes contained inside the concordance set D.Later, 2 threshold values were calculated by taking the mean of all the indices in concordance and discordance matrices.
Step 7: The very last concordance and discordance indices are computed as follows:

𝐶
√∏   √∏  where (10) Step 8: In the end, Boolean concordance and discordance indices are calculated to decide high-quality alternative.Alternative with the minimal net concordance index and most discordance index is the satisfactory alternative among every alternative.

C. Triangular Fuzzy Number(TFN)
Fuzzy is set of numbers where the quantity is not specific, it can also be addressed as an extension of the popular Boolean logic whose sets is not just 0 and 1 but a connection of different values where each and every value is assigned a weight.It can be defined as a set of values ranging from one interval to another.Fuzzy attaches more dynamism to expressions.So far, many types of fuzzy numbers exist such as triangular, trapezoidal, octagonal, pyramid, pentagonal, diamond and hexagonal fuzzy numbers.Among them, triangular and trapezoidal fuzzy numbers are the most frequently applied due to the ease of use and simplicity.The researcher in [16] gave detailed description on the use of triangular and trapezoidal fuzzy numbers.A triangular fuzzy number consists of the set of three real numbers ranging from minimum, most expected and maximum weights.Fig. 1 below depicts the triangular fuzzy number with its three values;    Fig. 2 represents the membership function used in this study that converts linguistic variables into triangular fuzzy numbers and into crisp values by calculating mean value of each TFN known as defuzzified crisp values at the interval [0, 1].The linguistic variable and crisp values were adopted from [26].

D. Framework for Ranking
Fig. 3 shows the proposed evaluation criteria based upon technical and pedagogical aspects of the mobile applications that were adopted from [24] and [25].

E. Ranking Procedure
Fig. 4 represents the steps followed during ranking of 5 alternatives using fuzzy scale weights and ELECTRE I method.First phase involves about deciding about the number of decision makers.In this case, expert who evaluated the alternatives with respect to given set of criteria has a PhD on Mathematics Education which could be considered as qualified for this kind of task.Later alternatives were decided for ranking process.Then pairwise comparison of each criteria was done in order to determine their significance over each other.Afterwards, chosen alternatives were evaluated by the expert with respect to each criteria.Finally, the ranking by using ELECTRE I method was performed to obtain the most suitable mobile application for Mathematics.

IV. RESULTS
The ranking process was implemented in five mobile applications for mathematics using fuzzy scale weights obtained from triangular fuzzy numbers (TFN).For ELECTRE I method, a pairwise matrix for correlation was made utilizing a relating fuzzy scale.
The 11-point linguistic scale of an earlier study was adopted to evaluate pairwise comparison for the each pair of criteria by the decision maker.Later, defuzzified crisp values were used in this study as fuzzy scale weights.The corresponding eleven linguistic scale are; excessively low significance (EXL), extremely low significance (EL), very low significance (VL), low significance (L), semi-low significance (SL), neither low neither high significance (NL NH), semi-high significance (SH), high significance (H), very high significance (VH), extremely high significance (EH) and excessively high significance (EXH) independently, which were used to depict the importance of weights of every criteria.Table II represents the linguistic scale and the corresponding crisp values that were adopted from [26].In addition, Table III shows Step 1 involving pairwise comparison of criteria with each other that were evaluated by the decision maker using linguistic scales stated in Table II.Using Step 2, the value of pairwise comparison of each criterion was converted into fuzzy scale weights and their corresponding reciprocal value in Table IV.The sum column represents the sum of the rows for each criterion and weight column values were calculated by dividing sum to the number of criteria.Using Steps 3 and 4, normalized weight values were obtained dividing weight value for each criterion to the sum of the weight values.Sum of the normalized weights given in Table IV     The decision maker rated alternatives between ranges 0-100 in Table V adopted from SMART strategy used by [27].Using steps 5 and 6 concordance and discordance matrices were calculated in Tables VI and VII, respectively.The concordance matrix is calculated by pairwise comparison of each alternative's rating with the other alternative for each criteria.If first rating is greater or equal to second rating then the corresponding value would be the sum of normalized weights of the criteria which satisfy this condition divided by the sum of all normalized weights (equals to 1).
For discordance matrix, not preferred performance rating values were considered in pairwise comparison.The value is the maximum of differences of rating for each pair wisely compared alternatives with respect to specific criteria.After calculating concordance and discordance matrices 2 threshold values were determined by taking mean of all concordance indices and the mean of all discordance indices.The threshold values obtained from concordance and discordance matrices were calculated as; 0.64 and 0.2, respectively.
Using Steps 6 and 7 concordance and discordance domination matrices were calculated (see Tables VIII and IX).The concordance Boolean matrix is calculated by comparing all indices in the matrix to the threshold.If the value is greater than or equal to threshold then the corresponding value is 1 otherwise it is 0. For the discordance Boolean matrix is calculated by comparing all indices to the threshold value.If the value is less than threshold then it is 1 otherwise it is 0.  From the global matrix, using ELECTRE I method, the most suitable alternative is determined by scanning through the rows and selecting the alternative which has the most number of ones which represent connections.Therefore, according to the evaluation of the alternatives with respect to criteria yields to the ranking; Math42 > MathPapa > MalMath > CyMath > Mathematics.According to the decision graph given in Fig. 5, Math42 (A 5 ) was ranked as first with respect to the chosen technical and pedagogical criteria by using ELECTRE I method.In addition, ELECTRE I appear to be robust due to understandable and easy to follow steps which are fewer than other multi-criteria method steps with error free and less time consuming.
The decision graph of the ranking was given in Fig. 5.

V. CONCLUSION
This study adopted technical and pedagogical criteria to rank top 5 highly rated and downloaded mobile applications for Mathematics by using fuzzy scale weights together with ELECTRE I method.Increasing usage rates and the abundance in the number of mobile applications and their pervasive integration to teaching and learning have led users to choose the desired application instantly and with less time consuming efforts.Surprisingly this study is one of the rare studies to apply MCDM methods to an outranking of mobile applications.The only located earlier study was also carried out by the first author where researchers considered technical and non-technical aspects using hybrid MCDM method namely FAHP-TOPSIS to select mobile application for Mathematics.In terms of a higher number of alternatives and criteria ELECTRE I seems more efficient due to a fairly understandable method steps that is not only shorter compared to the other methods but more error free and less time consuming.So far, in this area, no studies have been located to employ ELECTRE I method, this study is a first that implements fuzzy scale weights and ELECTRE I to outrank Android based mobile applications for Mathematics.
In future with the increase usage and integration of mobile applications to teaching and learning, more research should be done using MCDM methods to evaluate the quality and select the desired mobile application.As for the evaluation of mobile applications for Mathematics requires concurrent thought of a few comparative and clashing criteria, MCDM methods are quite practical in handling such problems.
In this study, selecting the most suitable mobile application for Mathematics problem was remedied by using fuzzy scale weights with ELECTRE I method which is an efficient technique to deal with problems involving multiple criteria.The ranking was performed using 5 top rated alternatives with unclear and ambiguous judgment of decision maker's ratings.Finally, this method can be applied to any other disciplines as well.
The main limitations of the study can be listed as the number of decision maker is only one, there are fixed number of alternatives and the number of criteria.Also the judgment of the decision maker is effective in results.Therefore decision maker should have adequate background and level of expertise and should be objective as well.
As for future research thoughts, more number of decision makers, alternatives and criteria could be involved.In addition, the comparison of this single method to other multi criteria methods could be added to identify the effectiveness and efficiency of these techniques.Moreover, either web-based, mobile or stand-alone softwares for MCDM methods could be developed as an aid to decision making situation.

TABLE II .
LINGUISTIC SCALE AND DEFUZZIFIED CRISP VALUES Fig. 2. TFN representation of linguistic variables.

TABLE III .
PAIRWISE COMPARISON OF CRITERIA IN TERMS OF LINGUISTIC SCALE

TABLE VI .
CONCORDANCE MATRIX

Table X
represents the multiplication of the corresponding indices from concordance and discordance Boolean matrices were used to calculate the global matrix using Step 8.The value 1 represents first alternative outranks the second alternative in comparison whereas the value 0 represents no preference exists among the two compared alternatives.www.ijacsa.thesai.org