Applications of Multi-criteria Decision Making in Software Engineering

Every complex problem now days require multicriteria decision making to get to the desired solution. Numerous Multi-criteria decision making (MCDM) approaches have evolved over recent time to accommodate various application areas and have been recently explored as alternative to solve complex software engineering problems. Most widely used approach is Analytic Hierarchy Process that combines mathematics and expert judgment. Analytic Hierarchy Process suffers from the problem of imprecision and subjectivity. This paper proposes to use Fuzzy AHP (FAHP) instead of traditional AHP method. The usage of FAHP helps decision makers to make better choices both in relation to tangible criteria and intangible criteria. The paper provides a clear guide on how FAHP can be applied, particularly in the software engineering area in specific situations. The conclusion of this study would help and motivate practitioners and researchers to use multi-criteria decision making approaches in the area of software engineering. Keywords—Multi-criteria Decision Making; Analytic Hierarchy Process; Fuzzy AHP; Software Engineering


I. INTRODUCTION
Multi-Criteria decision making (MCDM) approaches take decisions in the presence of multiple, usually conflicting, criterion.MCDM approach handles both quantitative and qualitative choices and is able to combine the historical data and expert opinion by quantifying subjective judgement [1].There are many MCDM models which include Analytic Hierarchy Process (AHP), PROMOTHEE, ELECTRE, TOPSIS, VIKOR each having different algorithm [2].Most widely used MCDM technique is AHP, developed by Saaty and inspired by the intelligent behaviour of human beings.Since judgments given by decision makers are relative, any change in the relative values of the choices may significantly change the weights of affected choices, resulting in a problem known as Rank Reversal [3].The problem of imprecision and subjectivity in the weight calculation process is not handled in AHP and these problems can be overcome by using Fuzzy AHP.Software Engineering has always been an area of concern for researchers because of its real time applications in the era of computer science.In most of the applications the final decision is dependent on the outcome ranking of alternatives in respective to criterion [4].Software development and evolution is characterized by multiple objectives and constraints [5].Nowadays the problems have been become more and more complex and depend upon multiple factors.So applying multi-criteria decision making (MCDM) approaches for solving complex problems dependent on multiple aspects is required than simple linear algorithmic approaches.This paper focuses on AHP, FAHP and their comparison by taking a working example and how FAHP is widely accepted approach in the field of software engineering.The next section discusses about AHP, Fuzzy AHP process in detail.The further section summarizes the different application areas in which Fuzzy AHP can be used.Then an example illustrates the use of Fuzzy AHP in selecting the quality model.The last section concludes and gives the future scope of the paper.

A. Analytic Hierarchy Process
Analytic Hierarchy Process (AHP) is an MCDM approach, proposed by Saaty [6], for handling multi objective problems.This approach selects best alternatives based on criterion [7].AHP is well structured mathematical approach uses consistent matrices and their associated eigenvectors to produce relative weights [8].AHP combines historical data and expert opinion by quantifying subjective judgement [9].It structures the given problem as a hierarchy, with required goal as parent node and criteria for assessing it are placed in levels below it.Weights are assigned to each node and many pairwise comparisons and matrix multiplications are made assessing the relative importance of these criteria.The end result of this method is to provide a formal, systematic means of extracting, combining, and capturing expert judgements and their relationship to analogous reference data [10].
The steps followed by AHP for concluding the relative rankings of alternatives are as follows [3]: 1) Decomposition of problem to required goal, criterion, alternatives.2) Read the decision values/variable.3) Creating the reciprocal matrix for the pairwise comparisons of criterion.4) Find Eigen values and calculate the Eigen vector for computing weights.5) Find the consistency index of the weight.6) Repeat the steps from 1 to 5 for each value criterion.7) Calculate the overall weight vector of the hierarchy.8) Infer the alternative based on the overall weight vector.www.ijacsa.thesai.orgBased on the complexity of the problem the number of levels in the hierarchy may increase.Saaty 9 point scale in tableI is used by expert to describe the relative ranking of one alternative over other alternative.
Based on Saaty's scale, experts develop reciprocal matrix A, in which values are representing the dominance of ith element on jth element as shown in equation 1.

The reciprocal matrix is as given below
Finally, vector of weights is generated as normalized eigenvector using equation 2.
Where λmax represents eigenvalues and the resultant vector of relative weights is given as in equation 3.
In a situation where many pairwise comparisons are performed, inconsistencies may typically arise.The AHP has an effective method for identifying the inconsistency in the comparison made by the decision maker, called Consistency Index.It is calculated by equation 4.

CI
AHP offers advantages in comparison to other approached as being flexible, intuitive and simplistic approach for experts.Despite being popular approach, there are certain issue needs addressing.Firstly, as judgements given by experts are relative, so any arbitrary change in value of alternatives may affect the weights of other alternatives resulting in a problem known as Rank Reversal [3].Another issue with AHP is its subjectivity and imprecision due to Saaty's 9 point scale [9].Also, the AHP method cannot represent some comparison in which the alternative are bigger than 9 point.

B. Fuzzy AHP
Due to shortcomings incurred by the subjectivity of human judgements and vagueness of the data, the fuzzy logic, introduced by Zadeh, can be utilized.The combination of fuzzy logic and analytic hierarchy process is a hybrid approach for both qualitative and quantitative criteria comparison using expert judgements to find weights and relative rankings.Since most of multi-criteria methods suffers from vagueness, FAHP approach can better tolerate this vagueness [11], [12].The combination of both generates credible results than conventional AHP [13], [14].In FAHP, expert judgement is represented as a range of values instead of single crisp values.The range values can be given as optimistic, pessimistic or moderate [10].For finding the weights the extent analysis method given by Wang et al [15] is utilized.The membership function used for creating the fuzzy set is given in equation 5, where x is the weight of relative importance of one criterion over other criterion.
Triangular fuzzy numbers (TFN) provide an opportunity in deciding the weight of one alternative over the other.TFN is represented by equation 6.
Where l,m,u are pessimistic, moderate and optimistic values respectively.
The modified Saaty scale using TFN is given in table II.In FAHP table 2 is used for construction comparison matrix A= (aij) nxn where i, j = 1, 2, 3n.The next step is to use extent analysis method to calculate the relative ranking of alternatives, the synthetic extent values are obtained by equation 7.
The degree of possibility of M 1 ≥ M 2 is defined in equation 8.
In equation 9, d is representing ordinate of the highest intersection point between between µ N 1 and µ N 2 .
The degree of possibility for a convex fuzzy number, is defined by equation 10.In order to normalize the weight vector,equation 11 is used.
After calculating the weights of criteria, the scores of alternatives with respect to each criterion is evaluated and composite weights of the decision alternatives are determined by aggregating the weights through hierarchy.

III. FUZZY AHP IN SOFTWARE ENGINEERING
Many application areas in the field of software engineering have been identified as the problems, where MCDM is utilized for solving multi-objective problems.Some application areas are discussed below:

A. Evaluation and Assessment
Sarfaraj et al [16] have successfully used Fuzzy AHP technique for identifying the appropriate web development platform.The proposed model took into account four criteria, namely security (C1), compatibility (C2), performance (C3) and licensing cost (C4) for choosing the best platform.Three alternatives namely, (Linux/Apache/ MySQL/PHP (LAMP) (A1), Microsoft's ASP.NET (A2) and Sun's Java 2 Enterprise Edition (J2EE) (A3) are evaluated at the level of problem hierarchy.The conclusion of the work is that criteria 'security' is most significant of all others and LAMP is chosen as web development platform.Fuzzy AHP approach is proposed by Vatansever and Akgul [17] for assessing the quality of service delivery of websites.In the study, the quality of four eCommerce company web sites which operate in Turkey having the highest sales volume have been analysed with the fuzzy AHP approach.The criteria used for evaluating the web site quality were 4 main and 22 sub-criteria.The most significant criteria affecting the quality of the Web site were determined as the information quality, system quality, service quality, and vendor specific quality.The most significant factor affecting the quality of the website is the vendor specific quality as per the proposed model using fuzzy AHP.A study has been conducted by Kong and Liu [18], about ranking of the factors behind the success of E-commerce.They have considered Trust, System Quality, Content Quality, Use and Online Service as 5 main criteria and 17 sub-criteria.They have concluded that Trust is the most critical factor and Security is the most critical sub-factor of Trust.All other factors have also been ranked by using FAHP.

B. Risk Analysis and Ranking
Risk analysis is procedure of finding, analysing and handling identified risk factors throughout the life cycle of a software project.The Fuzzy AHP provides the flexible and easily understood way to analyse project risks.Kahraman and Tuysuz [19] have suggested that MCDM can be used evaluation and assessment of project risks.They have measured the risk level of an information technology product by considering six different risk groups which can further be divided into 28 sub-risk factors by using FAHP. Lee [20] has used FAHP for information security risk assessment.He considered four criteria namely, assets, threats, vulnerability and safety measures for pairwise comparisons.Risk factors and the Customer-to-Customer E-commerce transaction system's security risk level can be identified by incorporating Fuzzy AHP as suggested by Wei et al [21].Ranking of risks has been achieved by Askari et al [22] by identifying the project objectives and alternatives i.e. risks of creating a FAHP model.By considering the different alternatives, the weights are calculated and then a ranking is assigned to the risks.

C. Quality Evaluation
It is hard to measure and quantify the quality of software product, due to that the approach followed is to evaluate development software quality of vendors.The selection of vendors has been shown in numerically by taking the data of a company designing and manufacturing smartphone.The criterion namely functionality, reliability, usability, efficiency, maintainability and portability of software quality model ISO/IEC 9126 are chosen as evaluation criteria for selecting an alternative i.e. vendor in the case.Challa et al [23] have developed a tool based upon the algorithm using Fuzzy AHP as the base for selecting the quality parameters.They considered the developer's perspective, the user's perspective, and the project manager's perspective.They also added several new sub characteristics to the base model i.e.ISO/IEC 9126.

D. Software Project Selection
Selection of software projects can be done by using MCDM methods.For this selection three phases are proposed by Bakshi et al [24].In first phase set of alternatives are identified, second phase uses quality function to find best alternatives.The sensitivity analysis is performed in the last phase for checking the robustness of selection methodology.

E. Testing Adequacy Criteria for UML Design Models
A model is presented by Srivastava and Ray [28] for comparing an automated functional and regression testing tool using the FAHP.Triangular Fuzzy numbers are used by decision makers from CMM level five organizations.They discussed the use of FAHP approach for incorporating the three aspects namely, DCD criteria, Interaction diagram criteria and deriving test objectives in testing adequacy criteria for UML design models.They stated that FAHP has the ability to cater to uncertain imprecise data.Upon evaluating the framework it was identified that DCD criteria is the preferred decision testing adequacy criteria for UML Design Models.Belton and Stewart [29] have assessed four aspect oriented programs qualitatively based on the five factors of software testability i.e. controllability, observability, built in test capability, understandability and complexity.Different testing environments and software change characteristics can affect the choice of regression testing techniques.

IV. NUMERICAL EXAMPLE FOR SELECTION OF SOFTWARE QUALITY MODEL
The essential part of software product is its quality, different quality models are preferred for quantifying the software product quality.The quality model evaluates the quality of model based on certain parameters.These software quality models define parameters that are related to quality of system or software product.For selecting the best quality model to quantify the software product depends upon many criteria [30].The identified criterion used are Reliability, Efficiency, Maintainability and alternatives are McCAll, Boehm and ISO 9126 software quality models as represented in figure 1.
The equation 11 has been used to calculate the normalized weights of the criteria comparison matrix and yielded the result w= (0.306, 0.302, 0.39) as shown in table III.After calculating the final weights for criterion same methodology is applied to find the weights for alternatives for each criterion.

V. CONCLUSION
AHP is an intuitive approach for solving decision making problems by breaking the it into alternative, assessment criterion and overall goal at the top of the hierarchy.Due to vagueness in human judgement and Saatys scale, sometimes it is hard to correctly making the relative ranking, which generates rank reversal problem.FAHP is modified version of traditional AHP as it uses fuzzy logic.FAHP is able to tolerate the human vagueness to greater extent.In this paper, different application areas of software engineering have been identified in which MCDM methods can be applied.FAHP approach is successful in solving MCDM problems such as objective of assessing and finding the right alternative for different applications like finding the web development platform, assessing the quality of websites and evaluating the success factors of e-commerce.Since project risks are multidimensional in nature, ranking and assessment of risks has been realized by using Fuzzy AHP.The application areas and numerical example discussed justify that Fuzzy AHP can be effectively implemented in software engineering application domains.

TABLE III :
Normalized weights of Criterion Using FAHP Table IV depicts the normalized weights for alternatives using FAHP.Table V depicts the pairwise comparison for an individual criteria with respect to different alternatives.Table VI presents the comparison of AHP and FAHP.A weight factor of 1.39 in case of AHP shows that Boehm's model has clear dominance.But when uncertainty has been considered by decision maker, results displayed that a weight factor of 0.38 in ISO 9126 has clear dominance over other software quality models.