Evaluating Software Quality Attributes using Analytic Hierarchy Process (AHP)

The use of quality software is of importance to stakeholders and its demand is on the increase. This work focuses on meeting software quality from the user and developer’s perspective. After a review of some existing software-quality models, twenty-four software quality attributes addressed by ten models such as the McCall’s, Boehm’s, ISO/IEC, FURPS, Dromey’s, Kitchenham’s, Ghezzi’s, Georgiadou’s, Jamwal’s and Glibb’s models were identified. We further categorized the twenty-four attributes into a group of eleven (11) main attributes and another group of thirteen (13) sub-attributes. Thereafter, questionnaires were administered to twenty experts from fields including Cybersecurity, Programming, Software Development and Software Engineering. Analytic Hierarchy Process (AHP) was applied to perform a multi-criteria decision-making assessment on the responses from the questionnaires to select the suitable software quality attribute for the development of the proposed quality model to meet both users and developer’s software quality requirements. The results obtained from the assessment showed Maintainability to be the most important quality attribute followed by Security, Testability, Reliability, Efficiency, Usability, Portability, Reusability, Functionality, Availability and finally, Cost. Keywords—Analytic Hierarchy Process (AHP); software quality; quality attribute; quality model; sub-attributes


I. INTRODUCTION
Software quality is a paramount issue to all software stakeholders in a given establishment and its demand is increasing rapidly due to customer demand [1]. In the last few decades, the importance of the use of quality software has increased exponentially [2]. Software users see software as a tool to enable them to carry out their daily activities with ease, and hence, use it to perform sensitive tasks [3]. The use of less quality software can, directly and indirectly, endanger one's life [30] as well as causing huge loss to software users. As a result, many software quality models have been proposed to evaluate software quality, yet, none of these models has been widely accepted as the benchmark for assessing software quality. This is because these models do not address all the important software quality attributes that are of keen interest to stakeholders and are tailored towards meeting specific project's requirements. To address stakeholder requirements, custom software quality models have been proposed [4]. These custom made quality models offer different benefits to the software industry and research community and hence do not cover a wide scope of quality attributes. This research presents an evaluation of software quality attributes using the Analytic Hierarchy Process (AHP). It was conducted based on a questionnaire given to stakeholders to assess the quality attributes they expect a software quality model to have. As a result, an evaluation of these quality attributes was made and represented with a graph to pictorially highlight the percentage of weight each quality attribute ranked. Ranking the software quality attributes will assist developers greatly in selecting the best quality attribute for evaluating developed software. Previous works have failed to rank quality attributes and have led to the proposal of numerous custom software quality models.
The rest of the paper is organized as follows: Section II discusses related work done on AHP, software quality models and quality attributes. Section III discusses software quality, quality models and attributes. The methodology used to select the software quality attributes is illustrated in Section IV. Section V presents the conclusion and discusses future works.

II. RELATED WORKS
Software quality models have been reviewed by numerous researchers in addressing software quality problems. The authors in [14] evaluated the quality of software in Enterprise Resource Planning (ERP) systems using the ISO 9126 model. They offered a comparison between existing quality models and identified the quality characteristics of ERP systems but they did not rank the main quality characteristics of the model.
In [15], research was conducted on an analytical and comparative study of software usability quality factors. They analysed ten famous quality models for developing a usability model that satisfies the demand of current business software and proposed an integrated improved usability model for assuring software quality. The new usability evaluation model was proposed from ten models of McCall, Boehm, Shackel, FURPS, Nielson, SUMI, ISO 9242-11, ISO 9126, and QUIM model.
A research was conducted by [16] on an approach for enhancing software quality based on ISO 9126 quality model. They were able to propose a new quality model for integrating some quality attributes in software development.
Another study was conducted by [3] on software quality attributes to enhance software quality assurance. The authors did this research because, in recent times, industries are giving more attention to software quality improvement. Therefore, they focused on meeting customer perspectives of software quality to propose a new model. The limitation of the research is that it did not address availability, testability and reusability problems.
Authors in [25] worked on extending Dromey's quality model to specify the security quality requirements in a software product. They conducted the research based on the increase in cybercrimes. The model was able to enhance the security requirement of software and trained people on how to develop secure software.
A study by researchers in [26] adapted the ISO/IEC 9126 quality model to evaluate Enterprise Resource Planning (ERP) systems. The model was proposed as a result of the urge in the increasing usage of ERP systems by organisations to get faster data transactions. The researchers proposed the model to have six (6) main software quality attributes including functionality, maintainability, reliability, efficiency, usability and portability. The limitation was that the model did not address some of the most important software quality attributes such as availability, testability and flexibility. It also did not rank the quality attributes.
In [27], the authors presented a software quality model for academic information systems. Their objective was to guide academic institutions that are in the process of building their Elearning systems to evaluate and choose the appropriate software attributes that are essential to the success of the entire system. The researchers identified the key attributes for Information System's Software Quality (ISSQ) from the users' perspective to measure the quality of the E-learning system. The proposed model consisted of six (6) standard attributes with their sub-attributes. This was achieved based on the ISO/IEC 9126 model. The limitation was that the proposed model failed to evaluate the importance of quality attributes.
These researchers have shown how the use of software quality models is gaining much importance in the development of software. Nevertheless, they have not ranked the quality attributes and hence, it is difficult to know the weight of each attribute to ease decision making. This research work employs the use of the Analytic Hierarchy Process, a multi-criteria decision-making tool to evaluate software quality attributes and rank them.
Analytic Hierarchy Process (AHP) has been applied by several researchers to enhance group decisions. The researchers in [17] applied this technique to evaluate and select Commercial-off-the-shelf (COTS) components. They found AHP to be useful in making trade-offs between tangible and intangible factors in calculating the weight of COTS components. Applying these weights as coefficients of an objective function in the proposed model helped to determine the best component under constraints such as budgetary constraint, compatibility among components and system reliability. Their findings have validated AHP to be an effective and flexible tool. AHP was applied by [4] to produce an integrated framework that applies statistical analysis to generate software quality models tailored to stakeholder specifications. They found AHP to be quite accessible and conducive for decisionmaking that requires the reduction of decisions complexity in pair-wise matrices.
AHP was also applied by [18] in evaluating the reliability of object-oriented software systems. They took the ISO/IEC 9126 model as the base model for the evaluation. Their results showed AHP to be useful for making decisions for the hierarchical structure of the model.
Authors in [28] applied the Analytic Hierarchy Process to develop an algorithm for evaluating software functionality. The research was due to the increase in the number of sub-attributes of software functionality quality attribute. They wanted to know the most important sub-attribute that has a great impact on software products. The AHP technique was seen to be a useful tool for the decision-making process.
In [29], the AHP technique was used to perform a risk assessment of software quality. The authors were able to construct an index system of software quality risk assessment by calculating the weight and order of risk factors. With the use of AHP, they were able to categorise risk factors into demand risk, technology risk, process risk and management risk.
The authors in [19] applied AHP to analyse software reliability. They reported that although software reliability is an important quality attribute, different stakeholders have a variety of views in that regard. Hence, they applied AHP which is designed to manage human assessment subjectively.
The Analytic Hierarchy Process has been seen to effectively aid researchers in solving complex decision-making problems in various fields but its rate of application in the software quality assurance industry is minimal. Most software quality attributes used for software quality assurance have not been ranked, hence, it is difficult to note the important attributes to use to evaluate software projects. In this research, AHP is used to rank quality attributes by using the value of their criteria weights. The higher the criteria weight, the higher its importance in evaluating software quality.

III. SOFTWARE QUALITY
Software quality is a benchmark for measuring software requirements and the prerequisite to meet the user's specifications. Software quality involves user requirements, system design, documentation, and all the requirements needed for the development of professionally acceptable software [5]. It strictly follows the software development life cycle and evaluates and improves software performance [5]. Software quality can be enforced using software quality models.

A. Software Quality Models
Different software quality models have been proposed by researchers such as McCall [6], Boehm [7], Jamwal [8], Grady [9], Dromey [10] and ISO/IEC [11] among others as shown in Table I. These quality models contain quality attributes that may be used to ascertain the quality of a software product by determining how the software executes its code or how the software architecture is structured and organized with the system's requirements [12]. All the quality models have software quality attributes and sub-attributes used for the measurement of software quality [13]. Quality attributes and sub-attributes are used to characterize products and can be measured. They usually end with the word "lity". According to the ISO 9126 standard, a software quality model is expected to have the following attributes: Functionality, Reliability, Usability, Efficiency, Maintainability and Portability.

B. Software Quality Attributes
Software quality attributes are used to measure customer fulfilment of a product for other similar products. They are also used by software developers to develop quality software. These attributes include correctness, reliability, portability, efficiency, maintainability, supportability, functionality, usability, availability, among others. The software development life cycle ensures that implementing quality attributes in software development may result in the production of a well-engineered software product and is to be enforced throughout the development, implementation, and deployment phases of the software [5].

C. Software Quality Attributes Descriptions
This section itemizes and describes some software quality attributes.
• Correctness: Correctness refers to the capability of software to meet its required results.
• Usability: Usability is the ease of use and learnability of software by customers.
• Efficiency: Efficiency is the ability of software to perform well, given that tasks are completed faster while using fewer resources and saving computer power with great performance.
• Reliability: Reliability refers to the probability of software operating in a given environment within a specified period to perform well without encountering a breakdown.
• Accuracy: Accuracy refers to the degree to which a software product provides the right results during usage without encountering an error.
• Robustness: Robustness refers to the ability of a software product to cope with any form of error it may encounter during operation.
• Functionality: Functionality is the ability of software to perform the tasks for which it was intended.
• Performance: Performance refers to the total effectiveness of a software product.
• Availability: Availability refers to the degree to which a software product is operational and easily accessible when needed for usage.
• Maintainability: The ease with which software can be modified to correct faults or improve performance.
• Flexibility: Flexibility is the ability of software to adapt to possible future changes in its requirements.
• Portability: The measure of the ease of transferring software from one computing environment to the other.
• Reusability: Reusability is the use of existing tested and validated loosely coupled components in the development of software applications.
• Testability: Testability is the ease with which the correctness of software can be verified.
• Understandability: The capability of a software product to enable the user to understand whether it is suitable and its usability for specific tasks and conditions for use.
• Interoperability: Interoperability is the ease with which software is used with other software applications.

IV. ANALYTIC HIERARCHY PROCESS (AHP)
AHP is a method of multi-criteria evaluation that organizes and simplifies the decision-making process. It was originally developed by Thomas L. Saaty [20] to provide measures of judgement consistency; to derive priorities among criteria and alternatives, and to simplify the rating of preferences among decision criteria using pair-wise comparisons [21]. The AHP decision-making tool is robust and flexible in dealing with complex decision problems. It uses a multi-level hierarchical structure of objective or goal, criteria or attributes, and alternatives.
AHP is based on mathematics and psychology [22]. It helps decision-makers to find a decision that best suits their goal and their understanding of a given problem. It is a method to derive ratio scales from paired comparisons [23] and is based on a certain scale that changes subjective judgements into objective judgement and solves qualitative problems with quantitative analysis. It is simple and hence has seen its application in many fields.

A. Assessment of Quality Attributes
The research uses an Analytical Hierarchy Process (AHP) to perform a multi-criteria decision-making assessment to select a suitable software quality attribute for the development of the quality model. The selection will be made from eleven attributes (Maintainability M(s), Testability T(s), Reliability R(s), Efficiency E(s), Usability U(s), Portability P(s), Reusability Re(s), Cost Co(s), Functionality Fn(s), Security S(s) and Availability A(s)) and three alternatives ("Mostly addressed", "Doubles up as a Sub-attribute", "Has Subattributes"). This information will be used to develop a hierarchical structure with the goal at the top level, the attributes at the second level, and the alternatives at the third level as shown in Fig. 1.
The hierarchical structure obtained was synthesized to determine the relative importance of each attribute to the goal. This is done using a pair-wise comparison matrix with the help of a scale of relative importance as shown in Table II.
The quality attributes used for the judgement matrix are shown in Table III. It consists of eleven (11) main attributes and thirteen (13) sub-attributes. The AHP technique was only applied to the eleven (11) main attributes.

B. Selection of Appropriate Software Quality Attributes using the Analytic Hierarchy Process (AHP)
The judgement matrix was determined by twenty (20) experts' decisions, based on related research. The implementation was done using MATLAB/Simulink Software R2020b. The software allows for easy calculation and analysis for the decision-making process. It also helps in constructing the model and drawing analysis.

C. Quality Attribute Selection Judgement Matrices
A geometric mean of the scores from the questionnaire was found and represented in Table IV for effective criteria and pair-wise comparison. Twenty experts were given questionnaires to fill for the multi-criteria decision process. The geometric mean of these questionnaires was found by multiplying the values for each of the attributes in Table IV and setting it to the 1/nth power. The sum of each attribute was finally calculated. The geometric mean of the scores was found by Where n is the number of terms that are being multiplied.
The normalised pair-wise comparison matrix was found in Table V by diving each of the values for the attributes in  Table IV by the sum. To calculate the criteria weight, an average of the rows is found.
It is seen from Table V that the criteria weight of Maintainability is 17.37%, Testability is 13.02%, Usability is 7.22%, Functionality is 6.22%, Cost is 4.73%, Portability is 7.13%, Availability is 5.99%, Reusability is 6.86%, Security is 13.61%, Reliability is 10.35% and Efficiency is 7.49%. Maintainability is seen to have the highest weight while Cost is seen to have the lowest weight.
To check for the cost of the expert's evaluation, the consistency of the pair-wise comparison matrix is calculated. This is done by multiplying the criteria weight by the pair-wise comparison matrix which is not normalised in Table IV as  shown in Table VI. The weighted sum of the new matrix is found and then divided by the criteria weight. The overall sum is found for the calculation of λ max and Consistency Ratio (CR). The value of the consistency ratio must be less than 0.1 to make the judgement matrix acceptable. The selection judgement matrix shows consistency since the value of the Consistency Ratio (CR) is 0.079 which is less than 0.1.  It can be seen from Table V that Maintainability M(s) has the highest weight which is 17.37% while Cost Co(s) has the lowest weight of 4.73%. Fig. 2 shows a graphical representation of the weights of the software quality attributes.

D. Alternative Selection Judgement Matrices
The alternatives which are "Mostly addressed", "Doubles up as Sub-attributes", and "Has Sub-attributes" were also analysed for Maintainability M(s) as shown in Table VII. Table VII shows that Maintainability has a higher weight of 74% for being mostly addressed and a lower weight of 11% for doubling up as a sub-attribute.
The alternatives were also analysed for Testability T(s) as shown in Table VIII. Table VIII shows that Testability has a higher weight of 67% for being mostly addressed and a lower weight of 10% for doubling up as a sub-attribute.  The alternatives were also analysed for Reliability R(s) as shown in Table IX. Table IX shows that Reliability has a higher weight of 78% for being mostly addressed and a lower weight of 8% for doubling up as a sub-attribute.
The alternatives were also analysed for Efficiency E(s) as shown in Table X. Table X shows that Efficiency has a higher weight of 62% for being mostly addressed and a lower weight of 10% for doubling up as a sub-attribute.
The alternatives were also analysed for Usability U(s) as shown in Table XI. Table XI shows that Usability weights 80% for being mostly addressed and a weight of 8% for doubling up as a subattribute. The alternatives were also analysed for Portability P(s) as shown in Table XII. Table XII shows that Portability weighs 56% for being mostly addressed and a weight of 9% for doubling up as a subattribute.
The alternatives were also analysed for Reusability Re(s) as shown in Table XIII. Table XIII shows that Reusability has a weight of 41% for having sub-attributes and a weight of 26% for doubling up as a sub-attribute.
The alternatives were also analysed for Functionality Fn(s) as shown in Table XIV. Table XIV shows that Functionality's weightiness for being mostly addressed is 57% and 10% for doubling up as a subattribute.  Vol. 12, No. 3, 2021 The alternatives were also analysed for Availability A(s) as shown in Table XV. Table XV shows that Availability has a weight of 60% for doubling up as a sub-attribute and a lower weight of 17% for being most addressed.
The alternatives were also analysed for Cost Co(s) as shown in Table XVI. Table XVI shows that Cost has a weight of 77% for doubling up as a sub-attribute and a weight of 11% for having sub-attributes.
The alternatives were also analysed for Security S(s) as shown in Table XVII. Table XVII shows that Security has a weight of 56% for doubling up as a sub-attribute and a weight of 9% for having sub-attributes.
The overall weights for the software quality attribute selection are summarised in Table XVIII.   The results in Table XVIII show that "Mostly Addressed" is the highest-ranking software quality alternative with 52% and "Has Sub-attribute" is the lowest ranking alternative with 22%. The result also shows Maintainability as the highestranking software quality attribute with 17%. Table XVIII also shows that the overall analysis is consistent since the value of CR is 0.057 which is less than 0.1.

V. RESULTS AND DISCUSSIONS
The software quality attributes have been evaluated and according to Table IV, Maintainability is seen to weigh 17.37%, Testability has a percentage of 13.02%, Reliability has a percentage of 10.35, Efficiency has a percentage of 7.49, Usability has a percentage of 7.22, Portability has a percentage of 7.13, Reusability has a percentage of 6.86, Functionality has a percentage of 6.22, Security weighs 13.61, Availability has a percentage of 5.99 and Cost has a percentage of 4.73. This was pictorially represented in Fig. 2.
Tables VII, VIII, …, XVII has shown that Maintainability has a higher weight of 74% for being mostly addressed and a lower weight of 11% for doubling up as a sub-attribute. Testability weight of 67% for being mostly addressed and a weight of 10% for doubling up as a sub-attribute. Reliability has 78% for being mostly addressed and a weight of 8% for doubling up as a sub-attribute. Reliability is seen to also have 78% for being mostly addressed and a weighs 8% for doubling up as a sub-attribute. 62% was the weight of Efficiency for being mostly addressed and 10% for doubling up as a subattribute. Usability's weightiness for being mostly addressed is 80% and 8% for doubling up as a sub-attribute. Portability also has a weight of 56% for being mostly addressed and a weight of 9% for doubling up as a sub-attribute. 41% was the weight of Reusability for having sub-attributes and 26% for doubling up as a sub-attribute. Functionality also has a weight of 57% for being mostly addressed and a weight of 10% for doubling up as a sub-attribute. 60% was also the weight of Availability for doubling up as a sub-attribute and 17% for being mostly addressed. Cost has a weight of 77% for doubling up as a subattribute and a weight of 11% for having sub-attributes. Finally, Security is seen to have 56% for doubling up as a subattribute and 9% for having sub-attributes.

VI. CONCLUSION AND FUTURE WORK
The paper uses a multi-criteria decision-making analysis based on the expert's evaluation and the use of the Analytic Hierarchy Process (AHP) to rank software quality attributes. A hierarchical model is presented for the AHP process. The results show the criteria weight of Maintainability to be 17.37%, Testability to be 13.02%, Reliability to be 10.35%, Efficiency to be 7.49%, Usability to be 7.22%, Portability to be 7.13%, Reusability to be 6.86%, Security to be 13.61%, Functionality to be 6.22%, Availability to be 5.99% and Cost Co(s) to be 4.73%. Maintainability is therefore the most important quality attribute followed by Security, Testability, Reliability, Efficiency, Usability, Portability, Reusability, Functionality, Availability and Cost.
The future work will include the integration of AHP with Linear Programming (LP) to select the most important software quality attribute among several attributes. The criteria weights produced from the AHP technique will serve as function coefficients in LP to build a linear model. Sensitivity analysis will also be performed to check changes in criteria weight and effect on the attributes.