Towards Linguistic-based Evaluation System of Cloud Software as a Service (SaaS) Provider

Cloud Software as a Service (SaaS) is a type of delivering software application by using Cloud computing Infrastructure services. Cloud SaaS used the global Internet connection to offer its services to the client consumers. The selection of Cloud SaaS provider is based on the evaluation mechanism that the Cloud SaaS consumer manage before making the service contract. In this paper, the linguistic-based evaluation of Cloud SaaS quality attributes has been proposed to help the consumer to assess the service for optimal service selection. Our proposed approach has been developed by the combinations of fuzzy logic and TOPSIS MCDM methods. The proposed approach helps the Cloud SaaS consumer to select the optimal service Cloud SaaS service provider. The case study has been proposed in order to demonstrate the proposed approach. Keywords—Cloud services; software as a service (SaaS); evaluation system; quality of experience (QoE); fuzzy logic; TOPSIS


I. INTRODUCTION
Cloud SaaS is one of three types of services that Cloud Computing provides. It is a kind of software component that runs on top of a platform as a Service (PaaS) that in turns works on top of the Infrastructure as a Service [1]. Cloud SaaS is accessed and utilized via the global network infrastructure (i.e. the Internet) rather than installing software on computer client machine (i.e. on-premises service) [2]. Cloud SaaS runs on vendor's datacenter Cloud without installing it into the end-user machine. The user has full administrative control of functions such as insert, edit, delete and etc. However, the Cloud SaaS vendor has the responsibility for care of customer data and upgrading and modifying without any associated burden to the user. In recent years, there has been notable growth in the adoption of Cloud SaaS as it can reduce the cost of building software applications and services, especially for small and medium enterprises (SMEs) [3]. The Cloud SaaS has therefore become the leading and fastest growing software development paradigm comparing with licensed on-premises software service which may classified as an old fashion of delivering software [4], [5].
In recent years, research on Cloud SaaS selection has gained much attention by developing the evaluation system to assess the selected service provider [6], [7], [8], [9], [10]. The majority of existing works have developed the evaluation system based on pairwise comparing each quality evaluation attribute in order to make a decision and select the optimal service. In addition, the existing works consider the quantitative evaluation mechanisms to make a decision for selecting the optimal service. Therefore, evaluation of the Cloud SaaS provider based on subjective quality attributes needs to be addressed for further studies.
In this study, a new technique that improves the evaluation of Cloud SaaS provider is suggested. The proposed approach is existing works that proposed in our previous works for ranking the services based on the consumer's preferences [11], [12], [13]. Our proposed approach introduces the main quality evaluation attributes based on asking expert people of selection Cloud SaaS provider as well as reviewing the literature on the field of Cloud SaaS selection. The fuzzy logic is developed to interpret the linguistic terms in order to evaluate the quality evaluation attributes. Moreover, the multicriteria decision making (MCDM) approach has been utilized by TOPSIS mathematical method in order to select the optimal service after the evaluation process. In the end, a case study is developed for demonstration purpose of Linguistic-based evaluation system proposed approach.
The remainder of the paper is organized as follows: Section 2 discusses outlines the literature review on the Cloud SaaS Evaluation system, Section 3 describes the proposed evaluation system framework. The case study is presented in Section 4; Section 5 concludes the paper.

II. LITERATURE REVIEW
Evaluation of Cloud SaaS provider aims to help a Cloud SaaS consumer to assess selected Cloud SaaS providers for evaluation purposes [9], [10]. As a result, there is an improved selection process with greater satisfaction in Cloud SaaS provider choice. The majority of Cloud SaaS providers offer a month trial version with limited capabilities to help a service consumer to test and evaluate their potential services to make an educated decision on whether or not to continue using the service or not. Therefore, the evaluation system is based on the consumer perspective, which is known as quantifying a service consumer's experiences. A few techniques [6], [7], [8], [9], [10] proposed on evaluation methods for the Cloud SaaS provider.
(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 13, No. 2, 2022 Godse and Mulik (2009) proposed evaluation method for Cloud SaaS provider using the analytic hierarchy process (AHP) approach in order to select an optimal service. They also consider some quality attributes for evaluation purposes such as service functionality, architecture, usability, vendor, reputation and cost [6]. In addition, and based the same evaluation mechainsm, Boussoualim and Aklouf (2015) proposed pairwise comparison for weighting the quality attributes as well as an evaluation service provider, and prior to decision making regarding a service. The authors were also considering the functional category of Cloud SaaS providers and nonfunctional quality attributes for the evaluation process. The main non-functional quality attributes considered in their work as follows: reputation, cost, usability, structure, configurability and personalization. The authors finally demonstrated their approach by a case study [9]. However, from existing approaches developed the evaluation system by an AHP method is only suitable with a small number of alternatives. Reixa et al. (2012) proposed an evaluation approach based on aggregation function based on a Cloud SaaS consumer perspective with also considering the opinion of an expert person. The highest score indicates that the service has many of the appropriate features. The authors provide factors that a service consumer should consider when evaluating a Cloud SaaS provider. The factors generated by extensive studies using the expert group report "European commission" that focus and analyze the main characteristics of cloud services. These factors as follows: suitability, economic value, control, usability, reliability and security. The author also provides a case study for selecting only three office cloud services: Zoho Docs, google docs and Microsoft office365 [7]. Jagli et al. (2016) proposed an evaluation mechanism using a decision tree-data mining model. The authors investigate the main challenges for representing the qualities that a Cloud SaaS consumer are considering when evaluating a Cloud SaaS service provider. They concluded with the following criteria: pay-per-use, availability, reusability, scalability, data managed by provider and service customizability [8]. However, the main shortcoming of this work is the lack of demonstration of its use in a case study or experiment. Naseer and Nazar (2016) proposed an evaluation of Cloud SaaS provider based on the analytic network process (ANP) to make a decision and then select the best service provider. The authors also investigated the main quality attributes for evaluating Cloud SaaS providers including usability, security, reliability, tangibility, responsiveness and empathy [10]. Therefore, we can summarize the main shortcomings of existing approaches as follows: (1) none of the existing approaches are considered an evaluation process using the consumer's experience perspective. (2) none of the existing approaches are used the evaluation selected the Cloud SaaS providers based on linguistic-based evaluation quality attributes. In order to address the above shortcomings, the evaluation mechanism based on the linguistic evaluation quality attributes is proposed. The system proposed is developed by combining the fuzzy logic with multi-criteria decision making.

III. LINGUISTIC-BASED EVALUATION SYSTEM FRAMEWORK
The main contribution of this work is for evaluation Cloud SaaS providers based on subjectively quality attributes. In our previous work [11], [12], [13], the Cloud SaaS ranking system proposed to help Cloud SaaS consumer for ranking and sorting the service providers based on quantitative quality attributes requested by service consumer. Our proposed system consists of three main parts as shown in Fig. 1: (1) Quality of Experience (QoE) Service Repository -QoESR, (2) Consumer Evaluation Handler CEH and (3) Decision Maker System -DM S.All the components are worked together in order to make a decision to select an optimum service provider. Each of these elements are discussed in the following subsections. This section details the service repository required to deal with the Cloud SaaS providers and the quality evaluation attributes. As was mentioned before, our proposed system considers the evaluation of service providers based on the consumer's experience. Therefore, the quality of experiences (QoEs) has been examined intensively to consider the most concerning factors when evaluate the service. However, due to the lack of a standard repository for the QoE for the Cloud SaaS service provider, we propose a new standard of QoE based on the views of an expert team and with consideration of the literature review in this section.
The QoEs have divided into three categories based on (1) service vendor, (2) service data and (3) service itself.  Software Support Helpdesk support, software update and problem solving response.
Vendor Reputation Service history, number of clients, service rating and social media rating.

Training
Training materials, videoed explanations and community support.

Security
Security certificates, HTTPS and trust certificate.

Recoverability
Supporting service backup.

Interoperability
Supporting the export of different kinds of files.
Usability Easy-to-use and supporting Mobile App.

Integration
Supporting integration with other services.
Documentation E-book, White papers and service report.
Offline support Supporting offline service and synchronizes service automatically after reconnecting service.
usability problems and provide updates to fix bugs and improve security. b) Vendor reputation: It is important that the customer considers the reputation of the service provider, taking into account the number of clients who have used the software and their satisfaction level. Another consideration is the brand value of the software vendor. Some consumers prefer to select an established software vendor which has a good reputation, however, a new software vendor may have a good reputation if their services have expanded and if they provide sufficient documentation on the Internet [6], [9]. c) Training: A service consumer should also consider whether a vendor provides training, such as workshops, tutorials, or webinars covering both local and international events and information on video channel platforms such as YouTube to help consumers understand the full range of features offered by the service. 2) Data category includes all the data-related factors of Cloud SaaS. The data category comprises security, recoverability and interoperability. a) Security: This tends to be the main concern for most Cloud SaaS service consumers since SaaS is a multi-tenant service concept. Security is a process or set of actions to ensure data is protected from unauthorized persons or systems [7]. Security involves confidentiality, authenticity, integrity, and availability. A Cloud SaaS service consumer should check if a service is certified with security certifications such as SAS 70, SSAE 16, ISAE 3402, ISO/IEC 9126, ISO/IEC 27001, and ISO/IEC 27002 to ensure the service has an appropriate level of security [14]. b) Recoverability: Backup periodicity specifies how often backups are made, either in a continuous manner or at regular intervals. Recovery velocity specifies how quickly data can be recovered from the backup in case of application or infrastructure issues, including the backup service in terms of using onpremise devices or cloud storage options. c) Interoperability: Data interoperability refers to the ability of data to be represented in different forms so that it can be accessible over different software platforms. A consumer needs to verify if a system can export their data in commonly used data forms. For example, if a service is selected by a hospital to manage the input of patient information, then the selected service must be able to export the data in various forms, such as PDF, XLS and CSV. This is an important feature in case a consumer wants to adopt another cloud service, or in the backup process, or even for further research or study purposes.
3) Service Category covers all the quality parameters involved with services such as the design of the windows screen and the integrity of the system. The service category comprises usability, integration, documentation and offline support.
a) Usability is a measure of the consumers satisfaction with a service and the quality of their experience in interacting with the Cloud SaaS [14]. A Cloud SaaS provider should ensure there is good user interface (UI) to enhance the users experience of service. Usability is evaluated in terms of the following parameters: (1) UI is intuitive and easy-touse for frequently implemented tasks and has an attractive graphical windows design; (2) the availability of user manuals to enhance consumer understanding of the windows design and e-Learning modules; (3) support for mobile applications such as mobile phones and tablets [6]. b) Integration refers to the ability of software to communicate easily with other service platforms to share information and is generally connected to the application programming interface (API) which enables one service to be integrated with other services and systems [14]. A consumer should verify if the service API has the features to enable it to connect with other software platforms. For instance, if a consumer selects the ERP service and needs additional functional software such as a storage service, then the consumer should investigate whether the ERP service for the selected software is able to be integrated with other services, which in this case, is a storage service. c) Documentation is important that the service consumer understands the functions of the service, and how the software operates in various stages. Documentation also includes different kinds of materials provided either by a service provider or service community or service consumers, such as E-Books, reports and white papers. d) Offline support is an important factor due (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 13, No. 2, 2022 to the continuous connectivity with the software server and measures whether the service supports the connectivity with the system in offline mode and synchronizes data once it reconnects to the Internet.

B. Consumer Evaluation Handler (CEH)
The main objective of CEH is to register the consumers feedback on the quality factors for each service provider and to assign weights to indicate the priority of the quality factor. The CEH deals with two important linguistic variables from a service consumer: (1) Quality evaluation (QEvaluation, ε), and (2) Quality priority (QPriority, ρ). The result after fuzzified is the quality evaluation score (QEvaluationScore, δ).
• Quality Priority (QPriority) (ρ j ) This is the first linguistic variable of our proposed system. It is used to assign weights to determine the importance of quality for the evaluation process. We propose seven linguistic values which are decomposed into triangular fuzzy numbers using the triangular fuzzy set shown in Fig. (2). Table II shows the seven linguistic terms which are used to measure the quality priority.   Fig. 3. • Quality Evaluation Score (QEvaluationScore), δ The final process before deciding and selecting a service is to aggregate the fuzzy weighting of the quality priority with the evaluation criteria to obtain the quality evaluation score. The quality evaluation score, δ, is the overall quality score of the individual quality of each service. This process is a metadata step to transmit all this information to the decision maker system to select the best service that matches the consumers requirements.
Additionally, we need to aggregate these two fuzzy numbers together. To do this, the fuzzy number can be aggregated with different arithmetic equations [15] such as:ã Therefore, equation (1) is applied to aggregate these fuzzy numbers from the evaluation of services and the weighting criteria.  [16]. CoG has been used recently in different research to deal with linguistic terms [17]. Equation (4)

C. Decision Maker System (DMS)
The TOPSIS method is proposed to select the best Cloud SaaS after the evaluation under a fuzzy environment. The TOPSIS standard of a technique for order performance by similarity to the ideal solution was developed by Hwang Yoon [18]. The idea of TOPSIS is to select the best alternative based on the shortest distance to the positive ideal solution and the furthest distance from the negative ideal solution. The advantages of the TOPSIS method over the other MCDM approaches is that it is easy to develop with different programming language platforms. Moreover, TOPSIS can be used with many alternatives, which can be easily applied with m × n matrix. where m denotes the number of alternatives and n denotes the number of criteria. Other multi-criteria decisionmaking approaches such as AHP, ANP and ELECTRE are only suitable for use with a small number of alternatives due to the expense in processing and time where there are numerous alternatives.
The TOPSIS MCDM process comprises six steps to make a decision and selects the best alternative [18]. However, if we apply the TOPSIS method in our proposed Evaluation system to select the best service, we leave out some steps, such as the normalization process and calculating the weighting normalized matrix because these two steps have already been calculated by the fuzzy logic calculation proposed previously.
Accordingly, to apply the TOPSIS MCDM method into our proposed approach, four steps are needed to select the best Cloud SaaS service provider after the evaluation of the services by a cloud SaaS service consumer as follows: Let us assume the QEvaluationScore matrix that will interact with the decision maker system is as follows: where δ i j is the quality evaluation score after the defuzzification process for service i and quality j.
The steps involved in our method are as follows: 1) Calculate the positive ideal solutions and the negative ideal solutions. This step is used to select the service based on measuring the shortest distance to the positive ideal solution and the farthest distance to the negative ideal solution. The result of each calculation will be a vector size of 1 × n. where n denotes the number of qualities that are of concern for a service consumer. Equation (5) is used to calculate the positive ideal solution, and equation (6) is used to calculate the negative ideal solution.
2) Calculate the separation measures for each service from the positive and negative ideal solution. To do this, the TOPSIS method applies Euclidean distance to measure the distance to the positive and negative ideal solution. The result of the separation measures will be between [0,1]. Equation (7) is used to calculate the separation of each service from the positive ideal solution and equation (8) is is used to calculate the separation of each service from the negative ideal solution.
3) Calculate the relative closeness to the ideal solution for each cloud service using equation (9). The result of this step is between [0,1]. The service that has a higher number indicates better performance and that it is a good match with the consumer's requirements.
4) The last step is ranking and sorting the cloud services based on the index value C j i . The service that has a higher index value will be selected first which means it is very close to the consumer's evaluation of services. The result represents the evaluation score for each service which shows the position of each service.

IV. CASE STUDY
To better illustrate our proposed evaluation approach, a case study is used to explain the process of evaluating services to make a final service selection using the linguistic-based evaluation technique.
Let us assume that a new business called Misbar is interested in adopting a Cloud SaaS solution using the CRM SaaS application. The expert team at Misbar understand the benefits of using cloud services instead of building their own software application, such as availability, elasticity and support for a mobile app. Therefore, six Cloud SaaS service provider were chosen for this purpose based on different QoS criteria such as service cost, availability and service rate.
For the final service selection, the Misbar team needs to evaluate their six services using the evaluation criteria described in Table I. We assume all these selected services can be trialed and evaluated during the free test period offered by the service provider. As shown in Table IV, the Misbar team evaluates each service based on multiple QoE parameters. For instance, the first service provider evaluates software support as medium. This means this service has been evaluated as being medium in terms of supporting problem solving, fixing bugs, updating software and providing help desk support. Similarly, they evaluate the following criteria: vendor reputation, training, security, recoverability, interoperability, usability, documentation and offline support for all the selected services. After this, the Misbar team weights the priority of each quality attributes among other conflicting criteria to make a service selection. Table V shows the priority of the criteria provided by the Misbar team. As shown in this table, the Misbar team used linguistic values to weight the QoE parameters. For example, the Misbar team considers software support as very important, however, they consider the usability criteria as important. Then, the Evaluation system transfers their linguistic values that describe the evaluation criteria into fuzzy numbers. Table  VI illustrates the fuzzy numbers of the linguistic values for weighting the quality attributes.  The Evaluation system then transfers the linguistic values of the evaluation of the quality attributes into fuzzy numbers. Table VII shows the fuzzy numbers of the evaluation criteria for all services. After collecting all the inputs from the Misbar team, the Linguistic-based Evaluation system is used to select the service which best matches the consumer evaluation for each service. Firstly, our system aggregates the fuzzy evaluation of the attributes (QEvaluation) with the fuzzy quality priority (QPriority) to obtain the quality evaluation score (QEvaluation-Score). For example, regarding the first service, for the software support criteria, the fuzzy (QEvaluation) is (0.2,0.4,0.6) and the fuzzy (QPriority) is (0.7,0.9,1.0). Therefore, the aggregation of these two fuzzy numbers is (0.2, 0.4, 0.6) ⊗ (0.7, 0.9, 1.0) = (0.2×0.7, 0.4×0.9, 0.6×1.0). The final result is (0.14,0.36,0.6). Table VIII shows the evaluation score matrix for all cloud services with respect to the quality evaluation score. Secondly, the fuzzy interpreter defuzzifies the fuzzy numbers for the (QEvaluationScore) to obtain the crisp number. To do this, the Center of Gravity (CoG) as proposed in Equation (4 is used to calculate the crisp number. For example, the fuzzy evaluation score for the first service for software support is (0.14,0.36,0.6). Therefore, to calculate the CoG is 1 3 (0.14 + 0.36 + 0.6) with a result of 0.366666667. Table IX shows the crisp numbers of all the services. TOPSIS MCDM is a method which is used to assist decision making based on measuring the distance for each alternative based on the shortest distance to the positive ideal solution and furthest distance to the negative ideal solution. The positive ideal solution is the highest value for each criterion, whereas the negative ideal solution is the lowest value for each criterion. In the next step, our proposed system obtains the positive ideal solution and negative ideal solution to make a decision based on measuring the distance to these two vectors. Table X shows the positive ideal solution vector and negative ideal solution vector. After this, the TOPSIS method makes a decision by measuring the distance to the positive ideal solution (S + ) and the negative ideal solution (S − ) for each service using Euclidean distance. Table XI shows the distance to the positive ideal solution and the negative ideal solution for all services.
Before the last step, the TOPSIS method calculates the similarity to the positive ideal solution in order to obtain www.ijacsa.thesai.org  The final step is to rank the services based on the value of the similarity to the positive ideal solution. Table XIII shows the ranking of the services for the Misbar company. It can be seen that service provider 6 has the most similarity to Misbar's preferences at approximately 74.4 per cent. The service provider which is ranked last is service provider 2 with 30.9 per cent. Therefore, service provider 6 is the most suitable for selection based on Misbar's evaluation mechanisms.

V. CONCLUSION
This paper has been proposed an evaluation system based on the linguistic terms for selected quality attributes. The proposed work combines two mathematical models Fuzzy Logic plus the TOPSIS MCDM in order to make the final evaluation technique for each service provider. This work also gathers all evaluation attributes that the service consumer considers when selecting the Cloud SaaS services provider. The case study has been presented in this paper to demonstrate the proposed evaluation approach. Our approach helps the service consumer for selecting the optimal service provider among multiple similar services based on evaluated multiple attributes.
Our future dimension is to combine the quantitative and qualitative attributes to make final service provider selection.