Identifying and Prioritizing Evaluation Criteria for User-Centric Digital Identity Management Systems

Identity Management systems are used for securing digital identity of users in reliable, automated and compatible way. Service providers employ identity management systems which are cost effective and scalable but cause poor usability for users. Identity management systems are user-centric applications which should be designed by considering users’ perspective. User centricity is a remarkable concept in identity management systems as it provides more powerful user control and privacy. This approach has been evolved from amending past paradigms. Thus, evaluation of digital identity management systems based on users’ point of view, is really important. The main objective of this paper is to identify the appropriateness of the criteria used in evaluation of user-centric digital identity management systems. These criteria are gathered from the literature and then categorized into four groups for the first time in this work to examine the importance of each parameter. In this approach, several interviews were performed as a qualitative research method and two questionnaires have been filled out by forty six users who were involved with identity management systems. Since the answers are perception based data the most important criteria in each category are assessed by using fuzzy method. This research found that the most important criteria are related to security category. The results of this research can provide valuable information for managers and decision makers of hosting companies as well as system designers to adapt and develop appropriate user-centric digital identity management systems. Keywords—management of information technology; digital identity management systems; evaluation criteria; fuzzy analytical hierarchy process (FAHP); user-centricity


INTRODUCTION
In today's Information Systems, users have various compounds of login-name and password for every online service or even distinct credentials for different roles inside the services which are available for them.This can result in privacy risk for end-users and jeopardize service providers by security threats.Applying identity management systems is therefore the solution.These systems issue a digital identity for each user and users can control the full life cycle of their identities, from creation to termination [30].In federated identity management model, identities from various service provider, particularly identity domains, are identified across all domains [3].The major goals of these systems are to increase user convenience and privacy, and to decentralize user management tasks inside or across the trust circle [1].It is complicated for user to choose his/her identity provider along with username and password in federated identity management systems which can be considered as a drawback.In addition, the user should remember which federations he/she belongs to or can utilize [33].In Single Sign-On (SSO) solution, the user authenticates him or herself only once and it is very similar to the federated identity scenario as the same identifier of the user is automatically used by each service provider when the user logged into.Capturing the information of authentication and identification by system and giving the user access to services is the functionality of single sign-on systems [29].
In order to overcome the complicated, unintuitive difficulties which affect the actual users' needs, the latest approach in identity management systems that is user-centric identity management systems has been emerged [32].These systems support user control and privacy and designed from the users' perspective [7].A beneficial control of the use and management of Personal Identification Information (PII) is considered in these systems [35].There are two different usercentric identity management concepts: relationship-focused in which a relationship between the user and identity provider must be established, and credential-focused identity management that is offering user a long-term credentials from the identity provider and keeping them locally [6].An instance of a user-centric identity management system is PRIME which is a European government-funded project [8].Enhancing user control which is accepted by user-centric identity management paradigms is one the objectives of PRIME project along with www.ijacsa.thesai.orgconsidering identity credential misuse and physical stealing of devices [26].Former studies have presented features and frameworks, and considered numerous metrics for usercentricity paradigm, but most of them have not examined the prioritizing of a comprehensive classification concerning the most important criteria.Since, user-centric digital identity management approach has been emerged in order to conquer the drawbacks of previous identity management models, and concentrates particularly on users' perspective rather than other entities, prioritizing the evaluation criteria which can be aggregated in a universal system is very worthwhile and will assist the design and implementation of these systems beneficially.This paper identifies and prioritizes evaluation criteria for user-centric digital identity management systems.These evaluation criteria have been surveyed in the literature and then evaluated through several interviews with experts in Iran and Canada.In this research, Fuzzy Analytical Hierarchy Process (AHP) has been applied for prioritizing identified criteria, by providing a web-based questionnaire based on the most important criterion in each group.
The rest of this paper is structured as follow.Section two provides literature review.The paper's approach is evaluated in section three.In particular, fuzzy AHP and pairwise comparisons which have been performed in each category are discussed in section four.Finally, we conclude and give an outlook for future work in section five.

II. LITERATURE REVIEW
In order to identify appropriate criteria for evaluating identity management systems, we have conducted a vast search in related literature.In this section, user-centric digital identity management systems are examined based on their characteristics and requirements.Vossaert et al. [40] proposed a user-centric federated identity management approach based on trusted secure modules which meets several requirements, including: 1) Verification to prove that the only information from identity providers for which they gave their consent, is inquired.2) Performing access restriction to the information by users.3) Managing the disclosure of personal information.4) Trustworthiness of service providers in order to request their information.5) A flexible revocation procedure can be predicted.6) Scalability property in order to add new identity and service providers.7) User consent on release of data.
According to Ahn et al. [1] privacy is a major issue as a result of the immense exchange of sensitive information.Pseudonymity is the key principle for protecting user identities and personal information.Furthermore, user-centric models used in the organizations are required to pursue four key principles: 1) Notice: gaining notice about information practice.2) Choice: Users have the capability of the usage of information type and its purpose.3) Access: Users should have access to their personal information and be able to modify it whenever is essential.4) Security: Organizational system must confirm securing users' personal information.
As stated by Ahn et al. [2] an identity metasystem is designed to provide minimal disclosure for a limited usage and consistent experience across contexts in order to improve security and privacy enhanced interoperable architecture, based on the laws of identity.
Poursalidis et al. [31] introduced a multi-pseudonym Identity Management Infrastructure in which users can manage and make an excessive amount of pseudonyms.Their scheme has several advantages.First, users can maintain their anonymity.Next, preventing the existence of a single point that keeps numerous digital identities to preserve the privacy of the user.
According to Ben Ayed et al. [5] the notion of usercentricity has emerged by offering convenience and control to the users over their personal data and fulfilling to their requirements.The attribute management systems are developed to guarantee that any system section can't collect an individual's confidential attributes.From privacypreserving perspective, keeping track of which digital identity attributes have been revealed and operate by whom, are also considerable issues.In order to prohibit other parties' unpleasant context-spanning linkage and profiling, pseudonyms can be applied.
Claycomb et al. [12] discussed that, the user control over the kind of information being kept, the actual content of the information and the authorizing individuals to view the information are the major motivations in the concept of usercentric identity management systems.Another motivation is privacy and confidentiality, accomplished by offering users the option about what is shared, and with whom it is shared.Furthermore, various service providers such as financial institutions or online merchants must use a centralized repository of user information.Scalability and data authenticity should be taken into account as well.
Jøsang et al. [21] proposed a user-centric identity management approach in a single tamper resistant device in order to improve usability, simplify the user experience, provide mobility by supporting the user in using any hardware platform while obtaining online services and enhance user control.These systems introduce process automation and system support of the identity management at the user side.
According to El Maliki et al. [17] there are some basic rules which have been considered in the new user-centric identity paradigm, specifically: 1) Enhancing the user privacy by providing them full control over their identity information 2) Usability and user experience quality as a result of consistent identity interface and using the same identity for each identity transaction 3) Decreasing identity attacks, including phishing 4) Reducing reachability/disturbances caused by spams 5) Policy specification on both sides, identity providers and service providers 6) Profiting from huge scalability 7) Providing secure conditions at the time of data exchange 8) Separating the digital data from applications.
As stated by Suriadi et al. [35] communication security, minimal data sharing and disclosure, negotiation, user registration, anonymous authentication, data storage, accountability and user control are the requirements for usercentric identity management systems.It also requires that users have an effective control of the use and management of their personal identifiable information, leading to a better privacy.Some properties have been laid out in Bhargavwww.ijacsa.thesai.orgSpantzeletal et al. upon which user-centric federated identity management is based on.The key properties of a user-centric federated identity management system are user control and consent, and numerous system properties help to achieve user control.The properties that are not based on the realization of other properties are basic properties whereas composite properties are composed of basic properties.There are four basic system properties: 1) User chosen identity provider 2) Policy specification and enforcement 3) Auditing 4) Assurance support.Another basic property is transaction property.Transaction properties concern all the transactions which deal with identity-related information that is: 1) Context bound transactions 2) Unlinkability 3) User consent.The final properties in this category are identity information properties which are: 1) Confidentiality 2) Integrity 3) Availability 4) Stealing protection 5) Revocation 6) Portability 7) Sharing prevention 8) Selective release and 9) Conditional release.Several composite properties are defined which build on one or more of the basic properties: 1) Attribute security 2) Service protection 3) Non-repudiation 4) Data minimization 5) Attribute privacy 6) Accountability 7) Privacy policy, obligations, and restrictions 8) Notification 9) Anonymity 10) User in the middle.It is also stated that multi-device management and usability are the unique properties which are essentials for these systems.Usability addresses the relationship between the user-centric tools and their users.Some key aspects are 1) To have consistent user experience, 2) An intuitive and easy UI which may also help required functionality from the user like policy specification, and finally 3) Process automation that is, automating user-side processes of identity management as far as possible through policy and preferences-driven methods [6].On the other hand, some research projects look at Digital Identity Management as the core of the Internet economy and from public policy concept [14].Or another research project studies identity through one's whole life.[19].
To sum up, previous research projects have surveyed key principles and properties required in user-centric digital identity management systems.Our work demonstrates taxonomy of criteria in terms of security, user control, system capabilities and cost-effectiveness.These groups of criteria and criteria within each group are first evaluated and then prioritized based on fuzzy Analytical Hierarchy Process.

III. EVALUATION APPROACH
As the first step to evaluate identity management criteria, a thorough list of identified criteria was provided to the specialists in this domain in order to obtain their verification.Then a common decision making tool has been used to prioritize these criteria.

A. Decision Making Models
In recent decades, researchers have paid attention to multi criteria decision making model (MCDM) for complex decision making.In such models, instead of using one optimal evaluation criterion, several evaluation criteria may be used.[22] These decision making models are categorized into two groups: Multi objective decision making models (MODM) and Multi attribute decision making models (MADM).Multi objective models are used to design the alternatives whereas multi attribute models include the choice of the best option [30].One of the methods for MADM is Analytical Hierarchy Process (AHP) which is based on pairwise comparison [37].

B. Analytical Hierarchy Process
Analytical Hierarchy Process was developed by Thomas L. Saaty in 1970 which is a tool of decision making that can deal with structured and semi-structured decisions [23].In AHP, both qualitative and quantitative features of human thoughts are included in decision making process.The analytical hierarchy process deals with the inconsistency because people are more likely to be inconsistent when they are making judgments.Therefore, the pairwise comparison matrix is used which is perfectly consistent [34].
The first step in AHP is creating a multi-level hierarchical structure of objectives, criteria, sub criteria, and alternatives [36].Then, the priorities for each level of criteria are required which come from pairwise comparison [34].These comparisons obtain the relative importance of each factor that is defined by their weights [37].The decision maker has to present his idea about the value of one single pairwise comparison at a time [36].After obtaining the relative weights, the best alternative can be determined from the aggregation value of them [37].Relative weights can be evaluated from least square, geometric means, and eigenvalue methods [36].In order to quantify pairwise comparison which is the most crucial step in decision making process, a scale is used.Since people cannot distinguish between two very close values of importance (e.g., 3.00 and 3.02), Saaty used 9 as the upper limit and 1 as the lower limit in his scale [11] and for the comparison of factors, the available values are the members of this set: {1.9, 1.8, …, 1.2, 1, 2, …, 8, 9} [38].

C. Fuzzy Analytical Hierarchy Process
Although the aim of applying Analytical Hierarchy Process is to obtain the opinions of experts, the typical AHP method does not reflect the human thoughts because the exact numbers are used in pairwise comparisons method.After supplying the graph of hierarchy in FAHP, the decision makers are asked to compare the elements of each level to each other and to express the relative importance of elements by using fuzzy numbers [9].
Van Laahoven et al. [38] have introduced the triangular fuzzy numbers based on vector operation to represent the decision maker's opinion for alternatives compared to each criterion.
Chang [9] introduced triangular fuzzy numbers as a new approach in fuzzy AHP.This approach uses triangular fuzzy numbers for pairwise comparisons in FAHP.Noorul Haq et al. [28] proposed a model to evaluate and select the supplier based on fuzzy AHP approach.The main advantage of their proposed method was considering qualitative and quantitative criteria in hierarchy structure and problem solving of supplier selection using fuzzy AHP. Lee et al. in [25] applied fuzzy AHP method for assessing the importance of effective factors in choosing the supplier.These factors include: cost, performance and number of suppliers.Then based on fuzzy www.ijacsa.thesai.orgAHP results, goal planning was used to formulate the constraints.Lee [24] utilized the fuzzy AHP approach in order to analyze and evaluate the relation between the supplier and purchaser.

A. Interview
Interviews are among the most familiar strategies for collecting qualitative data.The interview is a method in which, the researcher establishes direct contact with subjects and through this method he/she assesses the perceptions and attitudes.Table I shows the first full list of criteria which have been confirmed and modified by experts.For instance, according to them, confidentiality and user's privacy must be presented as one item, with respect to their definitions.In addition, it was stated that sharing prevention should be a second-order criterion related to security issues in that we only share credentials when we try to obtain services and then we need to invent security mechanisms to avoid identity theft and misuse.As a result of experts' verification and change, second list of criteria, as depicted in Fig 1, was prepared which indeed became an outline for the main questionnaire.
First interview resulted to removing some of the criteria.Security and stealing protection covers features and characteristics of some other criteria.Therefore, these criteria should be removed.In addition, unlinkability criterion should be eliminated since it can't be applied in face-to-face healthcare transactions.Policy specification and enforcement is also not obvious because it should be identified that the policies are related to entities or they are related to privacy policy.Sharing prevention should be considered as a secondlevel and related to security criteria since sharing the credentials; connection of apps is tempted to share feeds of "data" efficiently so, there is no need to share data by value.Data minimization is an important one but it's very hard to achieve given business model imperatives in most ecosystems.Scalability is one of the most main criterion because without that, no system is likely to succeed any more.
Another interview leads to merging security and stealing protection criteria as they both have the same meaning.In addition, anonymity criterion prevents from revealing identification information of a person and when the conditional release of information exist, this one is fulfilled too.Furthermore, Pseudonymity with anonymity were combined because a person has an identity in the system but he/she has a pseudonym and its anonym.
The outcome of third interview was that Notification should be considered as a second-level criterion related to systems capabilities' criteria.The definition of auditing criterion is that it must support enforcement of responsibility for actions among several loosely coupled identity actors in case of unexpected results.In addition, User Chosen Identity Provider criterion is a hard criterion to achieve but must be considered an important goal to strive for.Many governments are managing to achieve it through contracting with private sector partners.
In the last interview, it is concluded that Confidentiality and Privacy criteria can be considered as one criterion since they have two aspects: Data protection is usually about the service provider's intended security mechanisms, vs. its policies, where it may intend to release sensitive data because it suits the organization's own ends (such as making money).Additionally, Conditional Release seems very second-order criterion related to system and users' security criteria though it's an important one.Verifiability criterion must have remediation abilities in the face of incorrect data.Pseudonymity and anonymity www.ijacsa.thesai.orgStep1.Hierarchical decision tree of this project is created as it is shown in Fig. 1.Step2.In order to perform pairwise comparison, the verbal expressions are used, namely: Equally Important to Extremely Preferred, as depicted in Table II.
Results of fuzzy AHP approach for prioritizing the evaluation criteria are presented in this section.In other words, criteria in each category and their arithmetic means are illustrated in details.Forty six experts (20 in Canada and 26 in Iran) have filled the web-based questionnaires, as depicted in Fig. 2.
The experts were both male and female students and university professors with the range of age, 15 to over 45.The questionnaire begins with some inquiries including services in which the users have registered accounts as well as managing and dealing with identity management systems.
Figures 3 to 7 show the arithmetic mean of experts' opinions in which the numbers are separated by comma in Iran and in Canada within each table.Additionally, the bar charts illustrate the preference degrees of both countries.
Final weights of sub-criteria are displayed in Table III.
As mentioned before, identified criteria in level 2 as a result of interviews are categorized into four groups: 1) Criteria related to security 2) Criteria related to system capabilities 3) Criteria related to user control 4) Criteria related to cost effectiveness As it can be seen in Fig. 3, the highest rank is dedicated to criteria related to security, in both countries.Second and third criteria are different in Iran and Canada, but forth criteria in both countries are cost effectiveness.

V. CONCLUSION AND FUTURE RESEARCH
According to literature review, transformation of Identity Management Systems can be in the range of development of silo models to federated user-centric identity management models.User-Centric Identity Management Systems should consider scalability and cost-effectiveness issues from users' perspective.Scalability is important because users register with a growing number of services and deal with complexity of managing more personal credentials which has become an impediment [21].This paper presented an approach for identifying and prioritizing appropriate criteria in order to evaluate user-centric digital identity management systems.It is believed that no single perfect set of criteria is perceived which can be implemented in all user-centric identity management systems.four categoriesare proposed to place the evaluation criteria for accomplishing the notion of usercentricity.It can be observed that based on pairwise comparison matrix and preference degrees of sub-criteria, the highest rank is dedicated to criteria related to security.This could be due to the fact that security issues enhance the trust to these systems which is very important for the user.In addition, most of the survey participants have had users' account in financial institutions and banks.
The second-best criteria in developing country (e.g.Iran), are system capabilities whereas user control in the developed countries (e.g.Canada) have had this spot as the second best criteria.Perhaps for the reason that, digital identity management systems have been more used in developed countries like Canada than developing countries is because system capabilities are more advanced in the developed countries so users are more concern with user control.Lastly, cost-effectiveness criteria have had the least priority both in developed and developing countries.
Furthermore, considering sub-criteria of confidentiality and user's privacy, dependability, user consent and service protection in Iran, whereas Confidentiality and user's privacy, dependability, verifiability and service protection in Canada were the ones with highest preference degrees resulted from prioritizing criteria using fuzzy AHP.Based on literature review, it can be concluded that the future outlook of this research will be further taxonomies of appropriate criteria in which the most predominant one could be specified regarding to assessment of user-centric systems.Interoperability with traditional identity management systems would be an asset for this user-centricity concept as it should incorporate the advantages presented by the previous approaches and focus on adaptability [2].Another important direction for future work is unifying the corresponding criteria implementable in user-centric devices, applications and solutions that facilitates user control and privacy when accessing increasing amount of online services [35].Currently, web identity management is a technology centered concept, designed to be profitable for service providers but not for users.The browser must be a usercentered identity layer between the service provider and the user, leading to better control for user over his/her identity attributes [13].Progress in digital identity management systems will become feasible to deploy user-centric paradigm which operate on a massive scale and control the full life cycle of digital identities from creation to termination, maintaining its major advantage that is, involvement in each transaction and improving its main drawback which is not being able to handle delegation [6] along with focusing on users, controlling what information is shared about them, the content of the information and who is allowed to access it [12].

Fig. 1 .
Fig. 1. Second list of criteria-Hierarchical decision treeB.The Results of Solving Hierarchy Model Using Change ApproachTABLE II.FUZZY SPECTRUM AND THE CORRESPONDING VERBAL EXPRESSIONS Row Verbal Expressions Fuzzy Numbers

TABLE III .
FINAL WEIGHT OF SUB CRITERIA