Formal Concept Analysis based Framework for Evaluating Information System Success

This paper aims to propose a methodology for evaluating information system success. It is based on two main fields, which are formal concept analysis and multi criteria decision-making methods. A framework whose main objective is to visualize the synchronization between company processes and information system indicators via process mapping and formal concept analysis exploited the methodology. Moreover, owing to the application of multi criteria decision-making methods, we can rank the information system among the others system for the purpose to ameliorate system performance. In practice, we apply the steps of this framework on a Moroccan bank by choosing a combination of processes and indicators. Keywords—Formal concept analysis; process; multi criteria; indicator; evaluation

In this paper, we seek to present a framework to evaluate IS regardless the company sector. The proposed framework involves three main steps: data collection; to collect the data of company processes and IS indicators, visualization and calculation to visualize the concept lattice of a combination of process and indicators, then, the application of the two most used multi criteria decision-making (MCDM) methods namely AHP and Topsis. In the final step, results about the information system evaluation are given and possible recommendations to increase the information system success.
The structure of this paper is as follows: section 2 presents a state of art of information system measurement, formal concept analysis (FCA), process mapping and multi criteria decision-making (MCDM) methods. Section 3, presents the main steps of the proposed framework: data collection, visualization and calculation and finally results and recommendations. In section 4, we apply the proposed approach on a real case study of a Moroccan bank. Then, the concluding remarks and perspectives are presented in conclusion.

A. IS Success Models
The literature provides many definitions of IS success as evaluation methods. Thereby, the results of the researchers are diverse and sometimes even contradictory. Authors [5] specify that the most appropriate evaluation indicator is the system use, according to them the measure of the IS success through cost or benefit studies is insufficient whereas, Bailey and Pearson [6], express the importance of the users satisfaction which is guarantor of an increase of the productivity of the IS. Other authors [7] explain that the IS effectiveness is related to anything that can bring value to the organization. As for Goodhue and Thompson [8], they share the same vision of DeLone and McLean [9] and define the success of the IS through the individual and organizational impacts.
Regarding the evaluation of IS success, the literature is very rich, we will quote the most known and used models over the years. These models of DeLone and McLean (1992McLean ( , 2003 are part of the evaluation process and evaluation model of IS that is inspired by the measurement of the Balanced Score Card.
Through this presentation of the state of the art on the evaluation of the IS success, in the rest of this paper, we will use the DeLone and McLean model [10] which is the result of several authors' validation [11].

B. Formal Concept Analysis
Formal conceptual analysis (FCA) [12] is a method of data analysis describing the relationship between a set of objects and a set of attributes. It produces two types of output from the input data. The first is a concept lattice that represents a set of formal concepts in the data that is hierarchically ordered by a sub-concept-super concept relation. The second output of FCA is a collection of so-called attributes implications that describe a very particular dependency. 1) Concept lattice: is a mathematical formalism [13] derived from a formal context K = (G, M, I). The formal context K consists of G, a set of objects, of M, a set of attributes and I a binary relation defined on the Cartesian G × M product. In a binary table representing I ⊆ G × M, the rows correspond to objects and columns to attributes (Table I).
The lattice resulting from the AFC process is composed of formal concepts ordered by a partial order relation. A concept is a pair (A, B) where A ⊆ G and B ⊆ M, A is the maximal set www.ijacsa.thesai.org of objects sharing the set of attributes of set B (and vice versa). In a concept (A, B), A is called the extension and B the intension of the concept. The concepts in a lattice of concepts are defined with respect to a Galois connection that relies on two derivation operations:

C. Process Mapping
The process approach [14] consists in identifying and managing the processes used in an organization as well as their interactions. It presupposes the representation of the dynamic architecture of what is done in the organism (representation of all the processes, their sequences and their interactions).
The process is a set of correlated or interactive activities that transforms input elements in output elements.
The processes can be classified as follows [15]:  Realization process, which allows to realize the products / services in order to satisfy the customers;  Management process, which presents the strategies of a company (priorities, objectives, methods of communication and methods of treatment and control;  Support processes, which offer the means and the resources necessary to carry out all the processes. As soon as processes are identified, the process mapping must be carried out, which is an indispensable tool for measuring progress. Process mapping [16] provides a global view of how the organization works and visualizes its processes and interactions. Process mapping makes it possible to communicate identically to a large number of actors involved in a complex activity and to give meaning to the tasks to be more performed (Fig. 1).

D. MCDM Methods
The multi-criteria decision making process is an iterative and non-linear process that generally consists of its stages [17]:  The reformulation of the decision problem;  The modeling of local preferences at each point of view;  The aggregation of preferences to establish one or more systems;  Exploitation of this aggregation;  The recommendations.  I  Y1  Y2  Y3  Y4  X1  1  1  1 Among the MCDM methods, we will use the two most used methods namely AHP and TOPSIS.

1) Ahp method:
The AHP method developed in 1980 [18] has been used successfully in several fields. It starts from matrices of binary comparisons to arrive after several stages to compare the choice of the decision problem. The method consists of representing a problem by a hierarchy structure according to the following steps: 2) Topsis method: The fundamental idea of this method, which was developed in 1981 [19], is to choose a solution that comes closest to the ideal solution (better on all criteria) and to move as far as possible from the worst solution (which degrades all the criteria).
 Calculate the separation measures:  Calculate the relative closeness coefficient to the ideal solution: ;  Rank the alternatives.

III. PROPOSED FRAMEWORK
This paper provides a framework to evaluate IS success based on two main fields, which are formal concept analysis and multi criteria decision-making methods. The aim of this work is to give a generic tool, which can be applied in different sectors with the possibility to choose the enterprise processes, and system success indicators, which will be the subject of the study. The framework architecture involves three essential parts: data collection, visualization and calculation and finally results and recommendations (Fig. 2).

A. Data Collection
It represents the basic step that allows the user to enter the data of the study (company processes and indicators of the information system success) after authentication by login and password. To model the processes of the company, we chose to use the mapping process. This technique consists of identifying as finely as possible all the processes of the company based on its information system and related to the process of the company to present them graphically.
Regarding the identification of the information system success indicators, we used the DeLone and McLean model (2003), which is a reference in the field of information system evaluation. The six model indicators will constitute the key performance indicators of the study and will also serve in the construction of the analytical hierarchy in the next step.

B. Visualization and Calculation
This step is purely technical for the visualization and calculation. It consists of a first sub-part based on concept analysis more specifically concept lattices. The visualization of the binary tables lead to construct concept lattices that give pertinent information based on the studied process and indicators. We can apply this step to each combination of process and indicators that will be store in database for analysis and recommendations.
The second sub-part concerns the application of two famous MCDM methods namely Ahp and Topsis to evaluate the information system performance and even to rank the studied system with others; for the purpose of testing and verifying the proposed framework on others sector. For this reason, we have developed a prototype of software that implements the two methods. It is developed in Java language under Net Beans platform; we will present two principals interfaces ( Fig. 3 and 4).

C. Results and Recommendations
The database stores data that has two main uses. The first is the use of all concept lattice generated of formal concept analysis to evaluate synchronization between different company processes to detect inconsistencies. As for the second, it concerns the results of the application of the MCDM methods, which allow on the one hand evaluating the information system success and the possibility of ranking the studied system with respect to other systems based on the chosen evaluation indicators. This framework can produce recommendations that can help both to better synchronize company processes and evaluate the performance of information systems based on the study of possible combinations between processes and information system success indicators.

IV. CASE STUDY
In this work, we present a case study of an information system belonging to a Moroccan bank. We chose to implement our framework on a company in the banking sector since it is known as the most costumer sector of data processing and specifically information systems. Furthermore, the banking example appears as an analytical framework adapted to a study of process mapping. The bank operates through a set of correlated or interacting activities that use inputs to produce a result; these activities are grouped together as a process. In this paper, we apply formal concept analysis to evaluate some processes according to IS success indicators. The same approach can be done iteratively on all the processes of the bank. Subsequently, the MCDM methods will be applied to evaluate the IS success through the chosen indicators. As part of this work, we will just give an example of a process.
A. Implementation 1) Data Collection: The first step in our framework is data collection to identify the processes and indicators or evaluation criteria that will be the focus of the study.
It has already been mentioned that our approach can be applied to any combination of processes and indicators that allow the evaluation of these processes, in this case study we have chosen a list of processes (Table II) as well as a set of criteria (Table III)   Designates future intentions to use the system that is related with the use of the system and directly affects the net benefits;

I7 Assurance
The knowledge and courtesy of employees and their ability to convey trust and confidence;

I8 User Satisfaction
It influences the advantages provided by an information system and indicates whether a user is satisfied or not after using a system;

I9 Organizational Impact
It includes three sub-dimensions that are: strategic benefits, information benefits and transactional benefits. 2) Data Collection: After the data collection, we proceed to the technical step; the first sub-part is based on concepts analysis, more precisely concept lattices.
We consider the set of processes {P 1 , P 2 , P 3 , P 4 , P 5 , P 6 , P 7 andP 8 } that can improve the set of indicators {I 1 , I 2 , I 3 , I 4 , I 5 , I 6 , I 7 , I 8 andP 9 }. The formal context noted C is represented in the form of a table (in rows processes and in columns the indicators). If the process Xi ameliorates the indicator Yi then the cell Cij is marked by 1 (in the otherwise 0) as shown in Table IV.
The corresponding formal context <X, Y, I> contains the following formal concepts: The second subpart concerns the application of two MCDM methods chosen for this study. We will start with the first method namely the AHP (Analytical Hierarchy Process), and then we apply the TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution). a) Ahp method Implementation: The weights of the indicators (information quality, system quality, system use, reliability, individual impacts, intention to use, assurance, user satisfaction and organizational impacts) are estimated using AHP method. The data of this study are collected by an online questionnaire via the Google docs .   TABLE IV. REPRESENTATION OF THE FORMAL CONTEXT C   I1  I2  I3  I4  I5  I6  I7  I8  I9   P1  1  An aggregated pairwise comparison matrix (Table V)  b) Topsis method Implementation: In this work, we use the MCDM method named TOPSIS, which uses the weights calculated, by the previous method. The main purpose of this sub-part is to rank the system among others in terms of IS success in order to detect the sources of errors.
We will refer to the previous work [20] that focused on the evaluation of the banking information system in which five banks were compared. The bank of this study is referenced B S to differentiate it with the others; the values of the relative closeness to ideal solution are shown in Table VI.   TABLE V.  AGGREGATED PAIRWISE COMPARISON MATRIX   I1  I2  I3  I4  I5  I6  I7  I8  I9  I1  1  5  3  5  3  3  5  3  3  I2  0,2  1  5  7  3  5  7  7  3  I3  0,33  0,2  1  3  5  1  5  3  5  I4  0,2  0,14  0,33  1  3  7  1  3  3  I5  0, According to the value of CCi, the alternative with the lowest closeness coefficient is the bank of the case study with CCi= 0,282.

3) Results and discussions:
This last part concerns the display of evaluation results as well as the discussions. Visualization of the concept lattice gives a hierarchical representation allowing knowing the relation between the processes of the company and the evaluation indicators of the system. The figure above (Fig. 5) represents the concept lattice of case study; it consists of nodes and segments and contains all the objects and properties of the context, each node corresponds to a formal concept.
The figure can be analyzed as follows: Each process labeled by a set of indicators implies that the descendant objects of this process inherit the same indicators. Even more, in the figure, the intention of the formal concept corresponds to the empty set and the extension corresponds to the set of all the processes.
By analyzing more closely the lattice, we note that three set contained only one process. These are the P 4 , P 6 and P 7 processes that represent: adapt and Integrate applications, ensure system security and provide user guides for new systems. Which implies that the bank has problems in the performance of these processes, which is explained by the concept lattice obtained, and view the domain of the company the first critical problem that it has to handle is the security of the information system.
Regarding the application of MCDM methods and more specifically the AHP method, the bank that represents the case study is more interested in the indicators related to the quality (information quality, system quality, service quality) forgetting the importance of net benefits (individual and organizational), use of the system and user satisfaction.
The weight of the indicators influence directly the ranking of the bank obtained by the Topsis method, we note that the bank of our case study is ranked the last among the others banks belonging to the previous works. This ranking can be explained by several reasons: the concentration on indicators possibly related to the different types of quality (information, system and service), the mismanagement of the resources of the company, the lack of experience and skills for the system use in summarizing what has been said before, the bank must address the problems related to the system security management processes, establish user guides to ensure the proper control of the system and therefore the satisfaction of users and facilitate the steps process of adaptation and integration of new applications.

V. CONCLUSION
The purpose of this article is to propose a framework to evaluate the information system success based on the synchronization between the processes of the company. This work focuses on two main areas: the first is the formal concept analysis in order to identify the level of synchronization between the processes and the evaluation indicators that can impact these processes. The second is the MCDM methods, which aim to classify the studied system among the others based on the chosen evaluation indicators. In this paper, we have proposed a framework implementing the set of techniques and methods that were used to end up with an approach to evaluate the IS success that has been applied as a case study which can be generalized on any company. In our future work, we will work on other sectors to know if the change of the sector impacts on the relation between processes and evaluation indicators and therefore on the evaluation.