Evaluating the Effectiveness of Decision Support System: Findings and Comparison

Nowadays, regardless of the popularity and credibility of Decision Support Systems (DSS), measuring the efficacy of the decisions taken by the DSS is yet to be proven. As previous works identifies the complexities involved in measuring the efficiency of DSS, most of the time DSS efficiency is case dependent. The list of methods for collecting and analyzing data, building models, deployment models, data and model integration, and finally taking decisions are some of the major issues related to measuring DSS effectiveness. This paper focuses on measuring the effectiveness of DSS. The paper highlights the issues that still need to be addressed with efficient frameworks. Based on the literature review and discussion presented in Section I and II, this study proposed a framework and its implementation. Presents how the proposed model can improve the previous work. The major findings of this study reflect that every decision made by DSS is based on the collected data, analyzed by DSS tools, as well as depends on the developed models. Therefore, this study illustrated that each component of DSS plays vital role in measuring the effectiveness of DSS whatever the case and problem for which the DSS has been built and implemented for. In addition, the supporting methods and measuring factors for each component are other findings of this study. Any decision taken by DSS will be evaluated separately in order to measure the effectiveness of the system. The proposed framework resembles a new framework for the decision makers working in


I. INTRODUCTION
A decision support system (DSS) is a kind of information system that is developed to help organizations in storing, managing, analyzing and ultimately supporting managers in decision making process [1]. DSS is an old term, and researchers have been describing its major components from different perspectives, such as problem solver, system function, and development process [2]. In the late 1960s, an updated version of information system proposed based on model-oriented was known as decision support system in order to support organizational decision making process [1]. Since then the evolution of DSS has started at where different research articles were published in different information systems related journal. Furthermore, as described by [3], a famous business journal in 1979 took an advanced initiative to offer the researchers to publish articled related to decision making process, business decision systems, strategic management and decision management systems.
Thereafter, researches have been elaborating on DSS characteristics, components, and deployment in different industries. Vast literature works discovered that DSS implementation and usage has been tested and verified in different kinds of business and their applications. Examples covers areas such as logistic [4], customer relationship management [5], enterprise resource planning [6], supply chain management [7], clinical information system [8]. A system architecture of DSS based on GIS is proposed in [9], which can provide help in alarming disastrous situations such as floods in order to reduce disaster effects on living communities. Predicting and assessing the employee's behavior in an organization is a work presented by [10], using analytic network process. Another proposal has been presented to evaluate the customer reviews impact decision making while purchasing online [11]. The study highlighted that new customers are very much influenced by previous consumer reviews that ultimately help them to make purchasing decisions. DSS application employed in hospital industry as in [12]. The research proposed a diabetes decision support application using fuzzy based expert system. The idea is to provide assistance to medical staff and doctors in diagnosing the diabetes with potential patients.
Making a proper decision demands adequate data analysis, analytical and statistical study, evaluating alternatives, and evaluating optimization criteria for decisions, etc., which might result in exhaustive and extensive analysis. Given that DSS consists of four major components; namely, user interface, data, model, and knowledge base [15], this research proposes a framework that is component dependent for measuring the effectiveness of DSS by considering the importance of each DSS component to measure their impacts on decisions, and hence overcoming the associated complexities of decision optimization.
To conclude this section, the DSS is responsible to perform several tasks such as storing, managing, and analyzing vast amount of data to improve the efficiency of decision-making process. In this research, we focus on how an organization can assess the impact of decisions made previously. As such, the study elaborates on answering the following research questions:  Is each component of DSS required same attention while evaluating the effectiveness of the decisions made by DSS? www.ijacsa.thesai.org  What are the supporting methods and measuring metrics can be used for measuring the effectiveness of DSS?
The structure of remaining paper is organized as follows. The subsequent section elaborates the previous literatures discussing the frameworks or methods to evaluate the impact of DSS. The later sections describes the proposed framework and its implementation. Finally, the paper highlighting the future work and summary of the paper in the conclusion section.

II. MEASURING THE EFFECTIVENESS OF DSS
DSS is a kind of information system, which is for an organization to fulfill single or multiple decision demands [1]. Storing and managing data, predicting solutions, or list solutions can be provided depending on the type of problem and its degree of structure; namely structured, semi-structured, and unstructured in [13]. Since, categorizing DSS is based on context degree of structure and set of alternative, the effectiveness of decisions is still a complex issue for the researchers. The explanation regarding the list of alternatives and decision-making criteria given by [13], suggests that DSS effectiveness can be measured based on different types of decision. Mainly, they purposed that building DSS is based on data analysis and reports generated by a DSS with the help of model based and knowledge base. In [14], to measure the effectiveness of DSS, all components including the user interface need to be assessed thoroughly, whereas the list of input and output variables represents major components during evaluation phase as shown in Fig. 1. In [14], many factors such as; business process, participants, information, technology, infrastructure and strategy shapes the adequate DSS.
Many other articles discuss the quality and effectiveness of DSS decision making process [8], [15]- [18]. Since DSS is an information system, it requires a designated measurement model. System's user, organization's environment and success criteria are the major parts of measuring the effectiveness of DSS in [19]. Multiple strategies and models to evaluate different kinds of information system are presented in [6], [20]- [27].  In [13], data management is shown to be a compulsory element of any DSS. In this research a standalone database for specific DSS provides fast and intelligent decision making process. Since querying data and data quality are the basic characteristics of DSS, measuring the data efficacy is a part of evaluation process as described in [28]. Measuring the business intelligence is considered one of the major output of DSS [13], [29]. Data quality represents major part of successful DSS model as shown in Fig. 2 [28].
The above literatures highlight the major components of DSS (i.e. data, model, knowledge, user interface) can be considered as a part of a framework for measuring the effectiveness of decision support system. The discussion also leads to the conclusion that the boundaries of evaluating the decision making process is not limited with data and environment only, but there are other elements which need to be considered equally during evaluation process. This includes feedback from users, user interface, input process, output procedures, reports, analysis and strategy. However, as the development of DSS can either be based on structured, semistructured, or unstructured problem, the complexity of the evaluation process can considerably be high.
The major elements that affect measuring the effectiveness of DSS can be summarized as follows:  Data, user interface, and knowledge building  Model building using quantitative and qualitative analysis.
 Deployment of analytical and statistical methods such as; what-if scenario, data tables, and optimization tool.
 An interactive user interface including with customize query facility, user authentication, and separated dashboard for different kinds of users.
 An efficient and automated building of knowledge base, where different kinds of techniques can apply such as; data mining, classification of data, rules building, and prediction tools.

III. PROPOSED FRAMEWORK FOR DSS WITH EXTENDED MEASUREMENT PHASE
The framework presented in this research is the extension of traditional DSS model in a more structured way. The proposed model helps as supportive method for each www.ijacsa.thesai.org component and measurement criteria for assessing the effectiveness of decisions. As described by [30], that DSS consists of multidisciplinary contexts where it connects combination resources such as database, knowledge base, human computer interaction, machine learning, data analysis and statistical methods. Therefore, the proposed extension of DSS model is relatively exploring the sub-components of DSS need to be deployed to support the decision making process. Figure 3 highlights the composition of DSS of different layers. The description and working of each components is discussed subsequent sections.

A. Data
Data is the fundamental element for any DSS. The data management component provides several functionalities such as storing, process, and organize data, as well as generating reports and templates to enable adequate decision making process.
The data management component provides basic information that is very helpful to many cases such as; finances, funding, and most importantly customer's related strategies. The data management subsystem can be incorporated with large data sources such as; data mart and data warehouse, whereas data analytics and visualization tools can represents the data in more attractive way to the decision makers. Data extraction, retrieval, querying, tracking of data, log files, and flexibility are kind of functionalities that data management subsystem provides to the DSS. All of these functionalities are part of measuring the effectiveness of proposed DSS.

B. Model
The model component in DSS is a complete package including with basic building blocks' capabilities. For example, financial, statistical, graphical, quantitative and qualitative elements are all considered. Model management subsystem contains library of models with model catalog, and query facilities. The model management subsystem's capabilities are similar to data management subsystem. Both subsystems provide the management and tracking facilities. Model management typically depends on the pre-processed data coming from data management subsystem, which ultimately helps in developing appropriate model based on provided data. During measuring phase, model building facility, model manipulation efficiency, model tracking, and model availability are considered factors to assess the effectiveness of DSS.

C. User Interface
User interface is a component directly connected with data management subsystem and model management subsystem. This component is used for direct communication with DSS users with variety of user interface derivatives. In a DSS environment, users represent a major element of the system. They are communicated with through web browser, dashboard, ready-made programmable and customized gateway, and excel worksheet. User interface provide the facility to users to establish a data query from database, forwarded to model management and get the best solution from selected alternatives. The major functionalities of user interface may provide are customized query facility, flexible user environment, integration between multiple data sources using option boxes, capturing and data storage choice, and list of alternatives. Theses functionalities are enabled by the incorporation of user interface with data management subsystem and model management subsystem. Ultimately, the user interface friendly environment eases the work for DSS utilization and provides basic functions available all the time with simple steps. Additionally, the user interface plays significant role in measuring the effectiveness of decision proposed by DSS. Historical data would be available to calculate measuring factors reciprocally. The purpose of measuring phase is to evaluate the effectiveness of decision support system using different factors; i.e. number of queries asked, successful completion of queries, using data management, model management, and user interface.

D. Knowledge Building
Apart from the three essential components of DSS; namely, data management, model management, user interface, knowledge management system is another but optional component of DSS. The knowledge base is supposed to work individually and may provide collective support to other subsystems in DSS environment. The main purpose of knowledge base is to provide intelligence to the decision making process. A knowledge building can be accomplished by connecting knowledge management through organizational database, or it can take utilize external resources such as data warehouse or web servers.
Knowledge building follows a systematic approach to develop an efficient knowledge base management for future purposes. It performs symbolic reasoning and develops artificial neural networks for creating an environment where decision makers can get fast and accurate decisions. The data mining techniques like classification, clustering, and association mining are some famous techniques which may be integrated with DSS. It is very important to enhance the knowledge base within proper timing in order to improve the decision making process. Most of the time, the knowledge building tool is not directly connected to the decision making process except if required and approved by decision makers. On the other side, the main task of knowledge base is to keep updating the kind of problems occurs and the list of alternatives suggested by DSS. In this regard, knowledge base is helpful for predicting the solutions for same kind of problems in future.

E. How the Framework can Implement
Based on the previous discussion, the proposed framework of DSS is an extension of the ordinary DSS with a measurement phase. The model presented in this research is providing the approach for measuring the effectiveness of decisions made by any DSS. Table I presents the details of each element demonstrated in the framework. Overall, this table is showing three columns; DSS components, supporting methods, and measuring factors. For example, the -data‖ is one of the major elements in DSS environment, which is responsible for storing, managing and analyzing data extracted from different sources. This component supports the DSS as it represents data retrieval for analyzing alternatives of a given problem. Furthermore, the -data‖ component helps customizing queries according to the required context and problem statement. The whole process of -data‖ component involves data extraction, data analysis, alternatives development, criteria analysis, and querying facility. Accordingly, -data‖ component resemble the major DSS. The related challenging issue is how to measure the task accomplished by this component of DSS. The column -measuring factors‖ mentioned in Table I, represents efficacy evaluator of -data‖ component. It consists of variables that are extracted from the performed tasks. It reflects how the data being analyzed, communication cost, time taken during data extraction and analysis are some of measuring variables. Table II presents a comparison between previous related works and the enhanced framework presented in this study. It shows the contrast between the models presented in the main related works [15,29] and the proposed model of this study. As such, the major contribution of this research is considering all components of DSS for evaluating their impact on decisions, as each of these components resembles a major value to the decision. www.ijacsa.thesai.org  The framework is supposed to understand the architecture of DSS and its components with respect to data.  The model is not suitable for understanding the comprehensive list of measuring factors which can assess the effectiveness of DSS.
 The proposed framework highlights the list of all components including the knowledge base (an optional component)  The model represents all components with its supporting methods with is helpful to measure the impact of DSS.

The Evaluation
Realms [15] An evaluation framework which is based on assessing the set of alternatives.
 The framework does not consider all components during evaluation phase. It is only based on decision criteria, where the knowledge base is only the part of evaluation.
 The framework elaborates on the several measuring factors based on the integration of all components of DSS.  The framework consider each factors with its supporting methods.
The BIS success model [29] BIS success model is based on IS success model to assess the business intelligence.
 Mainly depend on measuring the business intelligence.  Information access quality and information content quality are part of framework, but model management subsystem is not the part of this framework.
 All four components in measuring the effectiveness are considered including model base, and knowledge base. www.ijacsa.thesai.org

IV. CONCLUSION AND FUTURE WORK
This study discusses DSS components, implementation, evaluation, and domains of utilization. The paper presented the importance of each DSS components and their roles to DSS in decision-making process. It proposed a framework for evaluating these components' effectiveness on decisions by considering the -measuring factors‖ of each component and its relevance for evaluation phase. A potential future work can be validated the framework using a real DSS scenarios implemented in organizations.