FARM : Fuzzy Action Rule Mining

Action Mining is a sub-field of Data Mining that concerns about finding ready-to-apply action rules. The majority of the patterns discovered by traditional data mining methods require analysis and further work by domain experts to be applicable in target domain while Action Mining methods try to find final cost-effective actions that can be applied immediately in target domain. Current state-of-the-art methods in AM domain only consider discrete attributes for action rule mining. Therefore, one should discretize continuous attributes using traditional discretization methods before using them for action rule mining. In this paper, the concept of Fuzzy Action Rule has been introduced. In this type of action rule, continuous attributes can be presented in fuzzy form. So that they can suggest fuzzy changes for continuous attributes instead of discretizing them. Because the space of all fuzzy action rules can be so huge a Genetic Algorithm-based Fuzzy Action Rule Mining (GAFARM) method has been devised for finding the most costeffective fuzzy action rules with tractable complexity. The proposed method has been implemented and tested on different real datasets. Results confirm that the proposed method is successful in finding cost-effective fuzzy action rules in acceptable time. Keywords—Action mining; fuzzy action rule mining; genetic algorithm


I. INTRODUCTION
Nowadays, the increase in computer usage has led to the sharp growth of databases and aggregation of data in the majority of organizations.In recent years, tendency to discovering actionable knowledge from data with the aim of decision making has seen a dramatic increase.Due to the large size of the data, human experts cannot investigate and analyze all data; on the other hand, the capability of fast and effective response to customer needs requires fast enough decisions which cannot be made without accurate evaluation and analysis of information and data.Therefore, discovering knowledge from database in almost real-time is of paramount importance.Data mining is a procedure which uses data analysis tools to find patterns and relationships between data.Data mining methods focus on discovering and extracting descriptive patterns and do not take into account the actionability of these patterns in target domain.For instance, consider the customer loyalty detection system.The conventional data mining systems predict which customer will leave the company in the close future; however, the companies need to know how they can prevent this and the subsequent losses.In fact, companies need profitable actions instead of just frequent patterns.
Action mining is among the proposed feasible solutions which have attracted attention recently.Actions determine for the user what should be done in order to reach high profits.Action is a type of knowledge which aims to maximize profits in desired areas by suggesting a number of alterations to a sample's condition.The methods proposed for action mining can be categorized into transductive and inductive [1].In inductive methods, such as what was done by Ras et al. [2]- [9], the obtained action rules are general rules which can be applied to a group of things with similar attributes.Ras method, which has been introduced as DEAR [2], initially extracts the conventional classification rules in data mining and later, by combining compatible classification rules, creates action rules (rules that suggest alterations in attributes).
Tsay et al. [10] improved the DEAR algorithm by DEAR2 system which initially classifies the rule table based on the number of existing decision attribute values and consequently creating a sub-tree based on stable attributes.At the end, leaves are compared with each other to extract action rules.
Tsay et al. [11] also proposed DEAR3 algorithm for incomplete databases which can reduce the errors of undetermined, unreliable and outlier data to obtain more reliable action rules.
Another method which was suggested by Ras to extract action rules is called action rule discovery (ARD) [3].This method extracts action rules without combining classification rules.The advantage of this method is its lower time complexity that makes it proper for large databases.
Meat-actions are actions that should be triggered to activate another action rule.In [12] Meta-actions have been used for mining business action rules and in [13] surgical meta-actions have been mined for medical diagnosis.
Inductive methods in other hand try to find most profitable actions for each case.The most prominent method of this type is presented by Yang et al. [14]- [15] which uses a traditional decision tree to extract actions.In summary, at first, a decision tree is inferred from the data.Next, for each new sample, it searches a leaf node and computes the net profit made from the movement of the new sample to the other leaves.Finally, the leaf that has the highest profit is selected as the destination leaf and the necessary alterations are performed for transferring the sample from the current node to the destination node.In order to extract action rules, Shamsinejadbabki et al. [1] proposed a new action mining method by using causal networks.This www.ijacsa.thesai.orgmethod can obtain practical actions in the real world by considering causal relationships between the attributes.
Traditional decision making trees (used by Yang) in [15] require data with discrete values and the continuous values need to be discretized before being put in the algorithm.Despite conventional in data mining, the continuous to discretized domain transformation is little compatible with the real world and it is therefore difficult to define a precise border for continuous attributes' values.For instance, consider the question of how the border between middle age and old can be determined.If it is 60 years, a person who is 59 years old is considered middle-age and a 60-year-old person is considered old.Obviously this classification is not compatible with the real world actualities.Using fuzzy logic in traditional fuzzy logic capabilities makes it possible to work with continuous data.Kalanat et al. [16] use fuzzy set theory to improve Yang method when we face continuous attributes in data.They have shown that although the final rules are not fuzzy but their actionability is higher than Yang rules.
In this study, the concept of Fuzzy Action Rule (FAR) has been introduced.This type of action suggests fuzzy alterations for continuous attributes and therefore, it is expected that the resulting actions will be more applicable in the real world circumstances.Besides, a method has been devised for derivation of fuzzy action sets with maximum net profit from data using Genetic Algorithm.
The rest of paper is as follows: In Section 2 preliminaries are explained.Fuzzy Action Rule will be introduced in Section 3 and GA_FARM, a GA-based FAR mining method will be presented in Section 4. Experimental results are argued in Section 5 and paper will be concluded in Section 6.

II. PRELIMINARIES
Set and element are the two primary concepts of set theory.In classic set theory (crisp), x either can or cannot be a member of set S. As a result, the membership function of element x to set S (µS) can be shown as: However, a large number of the real world sets are inherently fuzzy sets.Fuzzy set theory is a mathematical structure which allows an element to belong to more than one set.The membership degree of element x in fuzzy set S as µS can be shown as x→ {0,1} where 0≤µS≤1 [17].
Suppose three fuzzy sets (young, middle age, old) which are attributed to age variable.Now if in a classic set, 60 is considered as the border between middle-aged and old people, this is tantamount to considering a 59 years-old person as a middle age person and a 60 years-old person as old.However, fuzzy theory puts narrow borders between the values so that an element can be classified with various degrees.For instance, a 55 years-old person can belong to the middle-age and old sets at the same time with different membership degrees.

A. Fuzzy Decision Tree
A fuzzy decision tree is a combination of fuzzy theory with traditional decision making trees.The generality in learning from samples, clarity of the knowledge obtained from the decision tree and also the ability of working with fuzzy imprecise and incomplete data are among the benefits of using this combination.It can also increase robustness and applicability in inaccurate and unclear areas.The knowledge obtained from FDT more closely resembles human decision making and therefore, is more practical in real world circumstances.For example, in [18] FDT has been used for software cost estimation to handle imprecise data in describing software.
Fig. 1 depicts an example of FDT from [17] where attribute x 1 is the root and attribute x 2 is the internal node.The v th membership function can be determined from the j th attribute by µ jv .The K th probable class in the L th leaf is determined with β kL .The number of shown classes in tree is Equal to 2. As a result, each leaf includes two fuzzy variables: β 1L and β 2L Fig. 1.A sample fuzzy decision tree [17].

B. Action, Action Set and Action Rule
An action has a (A,a→a')structure in which A is an attribute except the goal (A is called the action attribute) and a, a'є Dom(A).This action indicates change of A from value a to a' by a force outside of the system [1].Action set is a set of actions.For example, (Service, low→high; Bill, notPayed→ payed) is an action set.Action Rule is type of rule that its antecedent is an action set and its consequent is a change in goal attribute.It is shown an action rule below: (Service, low →high; Bill, notPayed→ payed) => (loyalty, no → yes)

III. FUZZY ACTION RULE
In this work the fuzzy action concept is proposed.In fuzzy action, unlike the conventional actions, fuzzy alterations are suggested.For instance, fuzzy action suggests the reduction in smoking rather than cessation for pulmonary disease patients.The same is true with fuzzy action and soft drinks for patients.The motivation beyond this is that if fuzzy changes is considered it is possible to find more cost-profitable actions.Suppose attribute A has a i value in a sample.Then, fuzzy action on A is defined as: Where t is the change granularity of attribute A and can be computed using the following formula: www.ijacsa.thesai.orgt A max -A min n (3) A max : maximum value of attribute A in dataset.
A min : minimum value of attribute A in dataset.
n: determines to what extend t will be fine-grained or coarse-grained.Larger n results in smaller t and therefore more fine-grained fuzzy action sets.k: is change magnitude and can be positive, negative or zero.
If k is positive, the value of A increases and if it is negative, it decreases.It is obvious that if k is considered zero, there is no change in attribute value.
The following example can clarify the concept of fuzzy action: Consider three attributes X 1 , X 2 and X 3 with the values (12,3,5).Suppose that t and K are 0.6 and -2 for X 1 , respectively.By the fuzzy action, X 3 ,5→5 (-2 0.6), X 3 will change from 5 to 3.8.A set of fuzzy actions suggest alterations to multiple attributes and we name it Fuzzy Action Set.For example:

]
Since is constant for each attribute, an action set can be written as: The above action set suggests that value of attribute X 1 does not change, value of attribute X 3 with an initial value of 5 decreases by 2 t and value of attribute X 2 with an initial value of 3 increases by 5 .
Fuzzy Action Rule is type of action rule that its antecedent is a fuzzy action set.E.g. a Fuzzy Action Rule can be like this: [(X 1 ,12,0) (X 2 ,3,5) (X 3 ,5, 2)] ( , ) IV. GA-FARM The major problem in finding fuzzy action sets is the search space.Even in a dataset with a few attributes and small domain ranges of attributes the space of all possible fuzzy action sets is huge.So that exhaustive searching of this space is intractable.As a meta heuristic solution genetic algorithm has been used that is quite strong in solving such NP-Hard problems.We are looking for such rules that net profit, i.e. profit minus cost, will be maximum by applying them.At the following, different parts of proposed GA algorithm will be described.

A. Chromosome Structure
In GA, each solution is shown as a chromosome and each chromosome contains multiple genes.In our work each chromosome represents a Fuzzy Action Set (FAS) in which each gene is a Fuzzy Action.So that number of genes indicates number of attributes in FAS and because number of actions in FAS is various chromosomes' length is also variable.
Each gene has three components: attribute name A, current attribute value V and change magnitude k.So that a gene with structure (A,V,k) represents fuzzy action(A,V→V t).

B. Fitness Functions
The most important part of a GA is its fitness function.It is defined as: Where m is the number of values of goal attribute, profit(i) is the profit of being in class i that will be provided by domain experts.
( i) and ( i) indicate the membership degree of the instance to class i before and after applying the rule, respectively.For example, consider the following chromosome: ch: [(A,1,3),(B,2,0),(C,3,-1)] Also assume t , t , t and goal attribute G has two values G 1 and G 2 .
Cost will be computed using the following formula: Where ( ) is the cost of changing attribute i with value .Cost values will be fed into model by domain experts.

C. Crossover
Crossover is responsible for exploiting search space in GA.It tries to combine two good chromosomes with the hope of generating more fitted offspring.Here a one-point crossover operation has been designed for combining the most fitted chromosomes probabilistically.
For each pair of chromosomes, which are selected using roulette wheel selection strategy, two offspring will be generated by exchanging their genes around crossover point which is determined probabilistically.Designed crossover has been depicted in Fig. 2 for two sample chromosomes.

D. Mutation
Mutation is the core of exploration mechanism of GA.It helps GA exit from local optimums.Mutation operator tries to mutate some genes of a chromosome with the hope of generating a new chromosome from other parts of the search space.
In GA-FARM a predefined percentage of chromosomes are muted by changing value k of some of their genes by a random value.In Fig. 3, the result of mutation on a sample chromosome has been depicted.

V. EXPERIMENTAL RESULTS
For experimental results GA-FARM has been tested on three different datasets: Bank Loan dataset which has gathered from one of the most prominent banks in IRAN (dataset description has been shown in Table II), Pima Indians Diabetes [19] and Turkey Student Evaluation [20] both from UCI Machine Learning Repository.The summary of these datasets can be seen in Table I.
One of the major problems in action mining domain is evaluating the mined actions because it is necessary to apply them in real world.To investigate the effect of proposed method we have designed two research questions: How logical action rules will be mined using FARM and whether GA can converge in acceptable time or not.The former question is subjective while the latter is objective.Fig. 4 shows the results of applying the GA-FARM on bank data sets.It shows the convergence of GA for this dataset.As it can be seen the GA is successful in improving the overall fitness of population.Elapsed for this dataset is 281.31 seconds which confirms that GA-FARM has acceptable time complexity.One of the mined rules using GA-FARM from bank loan dataset has been depicted below: [ (X 1 ,0,0) (X 2 ,36,0) (X 3 ,0, 2) (X 4 ,1,1) (X 5 ,2, 1) (X 6 ,0,3) (X 7 ,0,3) (X 8 ,770000,0) (X 9 ,770000,0) (X 10 ,32, 2) ] This rule shows alterations of the values of two attributes: marital status and the number of deferred payments.There is no change in the attributes where k is zero.For instance, for attribute #10 which is the number of deferred installments, -2 is suggested for k and 0.5 is computed for .Based on this rule, the customer must reduce his or her deferred paid installments by -2 × 0.5, i.e. from 32 to 31.By applying this rule, the bank may reach a net profit of 190.This action rule shows that it is possible to achieve good profit but with less changes in attributes.This is the motivation behind fuzzy action rule.By applying proposed method on Pima Indian Diabetes dataset it is succeeded to find the most profitable fuzzy action Rules.Fig. 5 shows the convergence of GA-FARM on this dataset.One of the mined action rules is: This rule suggests that the patient can reduce the probability of getting diabetes by losing (4.697) kg weight.

VI. CONCLUSION AND FUTURE WORKS
Action Rule is a set of change suggestions with the aim of converting an instance from less profitable class to more profitable one.Current state of the art methods in this domain only suggest crisp changes in attribute values.In this paper the concept of Fuzzy Action Rule has been introduced which suggests fuzzy changes.Applying fuzzy changes help us to gain profit with less cost comparing to crisp changes.
A big burden against finding fuzzy action rules is the huge search space.So that, GA-FARM has been devised for extracting fuzzy action rules using GA.The proposed method then has been implemented and tested on different datasets.Results confirmed that GA-FARM can find optimum fuzzy action sets in acceptable time.
The most important limitation in the field of action mining is about evaluating the results because checking the effect of an action needs to apply it in real world.It is necessary to design some frameworks that can evaluate the effect of actions more realistically.For future works we will also try to devise new FARM methods that can find more profitable fuzzy rules.Incorporating fuzzy type-2 into action rules can be another promising research area.

Fig. 4 .
Fig. 4. Results of applying the GA-FARM on bank loan dataset.

TABLE II
55. Results of applying the GA-FARM on Pima Indians Diabetes dataset.