An Inference Mechanism Framework for Association Rule Mining

—Available approaches for Association Rule Mining (ARM) generates a large number of association rules, these rules may be trivial and redundant and also such rules are difficult to manage and understand for the users. If we consider their complexity, then it consumes lots of time and memory. Sometimes decision making is impossible for such kinds of association rules. An inference approach is required to resolve this kind of problem and to produce an interesting knowledge for the user. In this paper, we present an inference mechanism framework for ARM, which would be capable enough for resolving such problems, it would also predict future possibilities using Markov predictor by analyzing available fact and inference rules.


INTRODUCTION
Association rule mining (ARM) is the well-researched data mining technique [7,9].Most popular ARM application are market basket analysis, which uses a rule based knowledge representation which refer to the relationship between objects, it was first introduced in 1993 [2], in 1994 R. Agrawal and R. Srikant provided a candidate generation based technique formally Apriori algorithm [1] to generate rules, it outperforms when support count is high and number of items are less.The second approach for ARM is Frequent Pattern growth mining formally FP-Growth approach [10] proposed by J. Han, J. Pei and Y.
In 2000, it is two pass technique where in first pass it counts the number of occurrences of objects and second pass generates the Frequent Pattern tree (FP-tree), FP-Growth outperform when support count is low, but it requires much storage to design and store a tree structure space in case large transaction set is given.Other approaches are matrix based approaches which use Boolean logical and arithmetic operations to generate association rules [18,11,23,8,19,16,4,24] the pros of Boolean matrix based approaches are -It consumes less memory due to their bit data format and makes possible to access and process the huge Boolean relational database to generate frequent patterns.ARM algorithm uses interesting measures like support, confidence and additional measures are Lift and Conviction.
The major problem with association rule mining approach is that, it generates a huge number of rules that may be redundant and insignificant; here the decision making process is complex due to these useless rules so there is a need of an approach which is capable to find interesting rules to take inference decision.In this paper, we propose an inference mechanism framework for association rule mining, which analyzes the association rules and generate inference rules as well as future possibilities [5] using the Markov predictor.
The rest of this paper is organized as follows: Section 2 discusses the discovery of strong association rules, section 3 gives a brief overview of the inference mechanism in rule based systems, section 4 discusses related work and literature review, section 5 presents a detailed description of the proposed inference framework for ARM, section 6 explains problem with a real time example of medical database, section 7 discusses about obtaining results and section 8 finally concluded the paper.Confidence is a measure of strength of the association rules, suppose the confidence of the association rule X→Y is 80%, it means that 80% of the transactions that contain X also contains Y together.Confidence,

II. DISCOVERY OF STRONG ASSOCIATION RULES
To find frequent patterns and discover interesting rule also uses some additional measure like Lift and conviction.Lift (X → Y) < 1: So that the occurrence of X is negatively correlated (or discourages) with the occurrence of Y.
Lift (X → Y) = 1: So that X and Y are independent and there is no correlation between them.Definition 9: Conviction is the ratio of the expected frequency of occurrence of X without Y, that means -the frequency that the rule makes an incorrect prediction (if X & Y were independent)‖ Conviction(X → Y) = 1 − Supp(X)/[1 − Conf(X → Y)] Properties of a good measure.

A. Inference Mechanism
In the branch of knowledge engineering and artificial intelligence an inference mechanism is an approach that helps to drive answer from the knowledge base [6]; An inference mechanism works as a control strategy in decision making system, it processes the knowledge base by applying given facts to derive new knowledge, it uses reasoning by matching and unification of similarity between the objects.An inference rule has two parts, an -IF‖ closure and a -THEN‖ closure, for example, if a patient has symptoms S1, S2, S3 then he/she has the Disease1 i. e. X (S1) ⋀X (S2) ⋀X (S3) → X (Disease1)

Data driven reasoning 1) Start with known data (fact). 2) Fires the rule that has an antecedent that matches the facts in the database and add any reasoning facts to the database
Each rule can fire only once.
When no more rules can fire, then stop.
 Backward Chaining -The inference engine works backward from a conclusion to be proven to determine if there is data in the workspace to prove the truth of the conclusion.

IV. LITERATURE REVIEW
Many approaches are well investigated in the literature for Association Rule Mining, these approaches are based on candidate generation [1], tree based approaches [10] and matrix based approaches [8,11,19,24].But the limitation with ARM algorithms is that it produces a huge number of rules that might be superfluous, dead and also useless to overcome this problem some innovative approaches are well investigated in the literature as a rule based inference mechanisms these are as follows.
Chang-Hung Lee at al. proposed an algorithm PPM (Progressive-Partition-Miner) for mining general temporal association rules in publication databases [17], in this they first partitioned the publication database on the basis of exhibition periods of items, in first scan it produces candidate 2-itemset which are used to generates K-temporal item sets and K-sub items set, in second scan it generates frequent Ktemporal item sets and K-sub items set this algorithm employed scan reduction technique to effectively reduce the number of database scans.
In [22] Jian-Bo Yang at al. proposed an approach generic rule-base inference methodology using the evidential reasoning (RIMER) in this they proposed a new knowledge representation scheme in a rule base by analyzing existing knowledge base structure using a belief structure.In [21] S. Venus proposed a rule based backward chaining inference engine which is an Arabic expert system based approach on natural language for diagnosing diseases.Some built in rule based inference tools are as follows:

A. JESS (Java Expert System Shell) -
Jess is a Rule based inference engine which developed in a scripting language environment written in Oracle's Java language by Ernest Friedman-Hill [14] at Sandia National Laboratories in Livermore, CA, it uses a rule based reasoning algorithm to find inference, user can use it by just adding Jess package in java library and can use its feature by adding java's APIs in there java implementation, it is a light weighted and faster rule based engine.

B. Apache Jena -
Jena is an open source semantic web framework [12] for java, it provides inference java API to use in configuring own inference rules, it facilitate to work with models like RDFS (Resource Description Framework) and OWL (Web Ontology Language) to add extra semantics to users resource description framework data.

C. BaseVisor -
BaseVisor is a closed source rule based forward Chaining inference engine[15], it handles fact in the form of resource description framework (RDF) that triples with support for web ontology language(OWL) and XML schema data types, BaseVisor provides java API to add new features.It requires JRE 1.5.BaseVisor2.0 has following features OWL 2 RL processing, rule and query support, user defined function, user friendly syntax, inclusion mechanism.Basically it is a RDF database management system which has high scalability, loading and query evaluation performance so it is used in research projects and software tools.

V. PROPOSED FRAMEWORK FOR INFERENCE MECHANISM
In this approach we are proposing an association rule based inference mechanism framework, it works in five phases as shown in figure -4.

A. Data pre-processing & Featuress extraction -
This model first preprocess the dataset to map data in the required format by mapping objects/items with appropriate index values for further smooth processing and then examine the existing dataset and extract the features of dataset to decide which ARM approach/algorithm would be most suitable for performing association rule mining to discover frequent patterns.Features like predefined support count, type of dataset, the size of the dataset (Either it has less number of items or high) etc.

B. Selection of ARM Approach based on features of data:
According to the analysis of first phase it selects the most appropriate approach for efficiently performing association rule mining, these selection criteria are as follows: Apriori Algorithm -Outperform when predefined support count is high and number of items are less (If size of dataset is low).FP Growthoutperform when the low support count is given and fast results are needed.It requires much storage space in case large transaction set is given to design and store a tree structure.Discovered rules are stored in rule base which further filters in next step by applying filtering and strength checking techniques.The procedure is as follows:

C. Rule filtering & strength checking:
Phase 2 generates a huge number of generalized association rules stored in the rule base.In this phase the rule filtering and strength checking techniques have been adopted to find interesting rules, the process of rule filtering is as follows: Let K_Rule is a knowledge base and Rule [i] is an array of rules where N is the number of rules then.

D. Fact matching & generating inference rules:
In the fourth phase, we adopt forward chaining technique to discover inference rules.In this inference function process the filtered rule base by matching rules to the given facts (fact repository) to derive new knowledge (inference rules), it uses reasoning by matching and unification of similarity between the objects.In the process the pattern matcher matches the filtered rules with the available facts in fact database repository, if the rule matches with the fact then rule will be selected as inference rule and consequent part of the fact will be fetched from the respective fact and will store in inference rule database.The procedure is as follows:

E. Results & prediction:
This phase adopted the Markov Model predictor to predict what will happen in the future using the inference rules.The Markov theory was first introduced by a Russian mathematician Andrey Markov [3] and gave the concept of Discrete Time Markov Chain (DTMC) [3,20], Markov chain term refer to the sequence of linked random objects (represented in the form of states) with respective probability of occurred events over each other, where the prediction of next happening depends on the current state of the system.Formally, it often represented as the form of directed graph where each event is represented as a state and weight of edges are represented as occurred probability of states as shown in Figure 5.
Let S be the set of distinct states S= {S 1 , S 2, S 3…….. S n } and P is the set of distinct probabilities when the state takes moves from one state to other P= {P 1 , P 2, P 3…… P n }, Here state (S) denoted as S T at different time slot T. P(S|A, π) = P(S 1 ) P(S 2 |S 1 ) P(S 3 |S 2 S 1 )…P(S T |S 1….. S T-1 ) P(S | A, π) = P(S 1 ) P(S 2 |S 1 ) P(S 3 |S 2 )..

…. P(S T | . S T-1 )
Each state of hidden Markov model is associated with probabilistic function so if O t is the observation at time‗t' generated by probabilistic function F Then F i =P(O t |S t=i ).
If N states S 1 , S 2 , S 3 …… S N are involved in the process, then the Markov chain would be represented as follows.So by multiplying identity matrix with probability matrix for Markov prediction, equation will be as follows Here maximum outstanding probability in between will be responsible for next most probable event.

End do VI. AN EXAMPLE
For example, we use a medical dataset (patient's symptoms information) of a city to predict which disease commonly affects a city.For this purpose, we propose an inference mechanism framework for association rule mining in this, firstly it identifies most suited ARM algorithm on the basis of features (thresholds, object's type and size of the dataset) of giving data set in Table-1 to find association rules.Therefore, it would appear that after 4 months, approximately 50% of patients in the city are more likely to get prone to viral disease.On the other hand, approximately 50% of patients may get prone to Malaria.

VII. RESULT AND DISCUSSION
In this inference approach we are examining a sample database of a city hospital, it contains patients' symptoms information as shown in figure -7, where a MATLAB based Inference System is used to match facts with inference rule to identify disease.Algorithm AIRM accept this data as input and analyzes, after association inference rule mining it provides some inference knowledge as a result, which is represented in the figure-7, histogram shows that, according to the observations, after 4 months the disease probability does not change over time, it becomes steady.Eventually, continuing to multiply our answer by the transition matrix again and again, has no effect.So we can infer that the data can be used full for prediction up to 4 months.In Figure-8 Y-axis and X-axis have shown the probability and months respectively.

VIII. CONCLUSION
In this paper, we have proposed Markov model based inference framework for association rule mining, in first tier we adopted most suitable ARM techniques to find the frequent and interesting patterns.In second tier it checks the strongest rules as well as removes redundant and trivial rules in order to increase efficiency, and find the inference rules by applying forward chaining inference technique.
In tier-3 Markov predictor accepts these inference rules with their respective probabilities to predict about future possibilities.This approach would work as pave for future research because that approach can be used in weather forecasting, medical disease prediction and stock market prediction etc.

Definition 1 : 2 : 3 :Definition 7 :
Let I is a set of items which contains different items I 1 , I 2 , I 3, ………., I n which may occur in different transactions, I= {I 1 , I 2 , I 3,… ……., I n }.Definition Let T is set of transactions contains different transactions t 1 , t 2 , t 3 ,…., t m : T= {t 1 , t 2 , t 3 ,…., t m } where T ⊆ I in transactional data base D. Definition An association rule represented in the form of implication of X → Y where X, Y ⊂ I, X ∩ Y = ∅, I is set of items, X is called the antecedent and Y is called consequent.Definition 4: Let S is the support and C is confidence, then X→Y is said to be an association rule, if the minimum support count S (X → Y) ≥ Min (S) and minimum confidence C (X→Y) ≥ Min (C).Definition 5: Support (S) and Confidence (C) are two important measures of Association rule mining for finding interesting and useful items from user concern, user predefines the thresholds (Minimum support and Minimum confidence) to drop un-useful and uninteresting rules.Definition 6: Support (S) of an association rule is defined as the percentage of records that contain X∪Y to the total number of records in the database.Suppose the support of an item is 20%, it means only 20 percent of the transaction contains purchasing of this item.www.ijarai.thesai.orgSupport is the probability of occurrence of X∪Y in the total number of transactions Support, S (X→Y) = Prob (X∪Y) Confidence of an association rule is defined as the percentage of the number of transactions that contain X∪Y to the total number of records that contain X.

Definition 8 :
Lift is defined as -ratio of the observed support to that expected(if A & B were independent)‖ Lift(X → Y) = Prob(X ∪ Y)/Prob(X) × Prob(Y)Lift (X → Y) > 1: So that X and Y are positively correlated, i.e. the occurrence of one implies the occurrence of the other.

Fig. 1 .Fig. 2 .
Fig. 1.Traditional inference approach  Forward Chaining -It compares each rule stored in the knowledge base with the given facts stored in the database.When the IF part or antecedents of the rule matches the fact then the rule is fixed and its fact part is executed.e.g.Rule -1: If D and E Then F Rule -2: If A and B and C Then D

Fig. 5 .
Fig. 5. Markov Chain for ‗N' distinct states Above transitions diagram can be represented in the form of following transition probability matrix.

Figure - 6
Figure -6 shown the Markov chain, according to above given probabilities.Where Malaria (M) and Viral fever (V) are the states.

Fig. 6 .SSSSS
Fig. 6.Transition diagram (Markov Chain) Figure-6's Markov chain can be represented in the form of the transition matrix (TM) as follows

Fig. 8 .
Fig. 8. Probability comparative study OWLIM is the most efficient semantic repository [13] or a robust inference engine implemented in java with advanced features which is able to load huge number of Resource Description Framework (RDF) statements, it is packaged as SAIL (storage and inference layer), available in two additions BigOWLIM, SwiftOWLIM(free to download and use).
If we consider IP 1 , IP 2 , IP 3…….IP N are the initial probabilities and the initial state transition matrix is S 0 then

TABLE II
Fact repository shown in Table-3, if the antecedent matches with rules of Table-2 then rule are fixed and its fact is executed and commit as inference process is shown in Table-3.In the early stages of Malaria, Viral fever, Chikungunya and Dengue fever symptoms are sometimes similar to these.

TABLE III
In this step inference function perform the backward chaining on the rules stored in the knowledge base and compare each resulting rule with fact stored in fact data base process is shown

TABLE IV .
MATCH RULES WITH FACTS

TABLE VI .
PROBABILITY CALCULATION