Framework for Automatic Development of Type 2 Fuzzy, Neuro and Neuro-fuzzy Systems

—This paper presents the design and development of generic framework which aids creation of fuzzy, neural network and neuro fuzzy systems to provide expert advice in various fields. The proposed framework is based on neuro fuzzy hybridization. Artificial neural network of the framework aids learning and fuzzy part helps in providing logical reasoning for making proper decisions based on inference of domain expert's knowledge. Hence by hybridizing neural network and fuzzy logic we obtain advantages of both the fields. Further the framework considered type 2 fuzzy logic for more human like approach. Developing a neuro fuzzy advisory system is tedious and complex task. Much of the time is wasted in developing computational logic and hybridizing the two methodologies. In order to generate a neuro fuzzy advisory system quickly and efficiently, we have designed a generic framework that will generate the advisory system. The resulting advisory system for the given domain is interactive with its user and asks question to generate fuzzy rules. The system also allows provision of training sets for neural network by its users in order to train the neural network. The paper also describes a working prototype implemented based on the designed framework; which can create a fuzzy system, a neural network system or a hybrid neuro fuzzy system according to information provided. The working of the prototype is also discussed with outputs in order to develop a fuzzy system, a neural network system and a hybrid neuro fuzzy system for a domain of course selection advisory. The generated systems through this prototype can be used on web or on desktop as per the user requirement. I. INTRODUCTION Neuro fuzzy is an emerging branch of artificial intelligence. Various types of expert and intelligent systems are being developed using this methodology. Neuro fuzzy methodology involves proper hybridization of neural network with fuzzy logic. The fuzzy logic can be further fuzzified into type 2 fuzzy logic; this provides much better approximation and reasoning capability for the derived solution. However developing neuro fuzzy system is complicated and tedious task, because to develop a neuro fuzzy system, one has to develop neural network along with fuzzy rules and hybridize


INTRODUCTION
Neuro fuzzy is an emerging branch of artificial intelligence.Various types of expert and intelligent systems are being developed using this methodology.Neuro fuzzy methodology involves proper hybridization of neural network with fuzzy logic.The fuzzy logic can be further fuzzified into type 2 fuzzy logic; this provides much better approximation and reasoning capability for the derived solution.However developing neuro fuzzy system is complicated and tedious task, because to develop a neuro fuzzy system, one has to develop neural network along with fuzzy rules and hybridize the two components, also while hybridizing the developer has to take care that the input/output of the neural network are in accordance with the fuzzy rules [2].Hence developing neuro fuzzy system consumes much time and efforts for training such system appropriately [6].Hence there is a need to develop a framework which automates development of neuro fuzzy systems.This paper presents the generic design model for development of such neuro fuzzy system.Currently no such system exists and few are there which aids in development of individual areas of neural network and fuzzy logic separately.Software like ANFIS (Adaptive Neuro Fuzzy Inference System) or DENFIS (Dynamic Evolving Neuro Fuzzy Inference System) which are developed using MATLAB uses given input/output data set, the toolbox function anfis constructs a fuzzy inference system (FIS) whose membership function parameters are tuned (adjusted) using either a back propagation algorithm alone or in combination with a least squares type of method [1,16].This adjustment allows fuzzy systems to learn from the data which are to be modeled.However these two software's lack's the usage of type 2 fuzzy membership function which can be used during fuzzy inference as described by [12,13].Hence they are limited to type 1 fuzzy system which is merged with the neural network; however the result or prediction of such system is just derivation of mathematical formulation as type1 fuzzy logic is generalization of crisp logic.Hence this kind of systems lack proper reasoning and produce more mathematical result than human understandable results.To develop a system which takes input in form of layman's reasoning and understanding ability and give output in the same manner as expected by human being, the developer has to take help of type 2 fuzzy logic.Type 2 fuzzy logic interpretation reflects to human intelligence and logical reasoning ability this is demonstrated by [3,8].To generate type 2 fuzzy sets and separate rules the developer has to deal with development of type 2 fuzzy member ship function which are also complex mathematical computational tasks.Another important concept about the neuro fuzzy system is to train and keep the neural network in balanced state along with inferring fuzzy rules properly with the trained neural network.Hence while developing neuro fuzzy system manpower is consumed in http://ijacsa.thesai.org/developing neuro fuzzy systems, rather than its training and its s practical implementation.To avoid such wastage of time in development of neural network and fuzzy membership function, it is advisable to have generic neuro fuzzy development framework which will suit to develop different neuro fuzzy advisory systems in different subject areas and fields.The framework is developed using Microsoft dot net technology.Much of the code is developed using C#.net programming language.The development methodology addresses our problem domain to generate framework for automatic development of neuro fuzzy system by achieving following two objectives:  The users of the framework have to specify what task to perform on basis of their broad objectives and need not specify how to carry out task.


The output of the framework is an interactive advisory system which can be used by any novice user.Chapter II discusses the three methodologies used in the framework.Chapter III discusses neural network system, chapter IV discusses Type 2 fuzzy system and chapter V discusses neuro fuzzy system methodologies with an application example of each of them.

II. METHODOLOGY
The proposed framework allows generation of neuro fuzzy systems; hence to generate system with only one of the two components namely artificial neural network and fuzzy logic is also possible.Hence with our frame work it is possible to generate three kinds of systems which are listed as follows:  Artificial Neural Network Systems: The system where only training is required to reach the result along with mathematical activation functions are developed under this module. Fuzzy Systems: The systems where only reasoning with proper logical condition is required are developed under this module.
 Neuro-Fuzzy System: The hybridization of artificial neural network with fuzzy logic in proper manner, where both training and logical reasoning are required is developed under this module.
The framework is designed with different learning algorithms for artificial neural network which also allows control over different respective learning parameters.The main interface will provide three options to choose development in one of these three categories.The interface of framework is displayed in Fig. 1.There are three buttons respectively for three different systems.Pressing any one of these button will open an interface of the selected button's development approach.

III. NEURAL NETWORK SYSTEM
This module of the framework allows developer to generate and train artificial neural network according to developers own requirements.Artificial neural network is a computational simulation of the biological neurons inside a human brain [14].It mimics the human mind by using artificial neurons, input hidden & output layers and learning strategy in terms of learning algorithms.The developer of the neural network system has to specify parameter such as input/output variable, number of hidden layers and number of hidden neurons in each layer, selected training methodology namely supervised learning or unsupervised learning or reinforcement learning, specify learning algorithm along with its parameters & finally training set data for the artificial neural network.The developer has to train the network by specifying its learning rate, training loops (iterations), bias values if any and select the activation function as in [7,18] to transfer input to proper output.Fig. 2 shows sample screen for the neural network module.The trained neural network configuration can be saved and its output can be saved in excel style sheet file.

IV. TYPE 2 FUZZY SYSTEM
This module of the framework allows developer to generate Fuzzy Inference System with the use of type 2 fuzzy logic.Fuzzy logic presents the concept of linguistic variable as shown by [5].In order to infer the reasoning behind the fuzzy logic one has to use fuzzy set operations [15].However the drawback of fuzzy logic is that, it's a generalization of crisp sets.In the process of removing the limitation of fuzzy logic sets, type 2 fuzzy logic sets were introduced.Type 2 fuzzy logic sets provide better understanding and support to human logical reasoning.To use type 2 fuzzy logic sets, various set operations and membership functions are used on type 2 fuzzy logic as shown by [4,9].This will infer almost accurate logic behind the current problem domain like a human expert.Type 2 fuzzy logic covers the region called footprint of uncertainty under it [19].The developer of the type 2 fuzzy system has to specify fuzzy variables and condition.The membership function is chosen according to the application of the system to be developed.The chosen membership function is applied to generate fuzzy inference for the given problem domain.The architecture of type 2 fuzzy system is shown by [3].Type 2 fuzzy systems additionally includes type reducer that converts type 2 fuzzy logic to type 1 fuzzy logic, rest of the components in both type 1 and type 2 fuzzy systems work in similar manner.Fig. 3 shows sample screen of type 2 fuzzy systems module.
Inferred rule base is applied to the problem domain to obtain desired result with human understandable logic and reasoning abilities.More the number of rules more accurate are the predictions.If all possible rules are covered then the system will work as human expert generating advice to the respective problem domain.

A. Example 2
Consider a fuzzy system in academic field, then considering percentage & human intelligence as fuzzy variable the conditions generated for student in the situation for deciding to pursue further studies are mentioned in Fig. 4. V. NEURO -FUZZY SYSTEM This module of the framework allows developer to generate neuro-fuzzy systems.Neuro-fuzzy systems are expert systems based on hybridization of artificial neural network with fuzzy logic.The fuzzy rules are mapped with input/output broad categories of the artificial neural network.The power of machine learning that mimics learning of human brain and rule base generated from various domain experts make neuro-fuzzy system an expert intelligent system that can make decision on its own to guide and solve various day to day problems.Neuro-fuzzy systems are used widely when confusion and dilemma prevails in human mind, they not only support to overcome the dilemmas but also provide their decision with a strong support of logical reasoning.Thus behaving like an intelligent expert of the given problem domain.Fig. 5 shows sample figure of neuro-fuzzy system development module.
To facilitate the hybridization process, the developer has to select the options which will allow generating fuzzy variables and fuzzy inference before and after the neural network processing.This process will help to decide the architecture of system to be generated.The resulting system will be neurofuzzy based decision support system as in [10,11].

A. Example 3
Consider a neuro-fuzzy advisory system on course selection that will allow students and their parents to choose the carriers which are most suitable for the student's growth and development.By hybridization of Example 1 & 2, we get a neuro fuzzy decision support system for course selection.The resulting system when displayed in web is seen in Fig. 6 & Fig. 7.In Fig. 6 we see an interface to display educational detail of the system user.The user of the system has to fill in appropriate educational detail from the options provided by the system.On the basis of educational details provided by the user as fuzzy input to the system; the system will combine neural network broad categories as mentioned in Example 1 and apply fuzzy rules as shown in Example 2 to generate neuro fuzzy advice to user, which is displayed in Fig. 7.

Advantages of the Proposed Framework
 Generates Web based and Workstation based system in Neural Network, Fuzzy and Neuro-fuzzy area with respective problem domain. Reusability of the code for new development and modification in current systems. Faster development of neuro-fuzzy system without going into computational details of programming. The developer will just specify what to use and apply for development of the system, there is no need to specify how to apply. Availability of generic library for development.The library of functions and algorithms will also help to develop other major system in the field of computer science. It will save time and effort for development of neuro fuzzy systems.
 The developer will be able to concentrate on working of the system rather than making of the system. The resulting neuro-fuzzy system generated from the framework will be used by layman to solve their daily dilemma and help own self with proper guidance. Many experts knowledge combined with their wisdom and experience will be at finger tips of the user of generated system. Expert's knowledge is documented by the system and can be applied even years after it was actually fed to system. There is no retirement date of the system, once the system is in stable state the developer can continuously monitor and improve its performance by changing parameters and adding rules to the system. This framework is developed using Microsoft's Dot Net technology (Visual Studio 2010) and can be updated easily to future release of versions.

Future Scope of the Proposed Framework
The framework is developed to create and hybridize artificial neural network with fuzzy system for which class libraries have been already developed.In future it is possible to introduce more concept of artificial intelligence like Genetics, ACO, Chaos Theory, Swarm Intelligence etc as in [17,20] to hybridize with existing methodology to generate wide range of application areas of artificial intelligence.
The source code libraries of the framework will aid in developing various kind of neuro fuzzy application like time series analysis, neuro fuzzy controller for different appliances and many such respective areas of neuro fuzzy development other than neuro fuzzy advisory systems.

VI. CONCLUSION
In this paper we have proposed design of generic framework for automatic development of neuro-fuzzy advisory system.The framework shows development of three kinds of systems namely Neural Network Systems, Fuzzy Systems and Neuro-Fuzzy Systems.We have presented design, development strategy and a application example for each of these systems.We have proved that generic framework for development of neuro-fuzzy system is very important as it has many advantages.The derived soulution of the proposed problem domain is explained clearly in Example 3. The resulting advisory system genrated from the example is ongoing UGC based major research project for developing neuro fuzzy decision support system for course selection for students and their parents.The genrated system acts as an expert having domain in the field of academics and guides students as well as their parents that which courses are most suitable for respective students and justifies the advice generated with proper reasoning and concepts.The proposed generic framework will boost development of neuro fuzzy advisory systems and hence provide common man to take advice of field expert of the respective problem domain at finger tips at any place and at any time.

Figure 1 .Figure 2 .
Figure 1.Interface of the Proposed Framework

A. Example 1
An artificial neural network with eight input broad categories and with six output broad categories with three hidden layers of seven neurons each having threshold value at 0.6 and learning rate using back propagation algorithm is considered.The training set data are collected through questionnaires from Department of Computer Science & Technology, Sardar Patel University, which offers different courses at post graduate level and conducts PhD programmes.The department compromise of more than 500 students whose data are collected individually for training neural network.The sample test case data for input and output are shown in

Figure 6 .
Figure 6.Educational Detail of User.

Table I &
Table II respectively.

TABLE I .
DATA VALUES SUPPLIED FOR INPUT BROAD CATEGORIES OF ARTIFICIAL NEURAL NETWORK

Field Name Percen tage Stre am Area of Interest Scholars hip Family Background Technic al Skills Knowledge Domain Category
Selected Categories for Data 1: Job/Business Selected Categories for Data 2: Job/Business, Further Studies http://ijacsa.thesai.org/