Building an Artificial Idiotopic Immune Model Based on Artificial Neural Network Ideology

—In the literature, there were many research efforts that utilized the artificial immune networks to model their designed applications, but they were considerably complicated, and restricted to a few areas that such as computer security applications. The objective of this research is to introduce a new model for artificial immune networks that adopts features from other biological successful models to overcome its complexity such as the artificial neural networks. Common concepts between the two systems were investigated to design a simple, yet a robust, model of artificial immune networks. Three artificial neural networks learning models were available to choose from in the research design: supervised, unsupervised, and reinforcement learning models. However, it was found that the reinforcement model is the most suitable model. Research results examined network parameters, and appropriate relations between concentration ranges and their dependent parameters as well as the expected reward during network learning. In conclusion, it is recommended the use of the designed model by other researchers in different applications such as controlling robots in hazardous environment to save human lives as well as using it on image retrieval in general to help the police department identify suspects.


INTRODUCTION
There are many ways to describe what is meant by Artificial Neural Networks (ANNs).In essence, the artificial neural networks are modeled after the biological neural networks that constitute the basic building blocks of the nervous system.The biological neuron, which is the basic element of ANN, consists of: soma, axons, dendrites and the synapses.The biological neural network consists of many neurons connected together in a specific way to learn to perform a certain function.Therefore, to summarize, what is artificial neural networks?One of the well known original definitions is:"An artificial neural network is a massively parallel distributed processor that has a natural tendency for storing experimental knowledge and making it available for use," [1].
There are many functions that a human could need in his daily life.Starting from early childhood, that function could be learning how to hold a cup to drink and to adjust the hand eye coordination mechanism using specific neural networks to perform such task.As the network learns to perform a certain function, the whole experience is stored in the synaptic strength between neurons.
The focus of this research will be on single-layer and multilayer artificial neural networks as a model to draw ideologies from.The theory and algorithms behind single-layer and multilayer networks will be briefly discussed in the design section.Having defined Artificial Neural Networks, it is important to shed some light on the artificial immune system.According to Dasgupta and Nino, immunity could be regarded as: "The condition in which an organism can resist diseases, more specifically infectious diseases.However, a broader definition of immunity is the reaction to foreign substances, pathogens, which includes primary and secondary immune responses," [2].
Section two covers the literature related work.Section three describes different ANN structures as well as their general different training methods.Section four, on the other hand, focuses on the artificial immune networks and the interaction between antigens and antibodies.At section five, the new proposed artificial immune network structure is proposed.Section six has a discussion that offers analysis of the proposed ideologies.Finally, section seven wrap ups with the overall conclusion and future work.

II. BACKGROUND
Reviewing the literature during the past decade, it was found that, in general, there are many applications that benefit from the immune system's features like self non-self discrimination, specificity, and memory [3][4][5][6][7].The main idea behind their research was having a network that adapts itself by adjusting the concentrations of its nodes.On the other hand, a few researchers focused on analyzing immune networks to utilize it as a complement to artificial neural networks to make Neural networks more effective [8].On the other hand, other researchers focused on developing artificial immune networks as alternatives to artificial neural networks.Vertosick & Kelly combined the immune network theory with parallel distributed processing concepts to produce an alternative ideology to neural network architectures [9].However, these research efforts, although effective, were done more than a decade ago.Recently, researchers worked on another line of research trying to use hybrid system of artificial immune systems and other systems, e.g.neural networks [10][11][12].The goal of those hybrid systems was to get the best advantages of both systems, but the resultant systems were complicated.
Most of the aforementioned research made considerable efforts to highlight the importance of artificial neural networks as well as the artificial immune networks.However, integrating both network paradigms is not the goal of this research, and www.ijacsa.thesai.orgneither is the replacement of artificial neural networks by a more robust artificial immune network.What is being approached in this research is an attempt in which the complexity of the artificial immune network is being perceived from an artificial neural network lens.It is an exploration of opportunities that is expected to lay a foundation of an array of research projects that will benefit from the findings of the proposed research.One major advantage of this research is to produce a simple, yet a robust, model for an artificial immune network that could be useful in many applications.Moreover, producing a simplified model of AIN is expected to encourage researchers in the field to deploy the designed model in their applications.

A. Structure of Artificial Neural Networks
As mentioned earlier, Artificial Neural Networks build upon the model of the biological nervous systems.Based on the information about the function of the brain within the nervous system efforts were invested to obtain a mathematical model for the human learning habits.The results of these efforts laid the foundation of Artificial Neural Networks [13].The mathematical models started from the smallest entity in the nervous system: the neuron.Scientists started with introducing the mathematical model of the artificial neuron: The Perceptron.
A Perceptron has a set of n synapses associated to the inputs.Each of them is characterized by a weight.Each input signal, is multiplied by its corresponding weight, .The weighted input signals are summed and a bias, , is added to improve learning.Thus, a linear combination of n input signals is obtained, see equations 1 and 2. A nonlinear activation function φ is applied to the weighted sum , and the final output is expected to fire only if it exceeds a threshold θ. eqn.(1) eqn.(2) The activation function could have many forms according to the neuron design (e.g.Log-Sigmoid function and Tan-Sigmoid function).In the case of the Perceptron, the design is simple and entails using the threshold function as it is required to classify two-class inputs only [14].The Perceptron was followed by the introduction of single-layer and multiple-layers neural networks.In single-layer ANN the input layer acts as a fan-in layer that does no processing.However, in the output layer, each node is a neuron, which receives inputs and produces an output according to the previously mentioned process of the Perceptron.The activation functions used at the output layer may be any of the aforementioned functions and not necessarily the threshold function.Multi-layer ANN, on the other hand, consists of more than one processing layer.Considering feed-forward ANN, all calculations are done in parallel in each layer, and the output of one layer is broadcasted to successive layers until the final network outputs are calculated.Only the last layer is called the output layer, while all the layers located between the input and output later are called hidden layers, see figure 1.

B. Training of Artificial Neural Networks
In the previous section, artificial neural networks were categorized according to their structure.Artificial Neural Networks could be also categorized according to their method of learning.Its ability to learn resembles the method that the human brain learns.In general, researchers categorized the learning methods of artificial neural networks to be within three main categories: Supervised Learning, Unsupervised Learning, and Reinforcement Learning [15].The third method of training artificial neural networks is the reinforcement training method, see figure 3.In that method the network learns by trial and error, and the performance of each element is being monitored, which means that there is a certain level of supervision, but there is no predefined desired output.Instead, there is a common goal for the whole network to achieve.Network elements perceive their states and perform actions.After each round of actions, the performance of each element is evaluated and the corresponding critic is assigned.This process is repeated until the network overall goal is achieved.The biological immune system consists of lymphocytes that have two major types, T-cells and B-cells.B-cells are responsible for humoral immunity that secretes antibodies.Tcells are responsible for cell mediated immunity.Each B-Cell has a unique structure that produces suitable antibodies in response to invaders of the system.That type of response is called innate immunity and eventually results in antibodyantigen relations to be stored in case the host encounters the same invader again.In that case, the immune response in expected to be faster given that the network has seen it before, i.e. learned how to deal with it [16,17].
The immune system has Idiotopic networks that use stimulation and suppression among its elements to achieve immunity against antigens (Ags).The part of an antigen that can be recognized by antibodies is called epitope (Ep).As a result of this stimulation the B-cells start to produce Abs (Yshaped).On the other hand, the part of the antibody that can recognize epitopes of antigens is called paratope (P).However, part of an antibody, called idiotope (Id), could be regarded as antigen by other antibodies in the idiotopic network [18,19], see figure 4. V. BUILDING THE NEW IDIOTOPIC IMMUNE NETWORK BASED ON THE MOST SUITABLE ARTIFCIAL NEURAL NETWORK MODEL In the previously reviewed literatures some researchers initiated an idea about how the AIN could be viewed from an ANN ideology.In their depiction, the network structure was abstract, while the pool of Idiotopes and Epitopes was loosely connected.However, there is a need for a more precise realization in order to have a robust artificial immune Network.It is our goal in this research to clarify the depiction of artificial immune networks from the artificial neural networks point of view by giving an explanation for the investigated ideology.That could be done by starting the design from the original constructing Lymphocyte units, and that will eventually lead to a concrete Artificial Immune Network structure.The artificial immune network is not necessarily constructed out of layers as the case in Artificial Neural Networks.Therefore, this project design started from the basic Perceptron unit, B-Cell, and builds its way up to the full multi Artificial Immune Perceptron (AIP) network.

A. Network Structure Primary Design
Figure 5 illustrates stage-1 when an Ag attacking a host.As the features of that Ag, Eps, appear in the host, matching Abs are being produced by different AIPs when being stimulated by the Ag.Based on the concept of complimentarity, each Ep is paired with its corresponding Ab Paratop P, and eventually a set of matching pairs (Ep, P) are produced.For example, AIP#1 could produce antibodies Ab 11 and Ab 12 .Those antibodies could bond with the Ag Epitop Ep 1 if they match it.That process is called affinity measurement.The Ab required concentrations are then decided, affinity maturation, and then sent back to the host to counter the effect of the Ag. Figure 6 illustrates stage-2 where Abs are secreted by the AIPs to suppress Ag stimulation.

B. The Final Design of the Multi-AIP network
The stages depicted in figure 5 and 6 could be combined to form one network structure that resembles the structure of a single layer artificial neural Perceptron network, see figure 7. The input layer has the Ag features Ep 1 , Ep 2 ... Ep n .These features are passed to the output layer where affinity measurement and affinity maturation are handled.The generated Ab's are fed back to the first layer to counter the effect of the Ag Ep's.
Based on the nature within the network, Farmer et al claimed that the interaction within the Idiotopic immune network, which is modeled in the Multi-AIP network, could be governed by the following equation, equation 3: ,where the first term, from the left, represents the total stimulation between different B-cells in the idiotopic network.The second term represents the total suppression.The third term represents the Ab stimulation received by an Ag based on the affinity between them.As for the fourth term, k, it represents the mortality of an Ab [20].N represents the number of Abs, while C represents the concentration of Ab within the network, and m represents affinity.The dynamics of the aforementioned Multi-AIP network structure could be described by explaining the dynamics of its components: antigens and antibodies.However, not all the artificial neural network models could be deployed in this artificial immune network designed model.For example, an artificial neural network models that are based on supervised learning are not likely to be implemented.The reason is that the concept of forward pass and reverse pass would not be fully implemented as a technique for handling the network dynamics.In particular, the concept behind the reverse pass dynamics cannot be deployed, because there is no desired output to compare with the actual output produced at the output layer of the network.However, it was found that there is a common feature among most artificial neural network models.Most models store the learned experience in the weights associated with each input, and in general these weights are updated during the learning process according to equation 4:

eqn. (4)
This equation means that the new weight during learning at time t+1 is based on the previous value of weight at time t plus an increment ∆w.The value of this increment varies according to the learning method.By the end of the learning process the learned knowledge is expected to be stored in the weight values that represent the strengths between network components.Similarly, in the artificial immune network antibodies concentration values changes during learning to suppress antigen behavior.In the end, the learned knowledge is expected to be stored in the concentration values.By projecting the weight update concept on the artificial immune network learning process, it could be concluded that antibody concentration could follow the same ideology: eqn.(5) To accurately model concentrations for antigens and antibodies, theoretical biology should be involved.According to Boer, population growth could be described by a classical logistic equation.Therefore, in this research, antibody concentration could be modeled using equation 6.
eqn. (6) , Where is the initial concentration of the Ab population, is the carrying capacity of the population, and r is the natural rate of increase represented as the difference between the birth rate, b, and the death rate d.Therefore, by controlling and the Ab population size could be controlled.However, it was noticed that as r increases, the concentration curve becomes more steep (i.e. when r 2 >r 1 ), see figure 8. www.ijacsa.thesai.orgThe Ag population mathematical representation is in general similar to the Ab abovementioned representation.However, in this research, the Ag is not assumed to proliferate, b=0, which denotes that the Ag population will ultimately extinct and its concentration will eventually reach zero, see figure 9. Unlike the Ab concentration equation, antigen concentration could be modeled using equation 7: eqn. (7) As learning progresses, Ab and Ag concentrations are expected to change.The value of determines the amount of expected change that takes place during that process.However, as mentioned earlier, that value varies as the learning mechanism changes.
Based on the nature of the Multi-AIP network structure, and the expected interaction between Paratop-Epitop as well as Paratop-Idiotop, the best ANNs model to use would be the one that is based on the competitive network ideology.Therefore, according to reinforcement learning could take the following values during learning as depicted in table I: , where is a discount fraction, between 0 and 1, raised to the power of the number of time steps, and the reward at time t is r t .More succinctly, the change in Ag and Ab concentration could be given by the following equation: The result of the analysis showed that as the number of iteration steps increases, the incremental change an Ag and Ab concentrations also increases see figure 10.However, in general it is concluded that as n→∞ the value of →0.In addition, results showed that as the value of the discount factor decreases, the curve becomes less steep ( ).
VII. CONCLUSION AND FUTURE RECOMMENDATIONS Artificial neural network has been a reliable model for at least the past two decades.In this project, it proved that it can deliver ideologies to inspire the creation of a robust artificial immune network model.There were many artificial neural network models in the literature, but current research analysis showed that the reinforcement learning model suits the dynamics of the artificial immune networks.Despite the diversity of modeling techniques for artificial neural networks, careful selection has been deployed to store the network learned experience as well as the way to update it.Finally, the effect of some network learning parameters values were investigated to better control the network learning process.That resulted in a more robust artificial immune network.In conclusion, it is believed that the designed model could be used for many applications.
The careful design scaffolding process that was implemented in building the model for an artificial immune network led to a robust model.The model in general suits different applications that deploy intelligent behavior.For future work, it is recommended to use the designed network model to build some applications in the Robotics field, for example, where robots can be used in rescue operations.These types of applications can intervene in hazardous situations and save many human lives.Another area that is recommended could be in pattern recognition to help the police department identify suspects even if partial interrogation information is available.That can narrow down the suspects' list, and eventually save time, money, and particularly huge office work efforts.These are some examples of applications that could benefit from the proposed designed model, but in general, there are several applications especially in the artificial intelligence field that could benefit from it.

Fig. 2 .
Fig. 2. Supervised Learning of Artificial Neural NetworksDuring training using supervised learning, the network learns and gains experience from a set of predefined training examples, see figure2.During the training session the input vectors are applied to the network, and the resulting output vector is compared with the desired response.If the actual response differs from the target response, an error signal adjusts the network weights.The error minimization process is supervised by a teacher.In general, supervised training method is used to perform non-linear mapping in pattern classification nets, pattern association nets, and multilayer nets.On the other hand, there is another method of training: the unsupervised learning method.This type of training doesn't require a teacher.In this method, input vectors of similar types are grouped together without a teacher.After training, when a new input pattern is applied, the NN provides an output response indicating the class to which the input pattern belongs.If a class cannot be found, a new class is generated.

Fig. 3 .
Fig. 3. Reinforcement Training of Artificial Neural Networks IV.INVESTIGATE THE ARTIFICIAL IMMUNE NETWORK PARADIGM

TABLE I .
VALUES DURING A LEARNING EPISODE