Evaluation of OLSR Protocol Implementations using Analytical Hierarchical Process ( AHP )

Adhoc networks are part of IEEE 802.11 Wireless LAN Standard also called Independent Basic Service Set (IBSS) and work as Peer to Peer network by default. These work without the requirement of an Infrastructure (such as an Access Point) and demands specific routing requirements to work as a multihop network. There are various Adhoc network routing protocols which are categorized as Proactive, Reactive and Hybrid. OLSR (a proactive routing protocol) is one of widely used routing protocols in adhoc networks. In this paper an empirical study and analysis of the various OLSR implementations (by different research groups and individuals) has been conducted in light of Relative Opinion Scores (ROS) and Analytical Hierarchical Process (AHP) Online System software. Based on quantitative comparison of results, it is concluded that OLSRd project is most updated and best amongst six variants of OLSR protocol implementations. Keywords—OLSR; MANET; AHP; Routing Protocols


I. INTRODUCTION AND BACKGROUND INFORMATION
We are working on Mobile Adhoc Networks (MANETs) based on 802.11WLAN Standard based mobile devices to build a trustworthy collaborative system.Due to peculiar nature of adhoc networks (as compared to infrastructure networks); the normal routing protocols used in infrastructure or Access Point (AP) based network can not be directly applied or used in adhoc networks.There are many routing protocols defined and used as an outcome of research in adhoc networks such as Adhoc On-demand Distance Vector (AODV), Fisheye State Routing (FSR) Protocol, Destination-Sequenced Distance-Vector (DSDV), Optimized Link State Routing (OLSR) etc.These are broadly categorized as Pro-active and Re-active routing protocols depending upon the type of algorithm used.OLSR is one of the commonly and widely used routing protocol in adhoc networks.It helps in multi-hop communication between the peer to peer nodes connected in adhoc network.

A. OLSR Protocol
OLSR is a proactive routing protocol used by MANETs.RFC 3626 [1] was implemented as OLSR daemon (olsrd) in 2004 [2].As per RFC 3626, the key concept is Multi-Point Relays (MPRs).Unlike link state routing, where every node transmits broadcast messages; in OLSR only MPRs transmit broadcast messages.Only MPRs generate link state information, hence reducing the number of control messages flooding the network.Thirdly, a MPR may choose to report links between itself and MPR selectors.Learning from experiences of OLSR version 1, RFC 7181 has been issued for latest version i.e.OLSRv2 [3].It is updated by RFC 7183, 7187, 7188, and 7466.
OLSRv2 is also a table driven, proactive routing protocol which retains the basic mechanisms and algorithms of its predecessor, however, few enhancements have been done in calculation of shortest routes through use of link metric (other than hop count), simplification of messages exchanged and efficient/flexible signaling framework.It is also the further optimizing of classic link state routing protocol and works on concept of MPRs.There are 02 sets of MPRs selected by each router i.e. 'Flooding MPRs' and 'Routing MPRs' used for reduction of flooding and topology, respectively.[12].
2) OLSRd Plugins -Additional Features: Plugin is an addon pluggable program fragment enhancing functionality of some program or system.Plugins in context of OLSRd is the supportability to load dynamically loadable library (DLL) for the purpose of performing different functions and to generate or process private package types.In Linux DLL functionality is available in .sofiles and in Windows as .DLL file extension.The olsrd plugins design has been chosen due to following reasons [2]: • To add any custom functionality or package, the source code of olsrd is not required to be changed.
• The plugins can be licensed separately as per conditions of user.
• Any language can be used to code the plugins and can be compiled as dynamic library.
• The plugins have backward compatibility.

B. Analytical Hierarchical Process (AHP)
AHP is a type of multi-criteria assessment (MCA) technique for analyzing complex decisions.It measures intangibles in relative terms.AHP is a mathematical as well as psychological approach and a structured technique to carry out complex decisions specially applied in group decision making.It was initially studied and researched in the 1970s by Thomas L. Saaty.It has been improved and applied in solving decision problems until now.A good resource on AHP is available at [13].An AHP process decision making recommends a most suitable choice of alternatives based on user defined criteria.As per this book following steps are involved in an AHP process decision making to recommend a most suitable choice of alternatives based on a defined criteria: • Step-1.The problem is modeled primarily in a hierarchy of three layers as under: • The Goal or Main Objective to achieve.
• The Criteria for evaluation of alternatives.
• The choice of available alternatives.
• Step-2.The elements of hierarchy are pairwise compared based on multiple judgments of each pair of element to establish priorities.
• Step-3.The overall priorities of hierarchy are calculated through synthesis of above judgments.
• Step-4.Check weather the judgments are consistent and conclude final result based on these judgments.
Various types of tools are available in the market to apply AHP such as BPMSG AHP Online System (AHP-OS), Priority Estimation Tool (PriEst), AHP Solver, MakeItRational, Open Decision Maker, AHP Analyzer, AHP Software, ABC AHP Decision Making Software, easyAHP, AHP.net etc.We in our research have used AHP-OS.
1) BPMSG's AHP-OS: AHP-OS is web based tool developed by Business Performance Management Singapore (BPMSG).It is one of the most latest, updated and easy to use web application/tool (developed in php) available online for Multi Criteria Decision Making (MCDM) based on classical AHP by Saaty.It does not cater other MCDM methods such as Fuzzy AHP, Modified AHP(M-AHP) etc and has peculiar advantages, disadvantages and limitations as associated with each method.It calculates weightings or ratio scales (by paired comparison of criterion) and consistency index based on input by the user (either calculated or subjective opinions).Mathematically it is based on calculation of Eigen value problem.The calculation of Eigen value gives the consistency ratio whereas, the dominant normalized right Eigen vector of the matrix gives the scale ratio.It is based on following features which are available to registered users: • AHP Projects -A hyperlink to handle complete AHP projects including group decision support.
• AHP Priority Calculator -A hyperlink calculate priorities based on pairwise comparisons.
• AHP Hierarchies -A hyperlink for defining complete set of hierarchies, evaluation of priorities and alternatives.
• AHP Group Session -A hyperlink for participating in AHP group sessions.
2) Application of AHP in Software Selection: A broad review of application of AHP has been presented in [14], [15].It includes but not limited to selection, evaluation, benefitcost analysis, allocations, planning and development, priority and ranking, decision-making, forecasting in medicines and related fields.Other areas are personal, social, manufacturing sector, political, engineering, education, industry, government, and others which include sports, management etc.AHP is also applied to selection of software.Simulation Software [16], Multimedia Authorizing Systems (MAS) [17], Project Management Software [18], ETL Software [19], Data Warehouse System for Large and Small Enterprises in Taiwan www.ijacsa.thesai.org[20], Forecasting Software [21] etc are best examples in this regard.These studies motivated us to apply AHP to different variants of OLSR software to select the best one to suit in our adhoc network based collaborative system project.

3) Mathematical Modeling in AHP:
A complete and elaborate description of mathematical modeling and application of relevant theorems in AHP is given in [22].The mathematics used in determining and calculating decision hierarchy and overall result in AHP-OS is given in [23].The same is summarized in following paragraphs.
• Scale of Intensity.The scale of intensity from the 1-9 (denoted by x) is used as an integer for each selection while comparison of paired criteria and alternative.The x is transformed into c (which is used as an element in pairwise comparison matrix) as under: Logarithmic scale: Root Square scale: Inverse Linear: Balanced Scale: When w = 0.5, 0.55, 0.6, ..., 0.9.
Power Scale: Geometric Scale: • Row Geometric Mean Method (RGMM).RGMM has been used to calculate the priorities P i , to input the N x N pairwise comparison of the matrix A = a ij .The calculation and normalization is done as under: Calculation: Normalization: • Consistency Ratio(CR).CR is calculated by calculating λ max (the principal eigenvalue) and putting in equation below (calculated by Lonson/ Lamata linear fit):

II. DEFINING OF CRITERIA/ ELEMENTS OF CRITERION
A. General Criteria -Mean Opinion Score (MOS) and Relative Opinion Score (ROS) A general selection criteria based on ROS (as applied in GeoSharing project [24] for selection of an embedded Operating System) where a relative score from 1-5 has been considered (5 being highest score awarded based on observation or judgment of each observer).This type of scoring is relative to one an other (of the projects under consideration) and once carried out by single person can be termed as Relative Opinion Score (ROS).A more relevant ROS can be related as mentioned in Table I.The same type of scoring once conducted through a group of peoples and their average is taken as Mean Opinion Score (MOS).

B. Elements of Criterion
The selection a software for practical usage depends on multiple factors.However, following factors are considered vital and have been considered as a selection criteria for selection of OLSR software.
• Stability of software -Measure of reliability and robustness that it should not crash and it is usable without bugs/ interruptions.• Usage -Is there any developer community which is using the software?User experience of general users and their views are also important factors.
• As discussed in preceding sub section of Maintainability, same ROS scoring is applied with respect to NRL-OLSR and QOLSR.• Considering usage and testing of software by the developer community; we assign ROS of 1 each to NOA-OLSR, OOLSR and PyOLSR.• Based on experiences of OLSRd; OLSRv2 is developed and being improved.Moreover, OL-SRd has been implemented and used in various adhoc networking projects such as GeoSharing, MANET Manager (SPAN), Byzantium, Commotion, Qual.net etc.Hence, ROS of 5 is awarded for Usage criteria element.
• Security -Are security features appropriately addressed in the software and the known vulnerabilities adequately addressed?
• OLSRd has a security plug-in and we assign ROS of 4. • Other variants lacks security feature, hence, we have assigned ROS of 0 to each one of them.
• Cross-platform -Does the software support multiple OS platforms such as Windows, Linux, Mac, Android etc.
• OLSRd is developed for multiple platforms including Windows, Linux, Mac and Android.We award ROS of of 5. • The QOLSR is developed for Linux platform.
We assign ROS of 2. • The NRL-OLSR is also supports multiplatforms and we assign it ROS of 4. • NOA-OLSR is also implemented for Linux.
We award ROS of 1. • pyOLSR is also supports multi-platforms i.e.
Windows, POSIX and Linux.We assign ROS of 3. • OOLSR is also Windows and Linux based.We assigned ROS of 3.
• Other Features -Are multiple features (other than the basic design) provided?
• OLSRd provides multiple features through plugins as discussed in sub-section above.We award ROS of 4.
• The QOLSR provides QoS feature.We assign ROS of 1. • The NRL-OLSR supports fuzzy-sighted routing and Simplified Multi-cast Forwarding.We assigned ROS of 2. • NOA-OLSR supports No Overhead Autoconfiguration; a very important feature in adhoc environment.We award ROS of 1. • pyOLSR works with basic OLSR.We assign ROS of 0. • OOLSR is improved to provide SMOLSR and MOLSR.We assigned ROS of 2.

C. Summary of ROS
The summary of ROS as per our opinion based on discussion in sub-section II-B are as given in Table II.

III. EXPERIMENTATION WITH BPMSG'S AHP-OS
With this definition of basic criteria we move on to our experimentation with BPMSG's AHP-OS.Register with AHP-OS website [22] and login with the provided user name and password.Further steps involved are discussed in ensuing paragraphs.

A. Defining Hierarchy
We defined the hierarchy using the basic node as "Selecting OLSR Software" with node leaf or sub-categories as Stability, Maintenance, Usage, Security, Multi-platform support and Other Features etc.The overall OLSR selection AHP hierarchy along-with requisite criteria and alternatives is as shown in Figure 1.

B. Compare Criteria
Each category of criteria is pairwise compared to find that which criterion has more weight or importance.In our opinion Stability, Maintenance and Security has more importance as compared to Usage, Multi-platform and Other Features criterion of OLSR software.Each pair of criterion was compared in light of following AHP Scale: The selection of pairwise criterion is shown in Figure 2. The overall result of pairwise comparison of each criterion element is shown in Figure 3.

C. Evaluation of Alternatives
On completion of pairwise comparison of criterion, the evaluation of alternatives i.e.OLSR software available in the open source community are also pairwise compared to one another based on subjective opinions or actual measurements.The software also provides group input based decisions as well.The pairwise comparison is made based on the ROS as given in Table II.As an example the pairwise comparison of only one criterion for all the alternatives is shown in Figure 4. Overall status of alternatives is shown in Figure 5.The threadbare analysis of the results recorded in ROS Table II and AHP-OS's Figure 5 reveal that AHP has more granularity with respect to analysis of each criteria element as compared to ROS.

D. Summary of Results
Six OLSR alternatives have been compared in light six criterion elements.Each criterion weighting based on the applicable importance is summarized in graph at Figure 6.Similarly, different variants of OLSR and their overall percentages are summarized as shown in Figure 5.We can clearly see that OLSRd ranked first, followed by NRL-OLSR as second and QOLSR as third.after testing in practical scenarios.Application of AHP-OS decision making helps and provides the objective mathematics to process the inescapable subjective and personal preferences of an individual or a group in making a decision based on various criterion.We have applied and configured a level-2 hierarchy.We intend to apply more relevant sub-criterion to a group of hierarchy such as selection of most adhoc networking project for building a nomadic collaborative information system.