Level of Confidence in Software Effort Estimation by an Intelligent Fuzzy-Neuro-Genetic Approach

Organizations are struggling to deliver the expected software functionality and quality in scheduled time and prescribed budget. Despite availability of numerous advanced effort estimation techniques overestimation and underestimation occur on a vast scale and results in project failures and significant loss to the organization. The paper proposes machine learning based approach to calculate the optimized effort and level of confidence. Genetically trained neural network evaluates the optimum effort for given COCOMO II variables. The level of confidence is evaluated by fuzzy logic and indicates the percentage that the predicted effort will not exceed the limits. Keywords—COCOMO II; artificial neural networks; genetic algorithm; fuzzy logic


INTRODUCTION
Human dependency on software is increasing continuously.Today most of the goods and services are realized with software systems.Software has become major driving force for progress even in domains that were traditionally reserved as completely mechanical or hardware systems, for instance major advances in automobile industry are being realized with software development.Companies spend 4-5 percentage of the revenue on software development [1].The figure is as high as 10 percent in highly IT dependent sectors, for instance telecommunications and finance [1].Thus, functionality and complexity of software systems is increasing manifold.Simultaneously time to market and cost should be reduced to stay competitive.In United States 250 billion dollars are spent each year on IT development [2].Project management and Effort estimation are key factors for success of a software project.Despite much research and technological advancement in effort estimation techniques, proportion of failed software projects is huge [2,3].According to the Chaos report submitted by Standish Group[2] only 16.2 percent of the projects are successful ,57.2 percent projects are over-budget and provides lesser functionalities than specified and 31.1 percent of the projects are cancelled during their development cycle.The percentage of the successful, challenged (over-budget with less functionality) and impaired (cancelled) projects is shown in Fig 1 .Project failure can be defined as combination of cost overruns, late deliveries, poor quality, and/or developing a product that does not get used.The two crucial reasons for failure of most software projects are Overestimation and Underestimation of the software effort [4].Most projects either cost more than they return or fail to deliver required projects in the expected time.Both the scenarios lead to huge loss or may also result in termination of the whole project.R. Charette [3] suggested unrealistic project goals and inaccurate estimates of needed resources as principal factors that lead to project failure.A. Trendowicz and etal [2] pointed that most of the effort estimation techniques provides point estimates with hardly any support for risk management if project overruns the expected cost.Moses and etal [4] in their research concluded that in addition to estimates the effort estimation should also specify a Level Of Confidence associated with the calculated effort in order to compensate the uncertainty associated with effort estimation.

A. What research has been Conducted so far?
For software effort estimation, numerous methods have been examined specifically data driven soft computing methods such as artificial neural networks, regression trees, evolutionary computing, rule-based induction, fuzzy logics etc.These methods exhibit many advantages like regression over other standard methodologies.Literature of software effort estimation endorse that important product feature characteristically reflects the software size which exactly impact efforts.Basically it is used to build cost models.
Initially, almost all models are based on the size of metrics which contemplates numerous coding lines coded for a software project i.e. lines of code (LOC) or thousands of source line code (KLOC), as shown in COCOMO [5], or function points (FP) which is there in models like Albrecht"s FP i.e.Function Point Analysis [6].
Many researchers analyze the feasibility of evolving software effort estimation methods exhausting various methodology, parameters, datasets, etc.In the comparative analysis study given by [7], amalgamation of estimation methods may generate more reliable, accurate cost estimation for software development as it is displayed that no method is good or bad in all the situations.Review papers given by [8] [9] grasp a complete description of such studies.In review paper [10], the effort estimation was assessed by back propagation Artificial Neural Networks on datasets such as Desharnais and ASMA, generally through system size to determine the correlation of size with effort.www.ijacsa.thesai.orgThe method produced reassuring estimates represent that the model need an additional methodical advance method to develop the architecture and parametric settings in order to achieve improved outcomes.In paper [11], the effort estimation evaluation of the relationship between effort and size analyzed using Genetic Programming technique in which advanced tree composition displaying number of power, linear, quadratic type common equations.The methodology lengthened to appropriate desirable stages of prediction correctness through hardly the size attribute but also approved to achieve additional enhancements.

Kumar et al. proposed a model exhausting Particle Swarm
Optimization (PSO) for tuning the elements of primary COCOMO model to compute the effort accurately considering hardly KLOC factor [12].
Finnie et al. [13] conferred a comparison of statistical regression based model with other artificial Intelligence based estimation models for evaluation of software development effort.The researcher establish that statistical regression model underperformed for difficult and complex software projects as the Artificial Intelligence based models gives satisfying evaluation results.They studied dataset amidst Projects from 17 organization and Desharnais.As an estimation criterion MMRE was used.In 2002 another researcher Heiat [14] examined Feed Forward Neural Networks with function point and Radial Basis Neural Network with Source Lines of Codes for various datasets including projects of different generation languages.The results embodied that artificial neural network method is prudent with regression though a third generation language data set is used.P.Rijwani et-al [15] used hit & try method to determine best network architecture, in an experiment for the training network using back propagation.For software effort estimation a FLANN was proposed by Trimula Rao [16] which generates effort and hence processes the final layer output.It has a drawback that the relation between input and output is not equitable.

Investigation on back propagation Artificial Neural
Network of 2-2-1 design based on dataset of NASA that includes eighteen projects.The inputs were development methodology and KDLOC and output was effort.
Attarzadeh [17] in which 17 cost drivers and 5 Scale factors were used as inputs.Sigmoid activation function is utilized while creating the network to achieve post architecture of COCOMOII model.The COCOMO algorithm is compared using Pred (0.25) and the results are shown in terms of MMRE.An innovative software development for effort estimation was proposed by Attarzadeh [24], exhausting neural networks, in which weights of the network were adjusted in such that it resulted in COCOMO II model.The neural network method suggested gives better result when related to COCOMO model after proper training.Even though back-propagation for neural networks is focused, complexity arises for adjusting weight and bias net parameters during training like slow convergence, sensitive to arbitrary initial weights, entering local minima, difficulty in selecting explicit optimum network configuration.The preeminent efficiency of genetically trained neural networks is mentioned in various researches.For instance After a series of experiments and simulations Shukla [22] concluded that the genetically trained neural networks outperforms back propagation trained and quick propagation trained neural networks in software effort estimation.The recommended model evaluates software development effort in the function of seventeen cost drivers and five scale factors.Thus it is evident that over the years improving the accuracy of software effort estimation has remained main concern of research with little attention being paid to quantifying the uncertainty involved in effort estimation techniques.

B. Loopholes in Existing Research
Many systematic surveys have been conducted on software effort estimation.Moløkken and Jorgensen [30] provide an exhaustive review of surveys in software effort estimation.They concluded that most of the projects (60-80 percent) suffer from effort and/or schedule overruns.However the percentage overrun (30-40%) is significantly lower than suggested by Chaos report by Standish Group (80%).Jorgensen and Sheppard [29] identified and reviewed 304 research papers published in 76 journals.They found that majority of the research in software effort estimation has remained concentrated on effort estimation methods and less research is done on uncertainty assessments, data set properties and measures of estimation performance.The authors also  [26,40].The reason is it cannot be ascertained if formal methods are better or weaker than expert judgment [27].
Identification the human factors affecting effort estimation [28] and development of practical guidelines are crucial to get benefitted from expert judgment.Applications of software effort estimation is summarized in Fig 3.
Fig 4 shows the number of papers published from the year 1996 to 2016 related to software effort estimation J.Moses [4] suggested that Effort estimation using algorithmic models, statistical prediction systems or machine learning approaches, e.g.Case Based Reasoning (CBR), Artificial Neural Networks (ANN) or Rule Induction (RL), under perform when compared to subjective estimation given by human estimators.The author also added that the prediction methods does not provides any support for decision making in case if the actual effort is greater than or less than the predicted effort.The author developed an approach by using Bayesian Inference to improve effort estimation consistency.B. Clark and etal [20] developed a multiple regression based effort calibration strategy.D.Yang and etal [21] proffered that software industry suffers from frequent cost overruns, and the software cost estimation remains a challenging issue.The authors developed a model for accounting the uncertainty based on Bayesian belief networks.The block diagram of the adopted methodology is shown in Fig4.Genetic Algorithm is used for improving the output of neural network.Fuzzy inference and genetically trained neural networks are employed independently to evaluate level of confidence and optimized effort.The main form or GUI constructed in MATLAB is shown in Fig 5 . .The GUI prompts user to input COCOMO II variables.The user can click on the optimized effort button on the calculate panel for the GANN [3] predicted effort in person months.Similarly level of confidence button is pressed to obtain the probability that the effort will not exceed the specified limits.

A. Dataset Generation
For the present work COCOMO 81 dataset available on PROMISE repository [18] was converted to COCOMO II dataset using the tool Rosetta stone [33].The tool is developed at IBM research in order to make COCOMO estimates functional with COCOMO II model.The output is development effort measured in man-months.The above mentioned COCOMO 81 dataset was established from exploration of sixty three developed software projects.

B. Network Topology
The model is created with one hidden layer in MLF() neural network .One hidden layer with arbitrary units is sufficient for "Universal Approximation Property" [23].There is no rule of thumb for determining number of hidden units to be used, as it depends upon critical factors such as number of training cases and complexity of classification and learning.A convenient way is to try many different networks, calculate the generalization error for each network and select the network with minimum generalization error [23].Following the above rule, we performed a number of hit and trial experiments from 2 to 20 nodes.

2) Calculate probability of selection of a particular individual by dividing its chromosome's fitness by the total fitness values of the population. 3) Divide the roulette wheel into sectors based on probabilities calculated in the second step. 4) Spin the wheel 'n' number of times. The individual corresponding to the sector pointed by the pointer is selected.
The probability that an individual is selected from a population of n individuals is given by equation, where is fitness value of element.

∑
(3) (4)  Fuzzy logic is close to human interpretation of truth.This property can be utilized in effort estimation practices to balance the inherent imprecision with uncertainty to determine the level of confidence.The level of confidence indicates the uncertainty or the probability of overestimation or underestimation.The uncertainty can not only be used to improve the estimation consistency but can also be used in making statements to client indicating the chances of overestimation or underestimation.
Literature reveals that there are numerous approaches for incorporating fuzzy logic in effort estimation models.However available research on organization specific effort dependencies is scarce.For instance a company "A" is facing frequent underestimation on its past projects.It is possible that the value of COCOMO variables used by them make it inclined to Underestimation.Suppose the assigned value of Programmer capability is high but due to some discrepancies the actual programmer capability is lower than expected.Such conditions can occur frequently while calculating software effort by formal methods because of following reasons: 1) A huge amount of information is required in the starting phase.
2) Error due to human factor.
The developed fuzzy inference system calculates probability of overestimation.This acts as a warning system for effort estimators and prompts them to review the process and/or set appropriate risk factors (Table2).

Dataset preparation:
The historic dataset is converted into probability distribution functions of 23 input variables.In the proposed model conditional probability is used as basis for forming fuzzy rules.By probability theory conditional probability is defined as probability of occurrence of an event (A) by assertion that another event (B) has already occurred.The event A is hypothesis and the event B is observed evidence.It is expressed mathematically by equation Here A denotes occurrences of overestimation and B denotes instances of values of input variables.For instance P (O/STOR=1) denotes the probability of overestimation provided that the selected value of STOR is nominal.The model is based on chances of overestimation as underestimation and overestimation are mutually exclusive events.Thus, if probability of one is known the probability of other can be calculated easily.

(
) ( ) www.ijacsa.thesai.org The conditional probability for every value of each input variable is calculated and subsequently mapped into linguistic fuzzy rules.The developed fuzzy model uses 76 fuzzy rules to calculate output.

Input and Output Variables:
In this next step the range of 23 input variables as effort multipliers scale factors and lines of code is defined.The output variable is probability of overestimation and it lies between 0 and 1.

Membership Functions:
The functions are defined for each input as well as output variables.In our analysis we have considered Gaussian membership functions as it demonstrated by Kushwaha and Suryakane [31] that Gaussian membership function smoother transition in its intervals, and the achieved results were closer to the actual effort.The Gaussian membership function is governed by following equation.The Gaussian membership function for KLOC is given in Table3 [32] and STOR is shown in Fig7.         5 presents the computed effort values using COCOMO and proposed model.Subsequently, Table 6 presents the comparison of MRE values when effort is computed using COCOMO and with the hybrid model.

V. CONCLUSION AND FUTURE SCOPE
The learning exposes that the suggested fuzzy logic based COCOMO II model incapacitates the uncertainty and vagueness in the inputs that is present in the conventional COCOMO and hence increases the accurateness of software effort estimation.By determining additional appropriate fuzzy rule sets and by arraying technologies like type-2 fuzzy improbability can be handled further closely and hence more precise software effort estimation is thinkable.

Fig. 1 .
Fig. 1.Percentage of successful, challenged and impaired projects in large, medium and small organizations
www.ijacsa.thesai.orgThe 23 neurons in input layer correspond to the 23 input variables of COCOMO II model (kloc, 5 Scale Factors, 17 Effort Multipliers).The node in output layer represents the optimized effort.The input to neural network as scale factors and effort multipliers is be represented by binary vectors ( [ ]).The COCOMO II variables are converted into binary vectors by range normalization such that such that ( [ ]) The vector O represents output obtained in personmonths.And are weight parameters or synaptic strength connecting hidden layer to output layer and input layer respectively.The Network Genetic algorithm is employed for training the neural network.Fig 6 represents the algorithm of training neural network by genetic algorithm.The preeminent efficiency of genetically trained neural is mentioned in various researches.For instance After a series of experiments and simulations Shukla [22] concluded that the genetically trained neural networks outperforms back propagation trained and quick propagation trained neural networks in software effort estimation The suitable values for control parameters of genetic algorithm have been found by running various simulations and have been listed in Table 1.We have used binary string chromosomes.Six features (very low-vl, low-l, nominal-n, high-h,very high-vh, extra high-xh) are considered for each cost driver and subsequent weights are encoded with 3 bits(0-n,1-vl,2-l,3-n,4-h,5-vh,6-xh,7-n).0and 7 are assumed default values.Fitness function is reciprocal of MMRE as genetic algorithm maximizes the fitness function and a low Roulette Wheel selection: Following are the steps for Roulette Wheel selection: 1) Evaluate the sum of the fitness value of all individuals in given population...

Fig. 7 .Fig. 8 .
Fig. 7. Membership functions for STOR Fig 8shows the subsequent fuzzy rules.IV.RESULTS AND DISCUSSIONS In our project historical dataset of 63 projects is considered.The estimated effort and Mean Relative Error using COCOMO model, neural network model with back propagation and the genetically trained neural network model is shown in Table4 and Table5.Fig 10 demonstrates the error histogram obtained after training process.The histogram is centered on zero error Thus our selected topology is appropriate.Fig 12 shows the comparison between the two models.Mean square error found is 0.0124682 after 50 generations using 10000 populations.Fig 9 substantiates that the genetically trained neural network model outperforms the COCOMO model as well as BPNN by significant difference.The data of BPNN is taken from the authors" past research [19].

Fig. 12 .
Fig. 12. MRE comparison Fig 11 demonstrates the actual and predicted deviations from the actual effort.Where actual deviation denotes the actual percentage of overestimation/underestimation and predicted deviation denotes the predicted overestimation/underestimation.The Figure confirms that the developed model is accurate and provides optimistic view of uncertainty by effectively covering all range of deviation.Table5presents the computed effort values using COCOMO and proposed model.Subsequently, Table6presents the comparison of MRE values when effort is computed using COCOMO and with the hybrid model.
of the research relies on historical dataset for evaluation and validation of effort estimation models, only a few provide real life evaluations.The different estimation methods used in industries are shown in Fig 2.Presently Expert judgment is the prominent estimation method used by organizations www.ijacsa.thesai.orgproposed that most