Assessment and Comparison of Fuzzy Based Test Suite Prioritization Method for GUI Based Software

The testing of event driven software has significant role to improve overall quality of software. Due to event driven nature of GUI based software many test cases are generated and it is difficult to identify test cases whose fault revealing capability is high. To identify those test cases test suite prioritization is done. Various test suite prioritization methods exists for GUI based software in literature. Prioritization methods improve the rate of fault detection. In our previous work we have proposed a fuzzy model for test suite prioritization of GUI based software. In this method priority is assigned on the basis of multiple factors using fuzzy model. These factors are: The type of event, Event Interaction, and Parameter-value interaction coverage-based criteria. Using this method a test oracle will be organized in the descending order of its effectiveness. In this paper we are evaluating the proposed fuzzy model and comparing results with other prioritization methods. Keywords—Test suite prioritization; Fuzzy Model; Comparison of prioritization methods


INTRODUCTION
Testing is a quality assurance activity [10].Testing of GUI impose many challenges due to its event driven nature and interaction with the user [9].Event driven software takes sequence of events as input after change of state generate new sequence of input as output [1,8].Many test cases are generated to cover combination of all events.To improve the effectiveness these test cases generated for event driven software a prioritization method are developed.The aim of this prioritization is to reorder the test cases according to fault revealing capability of test cases.This paper is organized as follows: Section II describes the research background for the proposed work.Section III describes Fuzzy model.Section IV discuss about evaluation of fuzzy based test case prioritization method.Section V covers assessment of effectiveness of test suite.Conclusion is presented in section VII.

II. RESEARCH BACKGROUND
The problem of test suite prioritization is widely explored by many researchers.Few researchers have significant contribution in this area.One imperative work is proposed by Memon Atif M, Bryce Renee C. (2007).In this work test cases are prioritized using event interaction coverage.Rate of fault detection improves for the test suits which have the larger number of 2 way and 3 way interaction.Huang Chin-Yu et al. (2010) in their work proposed three weight based method.These methods are Equal weight, Fault prone weight and Random weight.They have classified events and assign weight value based on their importance.Results shows that high to low Adjusted weight method find more fault.Another work by Bryce Renee C. et al. (2011) proposed cost-based combinatorial interaction coverage metric as 2way interaction coverage and cost-based 2way interaction coverage.According to experimental results the difference in APFDC between 2way and cost-based 2way for CPM was less than 3%.Wei Sun et al. proposed a multi-objective test case prioritization strategy is proposed to combine two objectives: statement coverage and event coverage for GUI applications.Experiment shows that multi-objective has better performance than both singleobjective strategies.

III. FUZZY MODEL
This model was proposed in our previous work [5].In this model multiple factors are considered for prioritization.Major components of GUI are identified as prioritization factors.These factors are: The type of event, Event Interaction, and Parameter-value interaction coverage-based criteria.A fuzzy model is developed for test suite prioritization which considers these factors as input and assigns weight value to each test case.These factors are categorized as very low, low, medium, high and very high.Impact of these factors are categorize in five categories as very low, low, medium, high and very high.According to this impact priority is assigned.Random tiebreaking is used when weight value comes same for multiple test cases.Total 125 rules are generated for fuzzy model.Centre of gravity (COG) is used for defuzzification.www.ijacsa.thesai.org

A. Evaluation of fuzzy based test case prioritization method
For the evaluation of fuzzy based test case prioritization method application TerpPaint 3.0 and some of its L1 and L2 test cases are considered.For each test case its parameter count, event sum and number of event Interaction covered are counted for the input of fuzzy model.These values are shown in table 1.Once priority is assigned to test cases evaluation of prioritized sequence is done.In order to evaluate prioritized sequence faults identified by each test case in previous run is considered.
With the count of number of faults identified by the test case and its priority assigned by the fuzzy model its rate of fault detection has been calculated as shown in table 3.

B. Assessment of effectiveness of test suite:
Average Percentage of Fault Detection (APFD) metric is considered for the assessment of effectiveness of test suite.This metric calculates rate of fault detection.Higher value of this metric represents effective prioritized sequence.APFD is defined as (Rothermel et al., 2001): ( Where TF 1 : number of faults identified by first test case n: total number of test cases under consideration m: total number of faults identified by test cases Final APFD value for TerpPaint 3.0 is 0.49.These results are further compared with APFD of Parameter count method and event weight method and results are shown in table 5.For the comparison of results prioritization is done for the same test suite using various methods as shown in table 5 and their APFD value is calculated for the comparison.

TABLE I
A parser is developed to calculate the Parameter Count Event Sum and Event Intraction.These values are given as input to the fuzzy model and output value is generated.Output of fuzzy model is shown in table II as FIS output.On the basis of output of fuzzy model priority is assigned to each test case.Maximum priority is assigned to those test cases whose FIS output is highest and lowest priority is assigned to those test cases whose FIS output is lowest.

TABLE III .
PRIORITY TABLE FOR TEST CASES OFTERPPAINT 3.0

TABLE IV .
COMPARISON WITH OTHER METHODS

TABLE V .
COMPARISON OF VARIOUS PRIORITIZATION METHODS technique which was proposed in previous research to assign priority of test cases is evaluated in this paper.It is further compared with existing prioritization methods.Experimental results shows that fuzzy based Method www.ijacsa.thesai.orgprioritization method outperform most of the prioritization methods.APFD of Fuzzy based test case prioritization method is very close to best order of test cases.Experimental results shows that the proposed fuzzy model is proved to be an effective approach for test case prioritization for GUI based software.