Empirical Evaluation of Modified Agile Models

Empirical evaluation is one of the widely accepted validation method in the domain of software engineering which investigates the proposed technique via practical experience and reflects its benefits and limitations. Due to various advantages, agile models have been taking over the conventional software development methodologies since last two decades. However besides the benefits, various limitations have been noticed as well by the researchers and software industry in agile family. To achieve the maximum benefits it is vital to fix the limitations by customizing the development structure of agile models. This paper deals with the empirical analysis of modified agile models called Simplified Extreme Programing (SXP) and Simplified Feature Driven Development (SFDD), which are the modified forms of Extreme Programing (XP) and Feature Driven Development (FDD). SXP was presented to eliminate the issues of conventional XP such as, lack of documentation, poor architectural structure and less focus on design. SFDD was proposed to take care of reported issues in FDD such as explicit dependency on experienced staff, little or no guidance for requirement gathering, rigid nature to accommodate requirement changes and heavy development structure. This study evaluates SXP and SFDD through implementing client oriented projects and discusses the results with empirical


INTRODUCTION
Conventional software process models are replaced by lightweight agile development methodologies.The reason behind the widely acceptance of agile family by the software industry is the features these models provide such as: light weight approach for development, early delivery of partially working software (module), welcome changes at any stage of development and quick response.Agile models shifted the focus from process to people and valued those factors which were neglected by traditional models [7], [25], [26].Some of the famous agile models are: Extreme Programming (XP), Scrum, Test Driven Development (TDD), Dynamic System Development Model (DSDM), Crystal methods and Feature Driven Development (FDD), etc. [7], [8].These models follow the values, principles and practices given by agile manifesto which is considered a parent document of all agile models and contains twelve foundation principles of software development.XP and FDD, both are the widely used agile models in software industry [12], [40].XP was developed by Kent Beck and mainly focuses to overcome the limitations of traditional software process models.The working of XP consists of certain principles, values and practices, which work together rigorously to develop high quality software [9], [29], [34], [35], [39].XP provides a flexible and adaptive development approach which can handle the changing business needs in an effective way due to its well-known requirements gathering technique, "story cards".Its 12 practices provide the guidelines to govern the whole development process in an effective and efficient way.Besides the advantages, XP reflects some limitations as well.Drawbacks of XP include poor architecture, weak system design and lack of documentation [29], [32], [36], [37].Moreover its practices: "pair programming" and "on-site customer" are controversial and cannot be applicable in every situation [38], [39].Due to these drawbacks, XP is suitable only for small scale and low risk projects.On the other hand FDD follows the process oriented approach [9]- [11].It is highly adaptive and mainly focuses on design and building aspects of development.As its name reflects, features are the basic building blocks of this model.Feature is considered as a functionality which user wants in the software.Benefits provided by FDD model includes the iterative and incremental approach along with ETVX pattern which ensures the development of high quality software according to client valued features.However along with advantages, some limitations of FDD were also reported such as: little or no guidance for requirement gathering, explicit dependency on experience staff, rigid nature to handle changing requirements and heavy development structure including various activities and team roles.All these issues make it only suitable for medium or large scale projects.SXP [40] and SFDD [12] were proposed to overcome the limitations of XP and FDD respectively.This study empirically valuates the proposed models through empirical case studies conducted in software industry.

II. RELATED WORK
Drawbacks of agile models have to be eliminated in order to achieve the maximum benefits, for this purpose many researchers have proposed the modifications in agile models.XP and FDD were discussed and optimized in many studies from which some of are discussed here.In [13], researchers presented the Tailored Extreme Programming (TXP) model which was specifically designed for small scale projects where requirements have fewer or no tendencies to change.In [14], researchers proposed the feature of reusability in XP model.They introduced a framework to add the ability of component based architecture refinement reusability in traditional XP.The used framework provided a way to develop simple and loosely coupled design which can be modified easily in future.Researchers in [15] customized the XP by introducing parallel refinement iteration to the development activities in order to enhance the quality; however the proposed model is not www.ijacsa.thesai.orgsuitable for software projects having a lot of inter dependencies among modules.In [16], authors customized the software maintenance model by using many XP practices such as: on-site customer, planning game, small releases, pair programming, metaphor, test driven development and refactoring.In [18], researchers integrated Personal Software Process (PSP) with XP.The proposed model introduced "Personal Planning Phase" in which developer can plan the activities by using PSP practices.Six important practices from each model (XP and PSP) are integrated in proposed model.In [19], XP was customized to develop medium scale projects with large team by eliminating its drawbacks such as weak design and lack of documentation.Moreover a phase named "Analysis and Risk Management" was introduced to handle the failure risks.In [30], Analytical Hierarchy Process (AHP) was used with CRC cards during designing phase of XP.AHP was used to design a systematic approach of CRC cards prioritization.AHP is a hierarchal model consists of five steps which reflect the human thinking process.By using AHP the developers can select, design and implement the most important classes first.In [31], XP was customized for medium to large scale projects.The research highlighted the drawbacks of classical XP such as weak design, poor architecture, lack of risk management and lack of documentation.These issues of XP make it suitable only for small scale projects.To eliminate these issues, new phases were introduced in modified proposed model.Author in [1] proposed Feature Driven Reuse Development (FDRD), an enhanced version of FDD which considered re-useable feature-sets for development along with the new requirements.Author in [2] presented Competitor Driven Development (CDD), a hybrid process model which integrated the practices of Extreme Programming (XP) and Feature Driven Requirement Reuse Development (FDRD).The proposed model is a self-realizing requirement generation model which keeps track of market trends as well as competitor"s next product launch to extract requirements.Moreover CDD considers the market orientation of product to guess the product"s success rate.In [3], authors proposed a hybrid model SCR-FDD, an integration of Scrum and FDD.The proposed model covered the imitations of both models by taking the schedule related aspects from Scrum and quality related aspects from FDD.In [4], researchers presented Feature-Driven Methodology Development (FDMD), a modified version of FDD which integrated the features of object oriented approach with Situational Method Engineering (SME).In the proposed model requirements are represented as features, which are based on object oriented principles.The feature is defined by using action, result and object.Authors in [5] proposed Secure Feature Driven Development (SFDD), an enhanced version of FDD which introduced some changes in classical FDD to cover security related issues.The proposed model introduced two phases in classical FDD named "Build security by feature" and "Test security by feature" along with the "In-phase Security" element in each phase.Moreover, a new role is also added called security master to ensure the secure software development.Authors in [6] proposed an ontology based approach in FDD for semantic web application.The proposed model used the concepts of domain ontology from domain knowledge modeling.Ambiguity and inconsistency regarding Language is handled by RDF and OWL however the agility of FDD can be compromised by adding the concepts of domain ontology in each phase.

III. MODIFIED AGILE MODELS
The proposed Simplified Extreme Programming (SXP) is focused to overcome the limitations of classical XP.It provides more flexible and simple approach for small to medium scale projects.The issues of pair programming and on-site customer are handled in an effective way.On the other hand, SFDD [12] was proposed to overcome the limitations of FDD such as explicit dependency on experienced staff, little or no guidance for requirement gathering, rigid nature to accommodate changes in requirements, heavy development structure.SFDD focused on small to medium scale projects along with an effective requirement elicitation technique of story cards which simplified the requirement change process.Both the proposed models are briefly explained below.1.In the proposed model, customer involvement is restricted to initialization and release phase only and all other phases are executed by development team with the complete coordination.Necessary documentation is produced during each phase that helps to resolve change management issues."Initialization" is the first phase of SXP and is responsible to extract and manage the requirement as well as to create an overall plan for project.Requirements are extracted and managed through story cards, a story card consists of following features: functionality name, type, priority and the short description without any technical detail.Type defines whether the functionality is functional and nonfunctional and priority is assigned with number so that higher priority features can be developed in early iterations.Project planning includes the decisions regarding project scope, cost and tools to be used for the development."Analysis" is the second phase and deals with budget and schedule related activities which are performed by development team only.In this phase required budget is estimated and documented.An iteration plan is also formed which includes the detail about number of iterations, number of stories implemented in each iteration and the time of each iteration.A training session is also conducted to make the development team familiar with the tools and technology (if the team members are not already familiar)."Design Phase" is third phase of SXP which deals with two activities: "Designing UML Diagrams" and "Test Planning".Conventional XP does not include any documentation which makes requirement change management very difficult.This issue is effectively solved by SXP by focusing on system design with use case diagrams and sequence diagrams.Test cases are also developed in this phase.Writing tests prior to code help the development team to understand different design opportunities."Development and Testing" is the fourth phase and works in an iteratively.Activities of this phase include coding, functional testing, integration and integration testing.Developer writes the code for selected stories by keeping in view the design document which was developed during design phase.Functional testing is performed by using test cases, developed during test planning activity.Coding activity is repeated if any issue is reported in functional testing.These tests are performed by programmers and results are noted to keep the track of defects.Code is integrated with previous developed module in case of successful functional testing followed by another testing known as integration testing."Release" is the last phase in which customer performed acceptance testing.The developed workable product is released after the customer"s approval along with the User manual.If the customer is not satisfied with the developed product then whole development process can be repeated again with changed or modified set of requirements.

B. SFDD
Simplified Feature Driven Development (SFDD) consists of six phases and various activities as shown in Fig. 2. "Develop an Overall Model" is the first phase which deals with the identification of requirements and scope of project.Domain expert and chief programmer are the main roles of this phase.Domain expert provides the project requirements through story cards and chief programmer finalizes the project scope by keeping in view the provided requirements moreover use case diagrams and class diagrams are also developed in this phase."Build Feature List" is the second phase of SFDD and deals with the extractions of features from the documents developed in first phase.Features are basically the functions which a customer wants in the software.Related features are collected in a list called feature list.Chief programmer converts the requirements in to feature lists in this phase."Plan by Feature" is the third phase which deals with the project planning activities and starts with a meeting where domain expert and chief programmer finalize the budget and time frame of the project.Chief programmer further finalizes the number of iterations and assigns features to iterations by keeping in view the priorities.This phase also includes the estimation of effort (resource persons) and hardware/software resources which are needed for the project.At the end of the phase classes are assigned to class owners (developers)."Design by Feature" is the fourth phase and deals with the process of refining the class diagrams developed in the first phase.Object model is finalized in this phase and class owner completes the pseudo code for the assigned classes.To ensure the quality, a role of QA manager is introduced in this phase."Build by Feature" is the fifth phase of model and first phase of iteration.Development actually starts in this phase according to the pseudo code, written in previous phase.QA manager makes sure that the developing module is according to the features.Test by feature is the last phase of model and second phase of iteration which deals with the testing activities and starts with unit testing to make sure that the developed module is bug free and working properly, if passed then integrated with already developed module.
Integration testing is then performed to check the integrated working of modules.Finally domain expert performs the acceptance testing.Proposed model simplified the structure of FDD through effective customization.

IV. EMPIRICAL EVALUATION
This research aims to perform the empirical evaluation of proposed modified agile models.For this purpose two case studies are conducted in which both models, SXP and SFDD were used to develop small scale web based projects.The selected case studies were part of an empirical research project in which multiple agile models were used to develop various client oriented applications in a software house, situated in Islamabad, capital of Pakistan.The software house consists of experienced staff with dominating knowledge of software development along with higher degrees in computer science disciplines.The developers were using agile methods for most of the projects.Both case studies were implemented in same working environment but with different teams.Most of the characteristics of applications are same such as size of project, no of iterations, no of team members, and the tools used in development.The detail regarding the characteristics of developed projects is given in Table I.The case study of SXP is implemented by the team which had significance experience of agile development.On the other hand, to implement the SFDD, the chosen team had less or no experience of agile development however training session of 10 days was organized.
For SFDD, less experienced team was selected as the authors of proposed model (SFDD) claimed that the issue in classical FDD regarding the dependency on experienced staff has been eliminated.The detailed empirical results collected during the development are shown in Table II.Partial and aggregated results of selected case studies are discussed in [39], [33].However this paper demonstrates the complete results of empirical experiment including all the iterations by keeping in view the guidelines extracted from [17], [27], [28], [19].Both case studies are implemented with four iterations.After each iteration, partial working software (module) was released for the client.The second column of Table II represents the attributes/metrics which are measured in each release for both the models and the last column contains the cumulative/average values of metrics from all four releases.The remaining columns (release 1 to release 4) present the values of metrics (column 2) in each release for SXP and SFDD.Metrics are used to measure the software in terms of development, cost, working, productivity, quality, effectiveness and efficiency from various aspects [20]- [24].

V. CRITICAL ANALYSIS
From the detailed empirical results (Table II), significant differences can be seen among the performances of both the models.Even though the working environment as well as the size and nature of both the applications were same, but SFDD performed much better than SXP.KLOC of the application developed using SXP are 3.069 with the implementation of 21 user stories however on the other hand SFDD implemented 65 user stories with 13.1 KLOC (Fig. 3 and 4).No. of post release defects is an important software metric which reflects the quality of developed application as well as the satisfaction of customer.After the release 15 defects were reported in the application developed using SXP however only 5 defects were reported in the application developed using SFDD (Fig. 5).Time to implement pre-release change requests is also considered as one of the important quality metric which reflects the change management feature of software process model.10 pre-release changes were proposed during SXP case study which took 11 hours to implement however no. of prerelease change requests in SFDD case study were 12 which took the same time for the implementation (Fig. 6) as in SXP (11 hours).
Software productivity reflects the team effort during the application development.Productivity of the application developed by SXP was far lower than the application of SFDD (Fig. 7).During SXP case study, 3060 lines of code were written in 695 hours (Actual Work Effort) with the productivity of 4.4 however during the implementation of SFDD, 13110 lines were written in 592 hours and reflected productivity of 22.14.As compared to SFDD, SXP showed very poor performance by keeping in view the empirical results.SFDD performed very well according to all software parameters (Table II) even with the team having less experience with agile methodologies.There might be various reasons of poor performance in SXP case study.Complexity level of the developed application in SXP case study may be higher than the application of SFDD however according to best of our knowledge the nature and complexity level of both the application were same.As the performance of SXP is lower in every release so, there might be issues in code integrations as there were total of 37 integrations in SXP and only 18 in SFDD.Moreover the issue of communication among the team members can also be a reason of lower performance.The issue of awareness with agile development cannot be considered as the team of SXP was experienced with agile and team of SFDD had less or no experience with agile development.

VI. CONCLUSION AND FUTURE WORK
This paper evaluated the proposed modified agile models, SXP and SFDD through empirical case studies.SXP focused to reduce the reported issues of conventional XP such as: Lack of documentation, poor architectural structure and less focus on design.Due to these issues, XP is only suitable for small scale and low risk projects.SFDD has taken care of the issues reported in FDD, such as explicit dependency on experienced staff, no guidance for requirement gathering, rigid nature to accommodate requirement changes and heavy development structure.Empirical analysis was performed via development www.ijacsa.thesai.org of client oriented projects by using SXP and SFDD.Both projects were related to Human Resource Management (HRM) and also were same in nature as well as in size and complexity level.The development team for SXP case study was experienced in agile development however for SFDD case study the chosen team had less experience of agile as the proposed SFDD eliminated the dependency on experienced staff.According to empirical results, SFDD performed much better than SXP even with the less experienced team.In comparison of SFDD, SXP performance was very poor in each metric such as lines of code, implemented user stories, post release defects, productivity and time required to implement pre-release change requests.There might be various reasons of poor performance of SXP model such as complexity level, integration issues and communication problems within the development team.It is suggested that both the models should be further tested with large and complex projects.