Towards Agile Implementation of Test Maturity Model Integration (tmmi) Level 2 Using Scrum Practices

—The software industry has invested the substantial effort to improve the quality of its products like ISO, CMMI and TMMI. Although applying of TMMI maturity criteria has a positive impact on product quality, test engineering productivity, and cycle-time effort. But it requires heavy software development processes and large investments in terms of cost and time that medium/small companies do not deal with it. At the same time, agile methods deal with changing requirements and continuous delivery of valuable software by short time iterations. So that the aim of this paper is to improve the testing process by applying a detailed mapping between TMMI and Scrum practices and verifying this mapping with a study providing empirical evidence of the obtained results. The research has been evaluated on a sample of 2 large-sized TMMI certified companies. In conclusion, the experimental results show the effectiveness of using this integrated approach compared to other existing approaches.


INTRODUCTION
As the software industry is evolving rapidly but it still far away from zero defects software, so the software industry focused on developing the development process [1].
-Despite the fact that testing often accounts for at least 30-40% of the total project costs‖ [1], most of the widely used software industry standards like Capability Maturity Model Integration (CMMI) only give limited attention to the testing process improvement, so that the testing community created a TMMI [1].
-The TMMI is a detailed model for test process improvement and is positioned as being complementary to the CMMI.‖[1].TMMI aims to help software organization to improve the software quality by preventing defects and save time and cost [2].more and more software organizations moving from the waterfall model to more agile models like Scrum, as the benefits of agility including faster delivery to customer, flexibility with requirement changes and higher software quality [3].
The paper is organized as follows: Section 2 presents the background and related research of agile software development and TMMI.Section 3 describes the research problem.Section 4 and 5 describe the research goals, approach, and assumptions.Section 6 shows the theoretical mapping of the two approaches.Section 7 shows overall results of the theoretical mapping.Section 8 describes the background of the analyzed study.Section 9 summarizes the assessment results from the study.Section 10 the results of this study are discussed.The last section concludes the paper with the key findings, research limitations and future work related to the TMMI and agile integration.

II. OVERVIEW
A. TMMI Capability Maturity Model Integration (CMMI) includes few testing process areas like verification and validation with lack of details so Test Maturity Model Integration (TMMI) has been created to cover most of testing processes [4].
TMMI includes best practices of testing that helps in process improvement and assessment [5].
There are five levels in the TMMI, each level contains a number of process areas, to move to the higher level it should complete all process areas in the lower level [6].
-At TMMI level 2, testing becomes a managed process and is clearly separated from debugging.The main objective of testing in a TMMI level 2 organization is to verify that the product satisfies the specified requirements‖ [1].

B. Scrum
Scrum is not a defined process or standard it is a framework that enables the organizations to set their own customized processes and techniques for their software development projects [7].
-The main aim of using Scrum is to enable a project team to react quickly, simply and appropriately instead of wasting time and energy creating, implementing and updating outdated plans‖ [8].www.ijacsa.thesai.org-The Scrum team consists of a Product Owner, Scrum Master, and a Development Team.Scrum teams are selforganizing and cross-functional, they are not directed by others outside the team and have all the competencies to accomplish their work‖ [9].
There are 3 roles in Scrum process: Scrum Master who audits and monitors the scrum process, Product owner who responsible for product requirements and the team who develop the products [10].
Scrum begins with creating the product backlog that contains customer requirements.Then the team prioritizes the requirements.The iteration is a Sprint that consists of a period of one month's duration.Every sprint starts with sprint planning meeting that clarify the details of tasks and they discuss the tasks and the progress in daily scrum meeting and at the end of each sprint, sprint review meeting is held to evaluate the sprint and check the current status against the sprint goals determined in sprint planning meeting [11].

C. Related work
There have been a number of publications focusing on the relationship between CMMI project management process areas and Scrum practices, showing the gaps and strengths between them [12,13] but unfortunately their studies did not include an empirical evidences and Others analyze the feasibility of the CMMI process improvement in combination with the agile software development and project management area [14] however this study provide only a theoretical survey at the project management level .Neil Potter and Mary Sakry [15] listed only the main practices of CMMI that map cleanly to Scrum process steps.Few publications intend to increase the understanding of the relationship between Scrum and CMMI reporting empirical results that confirm theoretical comparisons between Scrum practices and CMMI level 2 [16].Unlike these researches, As known software does not work correctly can lead to many problems, including loss of money, time or business reputation, and could even cause injury or death so this study is concentrated on testing process and presents the details of the mapping between Scrum practices and TMMI, and illustrates this mapping with a study providing empirical evidences of the obtained results which will increase the understanding of the mapping in theoretical and practical study.This research covers Level 2 process areas in TMMI.

III. PROBLEM DEFINITION
Organizations seek to improve testing process by applying new methodologies and models.When talking about TMMI, it requires a considerable amount of time and effort for implementation, it may require additional resources and knowledge required in smaller organizations and also requires a major shift in organizational culture and attitude so the number of TMMI certified companies is limited to 23 only all over the world.Trying to merge TMMI with another methodology will be a good choice to avoid those disadvantages.So it is needed to know if using TMMI will be compatible with the Agility concept or not.And if the two approaches are compatible the next step is to identify how the integration will be analyzed between them and how the organizations will apply this mapping perfectly.

IV. RESEARCH APPROACH
The goal of this research was to study how TMMI model could be used in assessing Scrum practices and to help TMMI certified companies to shift toward agility.The approach was developed in Three-phase model.The first phase was to discover an appropriate possible theoretical mapping between TMMI and Scrum practices.The second phase was to verify the applicability of this mapping.An empirical study has been conducted, the mapping was applied to a sample of TMMI certified companies to make sure that merging TMMI, and Scrum processes within those organizations is possible.The Third phase has been conducted in order to measure the effect of this merge.

V. RESEARCH ASSUMPTIONS
 Using TMMI and Scrum practices together will be useful for the organizations to achieve better team productivity.
 TMMI needs a heavy documentation in contradiction with agile software development which focuses on small projects so organizations can reduce this problem and improve their test process areas by applying both of them together.

VI. STUDY PHASE 1: DESIGNING A MAPPING MATRIX
BETWEEN TMMI AND SCRUM PRACTICES After a deep study of TMMI Practices, and scrum recommended practices.A Mapping matrix has been produced to show how scrum practices could be mapped to the TMMI practices.
The tables below show several TMMI practices (using TMMI text taken from the model definition) and how Scrum can implement each practice.To appraise Level 2, it is assumed that the Scrum implementation is robust and shows evidence of the TMMI practice being performed.
For each process area, a mapping between its specific practices and the Scrum practices was carried out.Several considerations were identified in order to establish this mapping, identifying gaps and strengths.After that, a coverage rating for each practice was established considering the following criteria in table I.The data is filled based on conducting interviews, questionnaires and practical work with experts in testing field working in Vodafone Group, ITWORX, HP Enterprise Services and MCS and the results are combined in -Rating‖ column as Covered, Uncovered or Partially Covered.

PA 2.2 Test Planning
-The purpose of Test Planning (TP) is to define a test approach based on the identified risks and the defined test strategy, and to establish and maintain well-founded plans for performing and managing the testing activities [1]‖.
The mapping of TP process area and its related scrum practices is shown in table III.-CM is not specifically called out in Scrum.However, in an agile environment it is pretty easy to add a layer of CM to protect your work.Even for groups that like to use white boards, you can be creative and at least establish some basic protection by labeling items (e.g.-V1.1,‖ or -Story dated 1/2/YY‖) and taking a photo [15]‖.
The changes in requirements are managed implicitly in product backlog and sprint backlog [17].This practice is classified partially covered.

PA 2.7 Process and Product Quality Assurance
-The purpose of Process and Product Quality Assurance (PPQA) is to provide staff and management with objective insight into processes and associated work products [18]‖.The mapping of PPQA process area and its related scrum practices is shown in table VII.

VII. OVERALL RESULTS OF THE MAPPING
General results of the mapping are shown in the Fig. 1, a consolidated view of the coverage of TMMI Level 2 Process areas by Scrum practices.
This result shows that most of specific practices of TMMI Level 2 Process areas are Covered and 90% of Configuration management process area is Uncovered.This means that TMMI Level 2 process areas are mostly satisfied for the organizations that need to improve their test processes by applying TMMI within Scrum.STUDY PHASE 2: EMPIRICAL STUDY Once theoretical comparison between TMMI and Scrum were established, a sample of the certified companies was selected to apply the approach and find the strengths and weakness in using the provided study.

A. Study description
The number of TMMI certified companies from level 2 to level 5 is 23 all over the world like Vodafone UK, Tech Mahindra Global Test Factory, Wipro Limited, Virgin Media, etc.A sample of 2 large companies was taken which is 8.7% from the certified companies.Working on 4 projects; the average of testing team members in each project was 6 members.The process used in all the projects was Agile/Scrum.This study was limited to TMMI Level 2 and all the involved teams were less than 10 members.

IX. STUDY PHASE 3: VERIFYING APPLICABILITY OF RESEARCH WORK
The objective of the study was to discuss the effects of testing projects before and after using Agile/Scrum practices inside the TMMI process and to show an empirical evidence of understanding the mapping between TMMI and Scrum practices.Scrum practices widely used as they enhance the testing process in specific KPIs that mentioned in this study.The study was applied on a telecom projects which using Scrum process and the projects' data is extracted from HP ALM test management tool (represented in table VIII).In this project the KPIs are calculated based on the following equations and (P) stands for defect priority and the result is shown in table IX   The following figure (Fig. 2) displays the percentages of test cases effectiveness before and after using Scrum and it illustrates that the study helped in enhancing this KPI by [100*(84-51)] /51= 64.71%.The following figure (Fig. 3) displays the percentages of defect quality before and after using Scrum and it illustrates that the study helped in enhancing this KPI by [100*(89-69)] /69= 28.99%.The following figure (Fig. 4) displays the percentages of defect density before and after using Scrum and it illustrates that the study helped in enhancing this KPI by [100*(33-25)] /25= 32%.The following figure (Fig. 5) displays the percentages of defect removal efficiency before and after using Scrum and it illustrates that the study helped in enhancing this KPI by [100*(87-72)] /72= 20.83%.The following questionnaire is distributed among testing experts to display their opinions about how was the communication between involved stakeholders in the projects using legacy and others using Scrum practices.The result of this survey is shown in table x and the summary is represented in Fig. 6.Fig. 6.A summary of testing KPIs before / after using Scrum

X. DISCUSSION
From the previous study the results show that using the mapping helped in enhancing and improving test cases effectiveness to reach 89% in some iteration and also defects quality was increased to reach 93% in some iteration.The research helped the companies to make improvements in the quality delivered to the customers as project requirements were discussed and refined during sprint planning and daily scrum meetings.It helped in increasing the customer and employee satisfaction.Also applying the research helped the companies in the following achievements:  Business assurance -consistent delivery adherence and test effectiveness exceeding customer targets and improved business assurance.


Step change in cost -significant reduction in test costs given to the client.
 Tester error is reduced from iteration to iteration based on root cause analysis of defects.
 Small iterations helped in identifying issues early and made it easier to respond to change.

XI. CONCLUSION AND FUTURE WORK
The goal of this paper was to improve testing process by comparing Scrum practices with TMMI Level 2 process areas, showing the major strengths and gaps between them and identifying how organization are adopting complementary practices in their projects to make these two approaches more compliant.
The study results showed that most of the specific practices of TMMI Level 2 Process areas are covered and 90% of configuration management process area is uncovered and also the organizations that their process based on TMMI model can be improved by adding some Scrum practices.
In the light of that, it is suggested as future works: The field of software engineering is changing very quickly so more studies will help the companies to find a detailed reference containing the best practices regarding TMMI maturity levels and another point to map the TMMI with other agile methods for example XP, FDD, and DSDM.

Fig. 2 .
Fig. 2. A summary of test Case effectiveness after using Scrum

Fig. 3 .
Fig. 3.A summary of Defect Quality after using Scrum

Fig. 4 .
Fig. 4. A summary of Defect Density after using Scrum

TABLE I .
[1] purpose of the Test Policy and Strategy (TPS) is to develop and establish a test policy, and an organizationwide or program-wide test strategy in which the test levels are unambiguously defined.To measure test performance, test performance indicators are introduced[1]‖.The mapping of TPS process area and its related scrum practices is shown in table II.
CRITERIA FOR SCRUM PRACTICES RATING Rating Criteria U Uncovered The practice is not covered by Scrum.P Partially Covered The practice is partially covered by Scrum.C Covered The practice is fully covered.www.ijacsa.thesai.orgPA 2.1 Test Policy and Strategy -

TABLE II .
MAPPING BETWEEN TPS SPECIFIC PRACTICES AND SCRUM PRACTICES C SP 3.2 Deploy test performance indicators  Discuss the results with involved stakeholders in daily Scrum meeting.C

TABLE III
After the sprint planning meeting the Scrum master of the team calls the team to define the details of how the committed items are going to be implemented.

3 Test Monitoring and Control -
[1] purpose of Test Monitoring and Control (TMC) is to provide an understanding of test progress and product quality so that appropriate corrective actions can be taken when test progress deviates significantly from plan or product quality deviates significantly from expectations[1]‖.The mapping of TMC process area and its related scrum practices is shown in table IV.

TABLE IV .
MAPPING BETWEEN TMC SPECIFIC PRACTICES AND SCRUM PRACTICES

TABLE VI .
MAPPING BETWEEN TE SPECIFIC PRACTICES AND SCRUM PRACTICES C www.ijacsa.thesai.org

TABLE VII .
MAPPING BETWEEN PPQA SPECIFIC PRACTICES AND SCRUM PRACTICES  Scrum master periodically checks that the scrum process is being followed.Evaluatethe scrum process in retrospective meeting.C SP 1.2 Objectively Evaluate Work Products and Services  Scrum master periodically checks that the scrum process is being followed. Evaluate the scrum process in retrospective meeting.C SP 2.1 Communicate and Ensure Resolution of Noncompliance Issues  This is Scrum master responsibility.C SP 2.2 Establish Records  This is Scrum master responsibility.C

:
Test Case Effectiveness = Number of valid defects mapped to test cases / Total number of valid defects.

TABLE VIII .
CALCULATED TEST EXECUTION FACTORS BEFORE AND AFTER USING SCRUM

TABLE IX .
CALCULATED KPIS BEFORE AND AFTER USING SCRUM