Issues Elicitation and Analysis of CMMI Based Process Improvement in Developing Countries Theory and Practice A Case of Pakistan

Researchers have tried to find out the pattern of rising and fall of Pakistani software industry and also the reasons for what is going exactly wrong with this industry. Different studies have witnessed that in Pakistan, the software industry is not following the international standards. Another surprising fact, being observed in the past analysis, is the companies which have initiated CMMI-based SPI program have not even achieved the higher levels of CMMI from past three years, which is an alarming sign of the declining attitude of the industry. Therefore, it has become mandatory to look for the weak points or critical barriers or issues which are actually, the reason for this slow progress of CMMI-based SPI in Pakistan software industry. This study has identified that the issues for CMMI based SPI in Pakistan are much different than what it reported in the literature. Giving proper attention to the root of the problem can help solve many of the problems in this regard. Keywords—CMMI-based SPI; process improvement; inefficiency; Pakistani software industry


INTRODUCTION
Pakistan software industry is relatively new in the region, and its roots are not very deep.Therefore, the level of maturity is not yet attained.Lack of effort at government level and lack of presence of central controlling authorities or organization has caused slow growth for this industry.In 1995, it was felt by the Government of Pakistan that to guide and nourish the development of the software industry, ministries, departments and government agencies in the country, an organization needs to be created as no such existing department was providing the said services.
The formation of PSEB is also to make sure that it assists the software market of Pakistan regarding process improvement with the help of coping with the CMMI levels, human resource development, international marketing, making an effective strategy and promoting research providing innovation and enhancing technologies.
PSEB has successfully completed two different projects on CMMI so far.The first project, "Standardization of Pakistani IT industry (CMM Pakistan 2003)" was aimed at bringing the top five Pakistani IT companies to CMMI Level 3 or above, was completed with a cost of Rs. 39.9M.The second project, "Standardization of Pakistani IT industry on CMMI" was aimed at bringing at least 18 companies to CMMI Level 2 was completed with a cost of Rs. 39.3M.The objective of running these two projects was to initiate CMMI activities in Pakistan, and create a base for CMMI implementation at the mass level.
On close observation, we come to know that not many companies in Pakistan have initiated the CMMI-based SPI program which is obvious from the fact that out of more than 500 software companies, only 23 have initiated CMMI-based SPI program and out of these only one has achieved level-5.And this is why this research has become even more of an importance which will reveal these de-motivating factors in the industry.Avoidance and mitigation of these issues will result in a quick and smooth implementation of CMMI-based SPI in Pakistan.Since some demotivates are well known in literature for causing friction in CMMI based process improvement in developing countries this study aims to investigate if there is a gap between theory and practice when it comes to resisting the de-motivation factors.The real question is: are we fighting off the right issues for SMMI based SPI? II.
LITERATURE REVIEW Software industry in Pakistan has a lot of potential to grow and compete with the other software industries in the region.We have brilliant resources and creative people who are driving this industry.This industry can play a vital role in the economic growth of Pakistan.Quality of work produced by the Pakistani professionals is highly recognized by the European, American and African markets [1].We have a lot of work that is flooded towards Pakistani market."From its www.ijacsa.thesai.orgnascent beginnings in the late 1980s, the industry has successfully arrived to a point where its value proposition has been validated over and over again.The largest members are grossing 15-25 million dollars in revenues, and receiving 100 million dollar valuations.Most tech companies are growing in excess of 30% a year annually.The industry as a whole is doing over 2 billion dollars a year in revenue, up from less than a billion dollars a few years ago" [1].But over the years it has been noticed that the productivity of Pakistani software industry is not stable.There could be multiple factors that are causing such inconsistency.The most important factor that is hearting our industry very badly is poor process adoption."Remember that Pakistan is a country, which has only recently recognized the importance of ISO certification; although ISO has been around for much longer than CMM" [2]."One of the most prominent human aspects is that software practitioners are de-motivated to deal with SPI initiative in their organizations [3].Despite having such huge potential and manpower, we are unable to produce even a single" organization like WIPRO, TATA and GOOGLE of the world."The famous Indian firm named TATA Consultancy has thirteen centers in India -it has 12 CMM Level-5 and 1 CMM Level-4 certifications to its credit; WIPRO has three Level-5 certifications to its credit" [2].
Lack of adoption of software process improvement programs-like CMMI, is causing frequent closures or losses of software companies."Recent times have seen many Pakistani companies go bellies up -although I am not implying that lack of CMM initiative is the reason for their debacle-but this definitely was a contributing factor" [2].Processes are not adopted in their true spirits.Management is not aware of software improvement processes.In General, management of these companies wants ROI without spending money on the stability and continuation of the adopted processes."There are two challenges that a software development firm faces.First, to come up with reliable, efficient and pragmatic Official processes.Second, to make these processes a part of the company's culture i.e. to make the Official process the same as the Actual Process" [2] The adoption of software process improvement program has proved its value.Countries, where software industry have adopted such kind of standards, are in the fore front.Their software industry is contributing huge revenue segment."The Actual Process is what you do, with all its omission, mistakes, and oversights.The Official Process is what books say you are supposed to do" [4].
The MPS.BR project in Brazil was initiated in December 2003.This program was to propose the software process improvement model for the small to medium software companies according to Brazilian software industry needs."The MPS Model is a software process improvement and assessment model, mainly oriented to the small and mediumsize enterprises (SMEs).This model aims at: i) to fulfill the "business need" of these firms; ii) to be recognized, locally and internationally, as an applicable model to organizations which develop or acquire software" [5].
Through survey in different times, issues of software process improvement adoption, in general, and CMMI adoption, in particular, have also been identified.Through multiple reports published by PSEB, P@SHA, and other agencies that out of 500 software companies only 33 are CMMI certified for the different levels of CMMI.Most of the lot are at CMMI Level-3 or below.Reports are evident that most haunting factors out of the lot, which have been proved as issues are Time, Cost and ROI of CMMI.
Keeping in view the above discussion this study aims to find the difference between theory and practice when it comes to issues of CMMI-based SPI in Pakistan.Hypothesizing that this difference can be the vital factor that is causing the main hindrance in growth and process improvement of companies adopting CMMI and not reaping its fruits.

III. RESEARCH METHOD DETAILS
Considering the limitations of this research and the differences in the survey methods, Personal Method in Interviews and Mail Method in Questionnaire have been selected for this research.

Questionnaire Design
The questionnaire designed for final or comprehensive survey has three sections out of which: Section 1: Includes Company"s information Section 2: Includes Respondent"s information Section 3: Includes issues of CMMI-based SPI.

Data Collection Techniques and Methodologies
Questionnaires were floated via email and on personal contacts in different CMMI initiating organizations.It was made sure that the questionnaire reaches the maximum number of organizations with a pass-on strategy which suggests that while receiving the questionnaire the SPI practitioners were asked to forward it to other CMMI initiating companies if they have any personal contact with that particular organization.A total of 35 software companies were visited.A total of 33 companies were chosen to provide the research project with a cross-section of company maturity levels, company types, and sizes.

Sample
Whole population (all 33 companies SPI involved) employees were the sample size for this study.Since sample size was limited, a regular follow-up with the respondents was set up via emails, telephone calls and meeting them in personal.Some incentives were also introduced to get the maximum number of responses such as providing them with free discount coupons and scratch cards.

Identification of SPI Practitioners
In this survey, it is ensured that not just SPI Practitioners but software practitioners also responded to the questionnaire.For this purpose, some companies were visited personally, and respondents were contacted via phone and emails.It was ensured that this survey includes the whole sample of SPI practitioners and related persons.
Identification of SPI practitioners is achieved using the following criteria: www.ijacsa.thesai.org 1) SPI practitioner is currently working/has already worked in a CMMI initiating organization.
2) SPI practitioner is currently working/has already worked in an organization where SPI is achieved using the model, which is similar to CMMI e.g., ISO standards etc.
3) SPI practitioner"s willingness of being available for the interview.

Compilation of Issues
Once the feedback was received from the respondents, it was compiled into a spread sheet using SPSS and Excel categories-wise.When the data was successfully compiled, a list of issues was extracted based on agreed responses of the respondents.

Interviews to resolve open issues
Interviews had been conducted to resolve some of the open issues which couldn"t be addressed in the questionnaire.A total of four unstructured interviews with four SPI professionals regarding the extracted issues had been conducted.The opinions are included in conclusion.

Identification of Renowned SPI Practitioners
Selection of renowned SPI practitioners had been done on the following criteria:

1) At least five years of SPI related experience OR 2) Worked in CMMI implemented organization for more than two years OR 3) Taken or conducted CMMI training in past five years OR 4) Achieved SEI/CMMI or related certifications. Compilation of Data (Interviews & Survey Results)
A total of 33 software companies were visited.Participating companies were selected from a larger sample of companies who responded to the final questionnaire giving information about the problems they are facing regarding SPI intuitive in their organizations.The companies were chosen to provide the research project with a cross-section of company maturity levels, company types, and sizes.Since the questionnaires used for this research had both open and closed ended questions, therefore, the analysis of this research can be categorized in to qualitative and quantitative.

Type of Analysis used
There are two major types of analysis in survey research namely Quantitative Analysis and Qualitative Analysis.In this research, both of these analysis techniques were used out of which the quantitative analysis was focused more.

Quantitative Analysis
Quantitative research is best to be opted in a scenario which requires the numeric figure to be estimated from the questionnaire or in which numbers answer the questions rather than the answers to the questions.Usually, this type of research require a large amount of data which cannot be estimated or analyzed by qualitative research or a very time-consuming activity, the quantitative research can draw a meaningful result from this."The main beneficial aspect is that it provides the means to separate out the large number of confounding factors that often obscure the main qualitative findings.Quantitative analytical approaches also allow the reporting of summary results in numerical terms to be given with a specified degree of confidence."

A. Issues Frequency Distribution, Trend Agreement-Disagreement, Severity Measurement
In this section, independent variables are analyzed with the frequency of responses, the trend of agreement and disagreement levels in de-motivator categories corresponding to different independent variables elaborated, and severity levels of the de-motivator categories are also listed.

1) Response Rate vs. CMMI-Levels
According to the feedback from all the respondents, it has been observed that nearly 60% of respondents are associated with the software companies having CMMI-Level 2 which therefore can be considered as the largest population among all the CMMI implemented Organizations.
Statistics show that it is difficult or time taking activity for the organizations to climb to higher levels of CMMI.So it can be concluded that there is a mixed trend of agreements and disagreements for different De-motivator categories in comparison with CMMI-Levels.Likert Scale i.e., "Agree-Neutral-Disagree" has been translated to "High-Medium-Low" according to our understanding for measuring the severity of De-motivator categories.
As per above mentioned criteria, following De-motivator categories have been listed according to their severity levels measured in the table 2.

2) Response Rate vs. Company Size
The frequency of responses according to company size is varied across different sizes.As shown in the graph below, the only type of enterprise that has minimal responses is microenterprise.Therefore, it is evident from the fact that the sample of this survey mainly contains the Small, Medium and Large Enterprises.Above mentioned linear graph depicts that the agreement level of respondents increases as we move from micro to small enterprises, which then tends to drop equally in the medium enterprises and disagreement level increases steeply equal to four times the previous level.In the large enterprises, there is an equal trend of both agreement and disagreement for the Issues.The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to company sizes.

3) Response Rate vs. Company Age
Statistics shows that companies older than seven or more years have a vast majority reaching almost 80% to all responses.The difference can also be seen in the graph below: Rate of differences in agreements and disagreements can be observed in the linear graph mentioned above.As we can see that the agreement level is higher than the disagreement level in the organizations not older than four years which than decreases in the organizations older than 4 years but less than seven years, and the disagreement level increases gradually.And for the organizations older than seven years have a significant increase in agreement than disagreement.
The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to Company Age.The linear graph mentioned above states that the level of agreement is more or less constant in all type of companies except Project Based in which it reaches to forty percent agreement.On the other end, the disagreement level gets high in Project Based and Hybrid Type Companies.
The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to Company Type.

5) Response Rate vs. Company"s Type of Business
Our statistics have confirmed that Companies involved in Offshore Development are higher population rate than the rest of the companies equaling almost fifty percent of total population.The above-mentioned linear graph displays that there is a constant increase in agreement level among both Offshore and In-house Company types.There is a sudden decrease in the agreement level and steep increase in the disagreement level in the companies having both types of businesses.
The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to Company Types

6) Response Rate vs. Respondent"s Experience
Statistics have confirmed that majority of the respondents population in this survey is of the software practitioners having experience more than four years and less than six years equaling almost fifty percent of total population.Practitioners having experience more than seven years are on the second number in the list having a population of almost thirty-three percent.The above-mentioned graph explains that practitioners with one to three years of experience have almost higher disagreement level than the agreement level.The agreement level then increases among the respondents having experience of more than three years while disagreement level remains constant.

Agree
The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to Respondents" Experience.

7) Response Rate vs. Respondent"s Job Function
Statistics have confirmed that there are almost an equal number of respondents with different designations, whereas Software Engineers/Developers/Junior Executives are in the majority equaling almost forty-two percent.Manager/Team Lead/Senior Executive comes second with proportion equals to almost thirty-six percent.The above-mentioned bar graph shows that middle management is mostly agreed to the issues whereas the top management is mostly disagreed with them.However, in case of lower management, the agreement level and disagreement level among issues are equally divided.The analysis confirmed that following De-Motivator categories are at the High, Medium, and Low severity levels according to Respondents" Job Functions.

12) SPI Practitioners perspective on Mitigating Issues
In order to know the perspective of SPI Practitioners on mitigating issues, three-step approach is adopted as follow:

a) Identification of renowned SPI practitioners b) Conducting interviews of renowned SPI practitioners c) Compiling and listing interview results 13) Selecting Renowned SPI Practitioners in Pakistan Software Industry
To achieve this, a thorough inspection was performed on Pakistan Software Industry.As a result, total of five renowned SPI practitioners are approached for the interviews according to the criteria already mentioned above.These practitioners are working or have already worked in the organizations i.e, Pakistan Telecommunication Company Limited (PTCL), Siemens, National Database and Registration Authority (NADRA), Teradata and NCR.

14) Conducting interviews
Interviews of the SPI practitioners were conducted with the help of an interview questionnaire attached at Annexure D. The questionnaire is divided in two sections: 1.It contains all the issues which were exposed in the literature and which were confirmed in the initial and comprehensive survey questionnaires.2. Open ended questions which will reveal the proposed strategy to mitigate these issues.

V. CONCLUSION & FUTURE WORK
This survey is conducted on thirty three software companies of Pakistan who have implemented CMMI.There are a total of 48 software practitioners and 4 SPI professional who have participated in this research.The survey has extended the empirical research and case studies conducted on SPI Issues in other parts of the world.It has revealed more Issues which were missing in past articles.The analysis has revealed that Issues such as Time & Budget Constraints and Cultural Issues are on the top priority among all respondents.Whereas, there have been differences in responses among respondents with different job functions as the top management is mostly disagreed to all Issues.Surprisingly, most of the respondents with Middle and Lower Management agree with the existence of these Issues in the IT Industry and feel the need to mitigate them in order to successfully implement SPI initiative in their organizations.
A Model to tackle with these Issues has also been proposed after analyzing the results of the data gathered from the survey.The model has been tested regarding its reliability using SPSS"s AMOS showing Cronbach"s Alpha value above 0.7 which is required to keep the reliability intact among the elements in the Model.This Model can assist software practitioners to implement CMMI-based SPI not only in the Pakistan software industry but also in other under-developed countries.
The future work of this research is an implementation of the Model proposed, and the extension of this work i.e., Issues of SPI focusing CMMI Model only in other parts of the world.The differences between this research and the future research will in fact serve as the improvements in the Model and will lead it to perfection.

Fig. 6 .Fig. 7 .
Fig. 6.Bar Chart: Response & Company Age ijacsa.thesai.org4) Response Rate vs. company Type Statistics have confirmed that Companies with Hybrid (Project & Product based) type are in the majority in the sample equals 42 % of total population.It is also evident that Product Based Companies are not very far behind Hybrid Type equaling almost 40 % of total population.So it can be said that the sample size of this survey contains mainly Product-Based and Hybrid Type Companies.


Lack of Training Aggregate Statistics On Responses

TABLE VI .
DE-MOTIVATOR CATEGORIES WITH SEVERITY LEVELS ACCORDING TO COMPANY TYPE

TABLE VII .
DE-MOTIVATOR CATEGORIES WITH SEVERITY LEVELS ACCORDING TO RESPONDENTS" JOB FUNCTIONS

TABLE VIII .
MODEL "A" RELIABILITY TEST 9) Issues Mitigation Model "B"Model with Company Size as "Independent Variable" Fig. 17.Issues Mitigation Model-B www.ijacsa.thesai.org

TABLE X .
[9]]NDENCIES AMONG DEPENDENT AND INDEPENDENT VARIABLES IN LITERATURE AND COMPARED WITH SURVEY It is quite difficult for any SME to choose an improvement approach and to apply it in their organization without the help of external consultants or substantial investment in the time of their software manage[10].Documentation is also high on the list of developer problems.Project Managers are also concerned with documentation and state that CMM involves ""too much paperwork.It is not as automated as it should be""[9].Differences in practitioner group problems.Senior managers have below average concern for project issues such as documentation and tools and technology issues, as they concentrate on problems relating to people and communication.They have above average concern for requirements issues in terms of problem ranking (equal 2nd), but an average concern in terms of percentage of problems.Indeed, further examination of Table2reveals that developers devote a higher percentage of overall problems to requirements than senior managers do with 11% and 10% respectively[9].
[11]greed4.Kuvaja et al.[44]mentioned that the main problem of the small companies is that they cannot afford to maintain substantial expertise of software process improvement within their companies, but they have to buy it from external sources.product,withprocess often believed to be a "brake" on product development.The managers believed process to have a significant cost which, in their respective companies, they attempted to keep to a minimum.What the managers perceived as the Cost of Process centred on a number of factors and these are represented as a network diagram in Figure 1[11].Disagreed 11) Common Critical Issues in Literature and this Survey: www.ijacsa.thesai.org