An Early Phase Software Project Risk Assessment Support Method for Emergent Software Organizations

Risk identification and assessment are amongst critical activities in software project management. However, identifying and assessing risks and uncertainties is a challenging process especially for emergent software organizations that lack resources. The research aims to introduce a method and a prototype tool to assist software development practitioners and teams with risk assessment processes. We have identified and put forward software project related risks from the literature. Then by conducting a survey to software practitioners of small organizations, we collected probability and impact of each risk factor opinions of 86 practitioners based on past projects. We developed a risk assessment method and a prototype tool initially based on data that accumulates further data as the tool. Along with a risk prioritisation and risk matrix, the method utilises fuzzy logic to provide the practitioners with predicted scores for potential failure types and aggregated risk score for the project. In order to validate the usability of the method and the tool, we have conducted a case study for the project risk assessment in a small software organization. The introduced method is partially successful at prediction of risks and estimating the probability of predefined failure modes. Keywords—Software Risk Identification; Software Risk Assessment; Failure Mode Prediction; Fuzzy Decision Support


INTRODUCTION
According to reports [1], the global software market is estimated to have a value of US$333 billion in 2016 which is expected to grow by 7.2%.On the other hand, the success rate of global (mainly US and Europe) software projects in 2015 is only 29% [2].Therefore, it is highly desired to follow software engineering practices to prevent further loss in software spending.Among software development and engineering activities, risks assessment of software projects is a significant task, requiring effort and time.In many organizations, especially in small organizations, project managers do not have enough expertise and time for risk assessment.However, the consequences of ignoring this activity will result in loss of time and resources for the organization, as without risk assessment incorrect decisions can be made.
Although there are slight variations in definition of terms in the literature, a risk factor is a potential problem that may occur.Similarly in the software domain, risk is considered as an uncertain event or condition with negative or positive consequences on a software project on one or more project objectives such as scope, schedule, cost, and quality PMBOK [3].It should be identified, assessed by its probability of occurrence and impact as its two important dimensions, and a contingency plan should be developed for remediating the problem when it actually occurs [4].
In accordance with above definition, various studies have been conducted and risk factors, categories [5], [6] and analysis tools [7] have been introduced.However, most developed methods and tools either cover a limited set of risk factor that potentially occur later in software project lifecycle or only focus on the improvement of a method/technique within the risk assessment process, such as aggregation, root cause analysis, etc. [8]- [10].Most of the methods assume that the organization/team already has accurate near precise information about the project in the initial planning phase.Experience in risk identification, existence of a potential risk register and historical data is widely assumed.There are other studies focusing on a specific set of risk factors (Appendix 1), such as operational risk, requirements risk, etc.Furthermore, risk factors used in different studies may be disjoint or sometimes overlapping.In real world, software practitioners cannot benefit from these methods unless in a consolidate framework is provided.As for the available software tools, they are mostly enterprise, expensive and the rest only have limited predefined set of risks or no predefined risks at all.Furthermore, they do not provide any baseline and prediction on which the practitioner can use initially, benchmark his project, and predict potential failure types.Hence, there is a need to provide a consolidated method for the software practitioners of small organizations with scarce experience and resources.This will help them not to miss potential project risks, especially during early phases of the project.Related work section provides a more thorough review of the problems stated.
Therefore, one of our goals in this study is to put out a risk assessment method especially for practitioners of emergent software organizations with relatively low previous experience and historical data.To do this, from the literature, a wide coverage list of software project related risks was identified, which possibly rated at initial phase of the project with relatively little information.By conducting a survey to software practitioners, risk data have been collected; both in terms of impact and probability based on software practitioners' previous project experiences.In addition to risk www.ijacsa.thesai.orgprioritisation, the method assist the practitioners with an initial set of possible risks, probability and impact values to be revised for their specific project settings.Furthermore, based on the data provided by the risk assessor, the tool predicts probability of failure types, such as defects, overtime and over budget to the risk assessor.In order to validate the usability of the method and the tool, a case study was conducted for a project risk assessment in a small software organization.The conduct of this research is shown in Figure 1.
In the rest of the paper, first previous studies related to software project risk assessment is discussed.Then, the risk factors collection, the assessment method and the tool developed is described.Finally, the findings of the case study conducted for validations of the method and applicability of the tool is presented.In an early study published by Software Engineering Institute [11], a risk management model and a taxonomy based software risk identification has been performed.The method consists of a taxonomy based questionnaire and a process for its application.The taxonomy provided a structure for organizing software development risks i.e.Product Engineering, Development Environment, and Program Constraints.Carr et al. demonstrated the application of risk management model in five repeating activities of identify, analyse, plan, track, control, and communicate at the centre of activities.It has been observed that adopting Pareto 20-80 rule is important and dealing with very first 20% of risks will be most effective highlighting the importance of risk prioritisation in a risk assessment method hence also considered in our method.This generally accepted approach is adopted in widely used text books in software project management [4].
Identification of software risks has been tackled in Li Xiaosong's study [5], providing a wide range of risk factors and categories in addition to general risk matrix and risk levels with their definitions.Another similar study by [12] contains a set of proposed risks in development phases with their definitions.Finally, Verner [6] has performed a literature review of available studies.The study has extracted 77 risk mitigation advises alongside with 85 risks.Using this and some other works, we aggregated sometimes disjoint or overlapping risk factors.
Due to inherent problem of difficulty in assigning numeric values to risk factors, as an example Markowski's study [9] proposed to fuzzify risks ratings.Risk ratings and values are fuzzified and fuzzy inference system (FIS) is adopted for processing and prediction.The problem of aggregation (linear aggregation has been found misleading) of risk scores has also been tackled.Choquet integral based aggregation approach to software development risk assessment [10] is an example of second category of related work.The study provides a software risk aggregation method to estimate the risk of a project.In addition to aggregation method, a set of risk factors, categories, and their associations have been developed.The study proposes a multi Choquet integral based multi attribute aggregation method for decision making process.For the same aggregation problem, [13] defined a method based on fuzzy logic.The goal of this study is early assessment of operational risks in software development.According to the study, before and during developing software, there are not enough data to have a full-scale risk assessment.So a fuzzy method is implemented to address the issue of uncertainty.In addition, a causal model is developed using fuzzy rules.Among researches proposing software risk assessment methods, [14] proposes a method to statistically analyse and evaluate risk factors and their prices.The method enables to approximate risk-pricing parameters for four risk factors, namely, (1) Application Task, (2) Personnel Capability, (3) Process Maturity, and (4) Technology Platform.Hence, this study focuses on pricing dimension and in this respect granularity; hence the number of risk factors is small.In order to have a better software risk control, Hu's research [15] suggests planning based on causality.The proposed method is based on Bayesian Networks with Causality Constraints hence taking a probabilistic approach rather than fuzzy logic.In this study, in order to gather necessary data, a survey has been conducted which is similar to our approach.The data is used for constructing the Bayesian Networks.However, the paper mostly focuses on finding relations between variables rather than assessment.Bayesian Networks is used in numerous works due to their simplicity and ease of implementation [16].But a solid amount of data is required for constructing a proper network.
Reference [17] proposed a risk assessment technique for evaluating risk levels in software projects through analogies with economic concepts.This study defines project risk levels as the probability of project's failure in achieving goals and evaluates risk levels using a risk identification questionnaire.Structurally this study is similar to ours in terms of comparing risks and effects, but the definition of risks and methods to handle them and comparing them is different.Costa et al. uses www.ijacsa.thesai.orgweighted normalised medians for risk factor, in contrast, we used FIS to compare risks and weights gathered in initial questionnaire.Finally, Costa et al. addresses the issue of financial loss and gain prediction, whereas our method uses project failure modes as predictions.
The relationships between project setting, governance and project success are studied recently in [18].A survey has been conducted in attempt to prove the positive relationship between project management methodology (PMM) and project success.The relationship between the project management methodology and project success is moderated by project governance.The first hypothesis is shown to be valid showing the importance of the general project setting related risks for project success can be considered as valid hypotheses.
Recently, a similar study [19] implemented fuzzy method in aggregation of software risk factors.However, the application of the risk assessment is extraneous and differs in the method of data-point accumulations.In contrast to our study, this study relies only on 7 experts' data of the field and considers only a handful of risk factors.As a result, the study does not provide predictions according of expert data.
Further, tools potentially that can be used for software project risk assessment are publicly available and accessible.As an example, RisCal [7] is a proposed tool by Haisjackl.RisCal implements risk identification, risk analysis and risk prioritisation.In risk identification step, it allows for a user defined risk models in addition to the pre-defined risk models.There are also studies and tools with different approaches like esrcTool [20].esrcTool implements FPA (Function Point Analysis) to estimate software cost and risks.The study focuses on functional breakdown of software rather than considering overall project environment and attributes.Hence, the method and tool focuses on more software product risks rather than project environment.
In the literature, each study adopts a set of risk factors for the study in question, sometimes completely disjoint or overlapping.Initially, most of the reviewed studies only consider a limited set of risks (not in terms of number, but also in terms of coverage over different aspects of software engineering processes), so a wider coverage of software project related risks is put forward which can potentially be assessed at initial phase of a project.Secondly, aggregation-focused studies are generally difficult to implement, as they require technical expertise to apply.Thirdly, available software tools mostly enterprise solutions only to manage a limited predefined set of risks or no predefined risks at all.Therefore, a risk assessment method and a prototype tool were developed so that they can be used by practitioners of small organizations with relatively low previous experience and historical data at an early phase of the project.The tool suggests the practitioners with a set of possible risks and their risk values for a specific project setting provides risk prioritisation with risk matrix, project risk level using fuzzy aggregation and potential failure type score using fuzzy inference.In this respect, our approach consolidates and supports the early risk assessment task at initial phase of a software project.

A. Risk Factors, Scales and Data
In various prior studies, risk factors considered focus on a specific aspect or phase of software development.We aimed at a risk factor set that can be used as a project initiation phase.So as the first phase, risk factors and categories were extracted from related studies [5], [12], [14], [21]- [25].Therefore, a superset of 128 risks with a greater coverage was created.Then, similar and overlapping risks are unified.Furthermore, risk statements were changed into negative statements to ease practitioners understanding and ratings.
For the scales two components: probability and impact [7] is considered.Risk score is usually defined as the product of probability and impact [26].Hence, scale definitions of probability and impact levels are reused from [21] as shown in Tables 1 and 2.
In addition, as an assessment tool a 3x3-risk matrix is used.Probability and impact are two dimensions of a risk matrix.As one of widespread tool for risk evaluation, risk matrix is natural to understand by evaluators.There are also other configurations of risk matrices like 5x5, 7x5 and 7x4 risk matrices which are not adapted due to less accurate information at early project phase and simplicity of 3x3 matrices [9].Risk matrix dimensions or axes are divided into three level each, which creates a nine cell qualitative matrix [27].This matrix has three parts: (Figure 2). 1) High/Major Concern (red): Risk is high in these sections and an action should be taken.
2) Medium/Concern (yellow): Risk is moderate in these sections and there is a chance that risks in these areas may affect project.
3) Low/No Concern (green): Risk in these sections are low and acceptable and can be ignored.Later, as a data gathering method a questionnaire was designed within the tool for surveying developers accessible online (http://46.197.200.167/public_result.php).It comprises three parts such that; first part obtains general information regarding a previous project considered by the practitioner such as type, size (approx.LOC), methodology used, etc. Second part contains failure and challenges of final project which contains 10 questions (Appendix B) These questions were adapted from previous studies [5], [13], [14], [21]- [25], [28], [29].The last and 3 rd part of the gathers information about the risk factor ratings for 128 risks.Based on this, initially, a risk matrix is generated using 86 practitioners' ratings.The most recent version of this matrix is publicly available at the web address mentioned earlier.
In contrast to a related study conducted previously [30], wider cross-correlations are analysed between risks.According to Weinberg [31], Pearson correlation coefficients of r = ± 0.5 are considered strong and correlation coefficients close to ±1 are the strongest.Evans recommends a correlation coefficient of ±0.6 to ±0.79 is considered a strong correlation [32].As a result, to keep a safety margin, correlation coefficients which are among ±0.6 and ±1 are considered as strong.Table 3 demonstrates the highly cross correlated risk factors.Cross correlation creates a duplicate variable effect, which is not desired in the learning tools.Pearson correlation coefficients are obtained using Matlab software's [33] Pearson's correlation function of "corrcoef()".The Pearson correlation coefficient of two variables is measured as following where μ x and σ x are the mean and standard deviation of X: There are 48 highly correlated risk factor pairs, unique risk factors at left side.Statistically these 48 risks represent repeated data among 128 risk.These 48 risk factors may be eliminated from risk factor list.However, due to lack of enough data points for further analysis, it was decided to keep the 128 risk factors within the tool for now.When the definitions of risk factors were analysed, it was noticed that the most of high correlated risk factors do not have logical bounds -at least as far as we could observe, as correlation does not necessarily result in causation.

B. Description of First Phase of the Method
A multi-purpose method and tool is designed and implemented.The tool gathers information from experts and practitioners and produces a general risk matrix.It also can produce specific risk matrices for projects with varieties of project specifications.It calculates the overall project risk based on fuzzy aggregation and produces probabilities of 10 different failure types for the project based on fuzzy inference.The tool is developed using PHP scripts as a web based software to provide an easy and wide access.Figure 3 outlines functionality of the tool.Data from previous practitioners using the tool is gathered and pre-processed.This data may be referred as expert data later.The pre-processing includes filtering missing and inconsistent data.A general risk matrix is extracted from this data.Then practitioner input is taken for the project under assessment.Both data sets will go through Phase 1 and where initial risk matrix for practitioner is proposed.Then practitioner is allowed to alter proposed risks to get a more accurate risk matrix.Remark that, in case of initial projects risk assessment, it is difficult to measure risk quantitatively.As proposed by Xu [13], when dealing with qualitative variable (like low, mid, high), it is advised to work with fuzzy numbers.The altered and more accurate risk set will pass through Phase 2 for a failure mode analysis of the project.
In order to generate a risk matrix for practitioner, a module is designed to accumulate necessary data for risk matrix.In Phase 1 a query of data-points with parameters of Part I of survey is done.These parameters are "project size (LoC)", "project methodology", "project paradigm" and "development type".The result of is a filtered result of available practitioner data-points in form of a risk matrix and prioritisation.This filtered result come in form of averages of probabilities and impacts of selected data-points for all risks based on the prior parameters.Thus, a 3x3 risk matrix is generated from this data.Proposed risk matrix in this phase is valuable for practitioners and will give them an initial and brief view of possible risks and their importance in similar projects.But, this will not exactly match to the specific project setting to rely on before their data is collected.However, the matrix and initial risk sorting will draw a helpful guideline for practitioner.Practitioner can change risk impact and probability values manually in order to achieve a better rating in next phase.

C. Description of the Second Phase of the Method
In the second phase, as proposed by Xu [13], when dealing with qualitative variable (like low, mid, high), it is advised to adopt fuzzy numbers.Second phase implements fuzzy logic to assess the risks and predict failure modes.A decision matrix is used to evaluate and rank the overall and partial failure score of the project, using practitioner's inputs or predicted risk scores in first phase based on previous data.Practitioner input is acquired in form of probabilities and impacts.Probability and impact scores are turned into triangular fuzzy numbers and aggregated.Then Mamdani's inference model [9], [34] is used for prediction of failure types.To analyse failure modes, data points with negative scores for the failure mode are selected.For instance, in order to perform this selection, only the risks with a particular failure mode score of 3 out of 3 are taken as a match and remaining results are dismissed.In contrast, analysing overall project risk requires all data points.
Due to missing and imprecise information at initial phase of the projects, fuzzy decision matrix is used with triangular fuzzy numbers (TFN) [35].Fuzzy decision matrix has less complexity and is effective for ranking fuzzy numbers.For membership function μ i (x) of fuzzy number, ñ i can be defined as: ñ i > ñ j if and only if e ij = 1 and e ji < Q, where, Q is some fixed positive fraction less than 1.
First part of this equation requires expert data in form of fuzzy sets.To provide this, filtered data-points are sorted into two categories of probability and impact, with each containing risk factor scores.The risk factor scores go through fuzzification process (see Equation (3) and Figure 4) by a membership function for each corresponding risk factor.

{ (3)
A = (a, b, c) After fuzzification of probability and impact, scores in the data-points are aggregated to form a single expert opinion data.To do so, an aggregation operator is adopted from Pandey [36] which is based on arithmetic means of L-Apex and R-Apex Angles of TFN. () Risk value is obtained by calculating the product of probability and impact values; the method of Shang [37] with adjustments is used to calculate the multiplication of triangular fuzzy probability and impact numbers.Remark that there are other works including Taleshian's method, [38] which uses trapezoidal numbers and could be also used with some adjustments.Hence, multiplication of and can be obtained using (7).
For processing and fuzzification of data, membership functions must be defined.The fuzzy numbers must be triangular to match the Equation ( 3), so can be applied to aggregation and multiplication equations.
To prevent misinterpretation of results, probability and impact values are gathered in quantitative range of [0-8] with initial peak points in range of [2][3][4][5][6].Otherwise, aggregation and multiplication equations could lead in to due to producing negative and non-triangular fuzzy number.Remark that normally, the probability is expected to be evaluated in [0-1] range.However, we use the probability score as a variable to be rated by the practitioner and transform it as a fuzzy number, therefore it may range between 0 and 8 during calculations in the method.Impact score are calculated in the same manner.Now multiplication (for measuring total risk) produces fuzzy numbers from 0 to 64 which can contain any triangular fuzzy numbers produced using introduced techniques.Figure 5 demonstrates our predefined fuzzy membership functions, which L stands for low, M for medium and H for high probability or impact: L=(0.5,2,3.5), M=(2.5,4,5.5), H=(4.5,6,7.5)Fig. 5. Fuzzy membership function

D. An Example to Illustrate Second Phase of the Method
To clarify our method, a failure mode prediction example is given.We assume a risk set with corresponding scores of probability and impact described as below.Given the probability and impact scores of L (low) and M (mid) for "Lack of Development Technology Experience of Project Team" risk, fuzzy numbers for these values can be obtained using membership functions defined previously.
After calculating the expert and practitioner values by equation ( 2), minimum values of each risk among test and expert data is obtained as a vector of fuzzy numbers.Later, maximum values of fuzzy risk scores are used to obtain a single aggregated fuzzy number.This number is the total failure score for provided test data.Higher score means the chance of failure is also higher.The same method applies to all failure modes, but it's important to remember that all risks must be considered and the risk of "Lack of Development Technology Experience of Project Team" has been given only to demonstrate how a single risk is being handled in the method.Likewise, this matrix can potentially point out the most influential risk factors.After processing at Equation (2), result is defuzzified.Defuzzification for risk index can be expressed by Equation (11).∑ ∑ (11) Defuzzification of (10) using Equation (11) will result in ( 12): (12) This way, the defuzzified and final risk score for this example is computed, which is 7.709.As discussed earlier, this score is also in range of [0-64] as expected.This number may also be scaled to 12.04% to make the result more natural to interpret by the practitioner.Higher values are representing higher risks scores and lower values are representing lower risk scores.In order to put the applicability of the proposed tool and method, we have conducted a case study.In the case study, the extent of support and usefulness of the tool and provided predictions are meant to be explored.The tool does not include risk responses.Thus, it is not expected to do a complete risk management, but only a prediction and assessment in risks and failure modes.The case must be able to meet the target project specifications as explained in early sections.
As a case, we had to find a case project with small development team with relatively low resources and little experience in risk assessment.Additionally, assessment of a project with Agile methodology was desired, as most of the small organizations prefer agile approaches.Thirdly, the project must be in early steps of development so the practitioners have to guess the risk levels without measuring the actual risks and failures.Otherwise, the result can be biased and misleading.Data collection in this case study is conducted in a first degree, direct (interview) [39] manner.It would be preferable to perform this interview in second degree, but due to limitations explained in next sections, this interview was performed with interactions.
In the case study, we tried to answer the following planned questions.Answering these questions can help us explore the validity and quality of method and the tool.These questions are: 1) Does this tool provide expected risk list for emergent software organizations?
2) Are there any missing important risks?
3) Does the proposed risk model represent real (possible) risk levels according to prior estimations?4) How difficult is it to use the tool? 5) How long does it take to make an initial assessment of risks and failure modes?6) How realistic and accurate failure mode predictions are?
7) Does the tool provide necessary insight for emergent software organizations?

B. Setting of the Case
As our method is opting to assist emergent software organizations with relatively less experience and knowledge in Software Risk Assessment at initial phase of a project, after considering three organizations, a small software company in a University Technology Zone is agreed to participate in the case study.The characteristics of this organization matched with the definition of immature software organization given by Paulk [40], [41].This organization has seven personnel primarily working as a subcontractor for a larger organization developing solutions for a government organization.Hence, the organization has relatively little experience (only 5) on independent software development projects.
However, recently they obtained an independent contract for developing an Emergency Triage [42] Decision Support Software for the University Hospital.
The goal of the project is to develop triage decision support software.This software should be able to categorize patients after the "Triage Nurse" initially evaluates them when they arrive to the emergency department.A patient is categorized into a priority class based on a triage nurse's inputs and based on medical checks.The triage system that the software will implement is an already proven and accepted method, namely, the Canadian Triage and Acuity System (CTAS) [42] 5-level systems, with 5 priority categories.The software is only meant to serve as assistance, it should never take control from the user, as he/she should be able to override the software actions through his/her own professional judgment.At any time, the systems results can be overridden and life critical patients will be intervened outside of the system scope.Furthermore, the system will be delivered as a prototype and will not be fully operational until complete validation and verification; fully operational system will be developed if accepted.
The system is planned and developed by using SCRUM [43] by a team of two developers and a team leader (SCRUM master).Intensive commitment exists from the part of the emergency department management and highly dedicated involvement during development is established by assigning two emergency experts for the development.A Java based framework is planned to be used in order to minimise portability problems.In order to facilitate user interface development, user interfaces are planned to be developed with Jigloo GUI Builder [44].The triage system is planned to be integrated into the hospital's information system should be able to acquire patient medical history to aid the triage process.As the database, MySQL is planned to be used.The reason for these choices is previous expertise on the technology of the team or ease of integration with the hospital information system.
As there is not a formally defined risk management process in SCRUM the team has not conducted a traditional risk assessment.However, they have defined an initial set of 15 use cases as high level requirements such as View non-triaged patients, View triaged patients, Triage a patient, View patient medical history, etc.They have agreed that 3 (such as calculate triage category and assign treatment order of patient) of the 15 requirements will be more difficult to develop.They have foreseen to conduct state based verification for the critical objects within the scope of these requirements.However, they do not have any risk assessment output for the general software project risks.This is more or less typical for small teams working for with an agile methodology.They have considered a set of tools from the search engine including Jira [45], Risk Radar [46] and Risk Management Studio [47].Most of these risk management tools will not provide a predefined set of risk and probable results, except for a number of risks limited to area like security.In contrast, our tool provides initial predefined software project risk factors with probability and impact levels based project attributes.

C. Conduct of the Risk Assessment with the Tool
The risk assessment is conducted with the team lead and developers and the method/tool developer in a 3-hour meeting.As the tool is currently in prototype state, the tool developer was present in order to explain the details of the use tool and explanation of the terminology used in the proposed risk register and use of the decision support techniques implemented.A set of informally prepared documents related www.ijacsa.thesai.org to the scope of requirements of the project and system proposal for the bid were present.Each risk item is evaluated for its probability and impact level by the team and agreed with various discussions.As most of the information in the discussion was tacit, the referral to documents was little.The tool's graphical user interface has two stages.The first stage acquires general project information.In Figure 6, we provided the tool with this information and saved the progress.
Later input stage of the tool is the evaluation stage with default probability and impact levels and practitioner defined probability and impact levels as demonstrated in Figure 7.These scales are continuous that are called "visual analogue scales" [48] which give the practitioner better control and comfort in ratings.After customizing probabilities and impact scales according to the case, the tool generates a risk matrix for new data alongside the risk matrix for default data.
The team lead has made following observations during the conduct of the assessment: 1) Initial risk register (128 risk) embedded in the method/tool was useful for them different from any tools the team lead had used in his previous experience such as Jira, Radar, etc.He agreed that most of the risk factors might have impact on the software projects in general.
2) The historical data gathered from other practitioners used for generating approximate probability and impact levels helped the team to elaborate on their rating decision of impact and probability levels for each risk.In addition, the initial risk register provided guidance to rate risk factors for which they were unable to give levels either due to missing information or lack of consensus.
3) Risk prioritisation automatically generated by the tool will help them to focus risk remedial actions in a more focused and efficient way.
4) As scale rating, provided by a slider as 0-10 implicit levels was easy to assign for the practitioner intuitively rather than giving discrete 0-3 ratings.This visual assignment with more adjustments to the initial historical ratings eased the assign changes.(Note: This also provides us to make better predictions for specific failure types by the fuzzy prediction algorithm.)Fig. 6.Data register stage screenshot 5) In fact, historically generated risk ratings were proposed by the tool to assist decision making, it became clear that for specific project and organization setting, there could be radical differences for some of the risks.Figure 7 demonstrates these differences in this case.
6) The team agreed that some of the risks they have not really thought about the triage project risk existed such as Backup Issues, Potential Increase in database size, and Security Risks.
7) Overall risk score of the project calculated by the tool and potential failure type estimation provided by the tool may be used as adjustment factor for project cost, schedule and resource.www.ijacsa.thesai.org

D. Results
According to case data, High/Major Concern risks in custom risk matrix are available at Table 4.In addition, total quantitative risk score for case project is 50.30that verbally can be categorized as "high" level.The qualitative value of high is a fuzzy value.A fuzzy value solves the problem of uncertainty with uncertain answer.For instance, Figure 9 shows a peak point of 50/64 and 51/64 with the lowest point of 0.1 in 33/64.It means the risk can be called 50/64 and 51/64 risky (nearly high) most of the times, and with a very low possibility it can be called 33/64 risky (near mid).It also means the risk cannot be categorized as sub-mid and below 32/64 at all.The same also applies to all other fuzzy numbers in a similar manner.
As mentioned in early sections, a failure mode analysis is also provided (Figure 8).According to the analysis of case data, the risk of failure for "Project Over-Schedule" in the case is low to medium.Result of this analysis is represented in Figure 9.The result of failure mode analysis in a single iteration of risk assessment may not provide necessary information regarding the possibility of failure, as these results are more like relative results than absolute.
This means the inference model is meant to be used as comparison model than an evaluating model.For a better comprehension regarding the project's failure mode probabilities, all failure modes are calculated -using the tooland compared.The calculations are done using both proposed risk values and practitioner defined risk values of the case.Figure 10 is a demonstration of the comparison.As demonstrated in Figure 10, all failure mode ratings are in the range of 14/64 and 19/64 which can be represented in form of 22% -29%.It can be concluded from the results that all failure modes are pretty far from being high, but relatively "Defects in Application" is more probable to occur than the rest.This can help the developer to generate a response to "Defects in Application" failure mode.This failure mode has been marked as relatively high in both predicted data and practitioner data.These results are proposed to guide the www.ijacsa.thesai.orgdeveloping teams to take more precautions regarding the related risks.But for a better observation, it is recommended for developing teams to keep observing the risks and performing failure mode analysis in every step of the development.Failure mode values are mostly intended to be used as a comparison value of a failure mode in different time spans.
In a risk management cycle, it is very important to create responses for risks.As for this study, the response analysis is out of scope, but in this case study we decided to produce some suggestions to emulate a real risk management condition.Table 5 is a brief demonstration of possible and suggested responses in literature [6], without considering root causes.It is important to point out that only risk responses addressing the root cause of some, namely organizational risks may be truly effective [49].However, this study does not provide a root cause analysis.Therefore, it is not expected to have an accurate risk response analysis.


The management should have a concrete description about the capabilities of each member of development team while estimating for the scope, size, and cost of the project avoiding optimistic estimations.
V. VALIDITY There are threats to validity and we try to address them based on categories of validity threats which are pointed out by [39].First threat to validity (Construct validity) of this case can be considered as possible misinterpretations of risks during the assessment.Subject practitioners might misinterpret the questions under normal circumstances, but in this case study an interview was conducted in first degree and interruptions were made during the interview to assure correct interpretations.
Another validity issue (internal) is correctly predicting risk factors and failure modes.No logical link is considered.The relations are indirectly established by data and via prediction method introduced.To get the more valid results and facilitate the use of the method, it is may be desirable to reduce risk factors as high cross correlations are observed.With further data, the method and relations can be improved.Also as pointed out by [50], it is not advised to use too many criteria in FIS.Thus, reduction of dimensionality in risk factors is expected to be effective in further validity of prediction.
The case study is only valid for projects with agile methodology and organizations with lower maturity levels (1 or 2) and cannot be generalised any further.Extending the project setting further can be a threat to the validity (External validity) of this case study.In order to extend the case study further, data must be improved to cover wider project settings.This research proposes risk assessment method and tool that the results might alter with different input data, but the logic behind the method will not change.It is important for future interested researchers to consider input data for training of the tool and do not rely on the exact same outputs.As mentioned earlier, this case study and whole study can be improved by improving input data and the validity of the tool improves as the data set improves.In this study, we introduced a tool for small sized software development teams, with ability of providing initial risk set and rating recommendations.Additionally, we provided a fuzzy method based tool to facilitate the risk assessment by factors and their consequences in form of failure mode analysis.In addition, the method produces an overall project risk rating.All this information is useful for small-scale software companies with limited resources, especially at project bid, initiation phases and acceptance decisions.As explained in this case study, proposed risks and predicted scores are in mid to high level which are close to expert expectations.Another observation is based on comparisons of the automatically predicted failure mode scores (which are based on initial and automatically suggested risk ratings) and the predicted failure mode scores from practitioner manually altered input data shows a similar pattern in relatively high and relatively low failure mode scores.For instance, in both predicted failure mode scores (practitioner altered and automatically generated), the failure mode of "Defects in Application" poses a higher threat to the project and failure mode of "Customer not Satisfied" poses a lower threat to the project.Thus, in an overall conclusion, the method provides strong guidelines regarding the risk for practitioners and the steps of identifying, analysing and tracking risks.The method can possibly predict most common failure modes according to project data.
The tools risk rating proposal and prediction accuracy will certainly improve and results that are more generalisable may be drawn, as the usage of the tool by practitioners will increase the number of data points used by the tool.In addition, prediction method has potential for further improvements in order to point out influential risk factors for various failure modes.Additionally, a deeper study on risks and their characterisations can be conducted similar to [51] in order to have better risk control and management phases in future studies.It is also planned to provide root cause study and therefore a risk response advice in next version.

Fig. 1 .
Fig. 1.Study flow II.RELATED WORKS During last three decades, various software risk factors, assessment and analysis tools have been introduced and explored.Majority of these studies can fit into three categories of: (1) Researches focusing on software risks and risk factors identification, (2) Researches focusing on aggregation methods of risk factors ratings, and (3) Researches proposing risk assessment and risk management tools and methods.In the following, some influential researches, specifically related to software project risk assessment are provided.

TABLE I
Not much chance this will become a problem (x<%70) www.ijacsa.thesai.org

TABLE IV .
MAJOR RISKS IN PROJECT

TABLE V
Ensure that there is appropriate technical ability.Take into account the developers' skills assigning tasks.Staff Turnover  At project start up, define undisputed areas of responsibility for all participants as well as the relational roles being instituted people management