Fuzzy Failure Modes Effect and Criticality Analysis of the Procurement Process of Artificial Intelligent Systems/Services

—This study focuses on the ranking of risks associated with the procurement of Artificial Intelligent (AI) systems/services for UAE public Sectors. Considering the involvement of human-based reasoning, this study proposes to use Fuzzy Failure Mode Effect and Criticality Analysis (FMECA). The risks were identified from the literature and subsequently, using 40 interviews with practitioners, the final list is developed on the basis of the presence of risks in the AI procurement process. For Fuzzy FMECA, the input data is collected from fifteen experts. The values of Severity (S), and Detection (D) for each risk element are averaged to use as input. If-Then rule-based fuzzy inference system is employed to obtain the Fuzzy Risk Priority Numbers of risk elements. The traditional RPN and Fuzzy RPN numbers are compared and it is found that fuzzy RPN gives a realistic picture of the ranking of risks. Privacy and security risks, Integration Risks, Risk of Malfunction of systems/services, and Ethical risks are found to be high priorities. This study provides valuable insight to policymakers to develop strategies to mitigate these risks for smooth procurement and implementation of AI-related Projects.


INTRODUCTION
The United Arab Emirates (UAE) has been at the forefront of embracing technological advancements and digital transformation in the public sector.Artificial Intelligence (AI) systems have emerged as a critical enabler in the UAE public sector, transforming operations, improving decision-making, and delivering efficient and citizen-centric services [1].The UAE government has led smart government programs aimed at leveraging AI's potential for improved service delivery.These programs are aimed at exploiting Artificial Intelligence (AI) technology in fields such as healthcare, transportation, education, public safety, and e-governance [2].Artificial intelligence-powered platforms streamline administrative operations, provide personalized services, and enable datadriven decision-making.The UAE public sector hopes to increase operational efficiency, optimize resource allocation, and improve overall service quality by integrating AI technology.
Artificial Intelligence (AI) solutions have received much attention as part of this journey.However, the AI procurement process carries inherent risks that must be identified and handled efficiently [3].If not adequately handled, these risks can negatively influence the successful adoption and use of AI technology.In addition, AI procurement risks can have financial implications such as operational disruption, Ethical and legal concerns, reputation damage, and missed opportunities [4].Understanding the effects and putting appropriate risk mitigation measures in place is critical for public sectors looking to reap the advantages of AI while minimizing the negative outcomes.Failure Mode Effect and Criticality Analysis (FMECA) is a tool for identifying risks in complex systems.
FMEA may be used in various processes, including procurement (Skelton, 1997).In the process, it is also utilized to assure the safe functioning of complicated monitoring and control systems.FMEA methodically identifies the impact of risk elements on the system and assesses the importance of each failure mode in terms of system performance.The approach is primarily utilized in a wide range of technologies to investigate and comprehend component/system failure.FMEA is known as failure mode effect criticality analysis (FMECA) when it is used to prioritize failure modes.The failure modes are ranked using the Risk Priority Number (RPN) by FMECA.RPN is frequently computed as the product of failure mode occurrence (O), severity (S), and non-detection (D).According to Certa et al. (2017), O stands for the frequency of occurrence of the failure mode, S for the severity of the harm that particular failure modes with occurrences of O can cause to systems, processes, and the environment, and D stands for the likelihood that the failure modes with occurrences of O and S won't be detected.
Although there are various applications for FMEA, several researchers have highlighted the various shortcomings of the traditional Risk Priority Number (RPN) that is employed in FMEA.The most significant shortcoming is subjectivity and Lack of Consistency.The subjectivity and lack of consistency in the severity, occurrence, and detection ratings given to potential risks by RPN is one of its key issue.The same risk may be evaluated differently by various team members, resulting in variances in the final RPN ratings.This subjectivity can undermine the accuracy of the findings [5].To overcome this limitation, fuzzy based FMEA was used by various researchers such as [6][7][8][9].
The approach for creating a fuzzy RPN (FRPN) is described in this study.It may partially address some of the www.ijacsa.thesai.orgconstraints mentioned in the literature.The fundamental inputs for producing FRPN are the fuzzified form of frequency, severity, and non-detection of the failure modes.The following stage in fuzzy FMECA is to establish the fuzzy rules.For this stage, the original classes of O, S, and D and the corresponding RPN class are used to establish the rules.The next stage is to use linguistic variables and membership functions to transform the crisp input data into fuzzy values.The assessment of the rule bases created during the study comes next.The last stage to transform the fuzzy output into crisp output is called defuzzification.The AI procurement process in UAE public sectors is subjected to this process.In addition, Comparisons are made between the outcomes of conventional and fuzzy FMECA.
The rest of the paper is as follows: Section II presents the literature review highlighting the rating of risks associated with procurements and tools and techniques for this purpose.Materials and Methods utilized in this study are presented in Section III.Section IV presents the results and discussions.Finally, Section V presents conclusions drawn from this study.

II. LITERATURE REVIEW
For the procurement process to be effective, risks must be identified and managed.The identification of risks and mitigation processes can be facilitated by a number of efficient tools and methods.The risk register is one such instrument, which entails methodically identifying possible hazards, evaluating their effect and likelihood, and developing suitable mitigation strategies.There are numerous methods/ techniques are employed for risk analysis [10].Bathrinath, Bhalaji [11] used AHP-TOPSIS method for risk assessment and ranking in a Textile Industry.To analyze the risks in urban stormwater infrastructure systems, Shariat, Roozbahani [12] applied fuzzy spatial multi-criteria decision-making.Data analytics is also applied as a tool for the evaluation and improvement of procurement processes [13].Delima, Santoso [14] applied a dynamic system development model for the development of purchasing model for agriculture e-commerce.Among all the tools and techniques, Failure Mode and Effect Analysis (FMEA) is a widely used tool for risk analysis.
The FMEA approach, which was developed in the 1960s for the aerospace industry, is a powerful tool for assessing possible risks [15].In complex systems, this method is very helpful in averting undesirable outcomes [16].FMEA is widely utilized in a variety of industries, including those in the aerospace, automotive, nuclear, electronics, chemical, mechanical, and medical domains [17][18][19][20][21][22]. Aldenny, Kristian [23] applied FMEA analysis to determine possible risks with the government's electronic procurement process using the FMEA approach.To improve the purchasing process of public procurement in hospitals, Kumru and Kumru [24] applied the FMEA to indicate the levels of risks associated with potential issues.Nahavandi and Tavakoli [25] applied the FMEA combined with the TOPSIS method to identify the risks associated with procurement in the automotive supply chain.The Modified FMEA is also used for ranking risks associated with military weapons procurement [26].Handayani [27] also applied the FMEA analysis to evaluate the risks associated with supplier-buyer transactions in a supply chain procurement.These studies infer that FMEA is an effective tool for analyzing the risks associated with the Procurement process.
There are some limitations of the traditional FMEA process which doesn't capture the subjectivity in a precise way.To overcome this difficulty, a variant of FMEA was developed using a fuzzy set theory known as fuzzy FMEA.Incorporating the criticality analysis in Fuzzy FMEA, the Fuzzy FMECA was developed as an advanced method for ranking the risks.Numerous fields, including engineering [28], manufacturing [29], healthcare [30], and more recently, new technologies like artificial intelligence [31], have found use for fuzzy FMECA.When working with complicated systems where it might be challenging to exactly quantify risk variables, fuzzy FMECA is particularly helpful.Fuzzy FMECA allows decision-makers to consider linguistic variations and subjective aspects by introducing fuzzy logic, making risk analysis more adaptable and understandable.
A crucial step in purchasing cutting-edge technology to improve organizations operations as well as decision-making is the procurement process for Artificial Intelligent (AI) systems or services.Predictive analytics, machine learning, and other disruptive features that AI systems provide have the ability to completely change a number of sectors.It takes strategic decision-making to choose the best AI solutions that fit an organization's objectives, requirements, and capabilities when purchasing AI systems or services.For effective AI integration, suppliers and technology providers must be properly evaluated and chosen [32].In order to evaluate failure modes, their impacts, and criticality while procuring AI solutions, fuzzy FMECA (Failure Mode Effect and Criticality Analysis) is extremely important.By introducing fuzzy logic, which enables the management of uncertainties and ambiguity frequently found in AI systems, fuzzy FMECA expands classic FMECA approaches.The inherent complexity and changing nature of AI technology make this competence essential in the context of AI procurement [33].Complex algorithms and learning models are used in AI systems, which might result in erratic and unexpected behavior.Fuzzy FMECA can manage ambiguous inputs and hazy data, providing a more accurate evaluation of likely risk mechanisms [34].AI systems often produce results with varying degrees of ambiguity.Fuzzy FMECA utilizes linguistic variables to quantify the degree of criticality, providing a more nuanced evaluation of failure effects [35].

III. MATERIALS AND METHOD
FMECA typically consists of two steps: (i) Use failure mode and effect analysis to distinguish between various failure types and their impacts, (ii) group failure mode criticality analysis according to the likelihood of occurrence and impact [15].Traditional FMEA calculations involve generating a risk priority number (RPN).Three main factors are needed when doing an FMEA: occurrence (O), which indicates the likelihood of accident occurrences.The term "severity" (S) refers to the seriousness of the consequences of failure modes not being detected.Non-detection (D) eliminates the possibility of detecting failures before they occur.The sum of the three www.ijacsa.thesai.orgaforementioned operations yields a risk level known as the risk priority number (RPN).
The shortcomings of the traditional Risk Priority Number (RPN) employed in FMEA have been noted by several academics.The relative relevance of O, S, and D is not often considered in most FMEA analyses, according to Wang, Chin [36].According to [37], different O, S, and D combinations may result in exactly the same RPN number, but their hidden risk implications may be quite different.
An intelligent framework that summarizes established multi-valued logic for problem-solving under vulnerability is referred to as fuzzy logic.Control designers may easily implement control methodologies used by human administrators thanks to fuzzy logic [38].Its structure is based on the fact that some problems might be solved based on related knowledge or expert learning while others did not require the proper or accurate esteem.It is based on a likelihood hypothesis for the conversion of crisp to fuzzy input, which will be handled by referring to the state of the fuzzy rule base created by experts.Fuzzy rule base preparation, which converts fuzzy output to crisp output, might solve the problem.Fuzzy logic was a mechanism for making decisions when information was vulnerable and taking flexibility into account.
It is feasible to obtain a complete description of potential risk and its impact by using fuzzy logic in failure modes, effects, and criticality analysis (FMECA).This strategy might effectively address the problems and clearly identify any risks and implications.Furthermore, it could also build trust at the same time.It enables an assessor to directly assess the risks associated with failure by using language concepts in criticality evaluation.Ambiguity, subjective information, or data including quantitative information may be used in the evaluation and organization, although not always unambiguously.The combination of severity (S), occurrence (O), and non-detection (D) parameters had a more flexible structural design.The use of fuzzy logic in FMECA for ranking the risks associated with the procurement of AI systems/services is the main topic of this paper.

A. The Proposed Method
The proposed method is based on the work done by Makowski and Mannan [39].Risk is normally evaluated using two components, severity, and occurrence, during the risk assessment process.Traditional risk predicts frequency and severity as discrete values/categories.Due to a number of uncertainties, frequency and severity values are non-crisp in nature [39].When there are uncertainties in the parameters used for risk computation, fuzzy logic can be employed to estimate the risk.In the classification of these characteristics, fuzzy logic does not identify precise limits.The fuzzy risk matrix is created by combining a fuzzy frequency and a fuzzy severity.
The RPN in the FMECA research makes use of three parameters: O, S, and D. O, S, and D crisp input data are generally quantified on a ten-point scale (see Tables I to III).Using linguistic expressions and membership functions, they are turned into fuzzy values.Similarly, the output's membership function (RPN) is constructed.In this study, 1000 rules were designed to govern the output value.As an example (rule No. 996), if the occurrence is definite (10), the severity is substantial (6), and the detection is not possible, RPN is calculated conventionally by multiplying O, S, and D (10*6*6) = 360.The standard RPN, which corresponds to class 9, has been improved by integrating the linguistic phrase "high" matching to this class.A Mamdani fuzzy inference technique is used to convert qualitative rules into quantitative results.The fuzzy set for each rule is aggregated once the rules have been assessed.The centre of area approach is utilised for defuzzification in this work.

The severity of Each risk Effect Rank
No reason to expect risk to have any effect on safety, health, environment, or mission None 1 Very minor effect on product or system performance to have any effect on safety or health.The system does not require repair / restart.

Very Minor 2
Minor effect on product or system performance to have any effect on safety or health.The system can require repair/ restart.

Minor 3
Very low effect on system performance.A failure is not serious enough to cause injury, property damage, or system damage, but can result in unscheduled maintenance or repair Low 4 Moderate effect on system performance.The system requires repair.A failure may cause moderate injury, moderate property damage, or moderate system damage which will result in delay or loss of system availability or mission degradation.100% of the mission may need to be reworked or process delayed.

Moderate 5
System performance is degraded.Some safety functions may not operate.A failure causes injury, property damage, or system damage.Some portion of mission is lost.High delaying restoring function.

Significant 6
System performance is severely affected but functions (reduced level of safety performance).The system may not operate.Risk does not involve noncompliance with government regulations or standards.

Major 7
System is inoperable with loss of primary function.The main purpose of the AI procurement process in the public sector is to fulfil the requirements of public administration, such as government modernization and digitalization (Wegener & Müller, 2018).It can also be regarded as a means to promote good governance (Mehr, 2017; Solihati & Indriyani, 2021).The AI procurement process involves managing materials, purchasing transactions, and activities to ensure the quality of purchased AI products based on the requirements set for them (Karlsson, 2020).It involves all functions, from identifying needs to the selection of sources and awarding and administration of contracts as shown in Fig. 1.AI procurement risks refer to those risks which are associated with the purchase of AI technologies/systems.The increased visibility of associated risks in AI projects led to a more strident clamour for ethical, legal, and secure adoption.Based on the literature and subsequent interviews with practitioners from UAE public sectors, the participants identified the following risks in the procurement of AI projects in their respective organizations: privacy and security risks, integration risk, skills risk, risk of time frame, the risk of financial economic loads, and risk regarding vendors.They also identified other risks such as risk of miscommunication and the risk of system malfunction.The list of risks was also validated by a focus group discussion.The risks and their definitions are represented in Table V. Privacy risk refers to the likelihood of experiencing problems arising from data processing and the impact that it may bring.Security risk deals with the possibility of losses resulting from information security concerns.

Ethical risk
It is associated with the possibility of reputational or moral harm to individuals or organizations.In AI procurement, this risk involves moral dilemmas due to the process of AI-driven dehumanization and displacement of human control Integration Risk In AI procurement, it refers to the probability of failure of the integration of systems, technologies, or information due to system incompatibility.

Skill-related Risk
It deals with the likelihood of lack of or inadequate skills that the workforce may encounter by introducing new technology to the organization.

Legal risk
It refers to the potential failure of an organization to comply with regulations or terms of the contract.In AI procurement, AL algorithms or data misuse can lead to legal liability risks.Risk of Environmental Sustainability influencing Hazards It refers to the probability of hazards posed against maintaining an ecological balance in the natural environment and against conserving natural resources for the utilization of current and future generations.

Financial Load risk
It refers to the probability of losing money or danger than can lead to the loss of capital.In AI procurement, financial load risk may be due to exceeding allocated budget.www.ijacsa.thesai.orgTime framerelated risk It deals with the possibility of time impacting the project, such as delay.In AI procurement, this risk can lead to cost overruns.

Vendor related risk
It refers to risks associated with vendors, such as delays caused by vendors, overpricing of technology provided, security breaches, and unforeseen vendor in capabilities, amongst others, which can cause reputational damage to the organization.

Risk of Miscommunica tion
It refers to the likelihood of the personnel or workforce to fail to communicate adequately (including top-bottom and bottom-up channels of information flow).In AI procurement, this can lead to bias in decision-making.

Risk of System Malfunction
It deals with the probability of system failure, error, or malfunctioning.In AI procurement, this may arise from not being able to regularly update the system's network.
Fuzzy FMECA was performed for AI Procurement process (see Fig. 1).To identify the risk elements, Literature review, Interviews with experts and Focus group discussion were employed.The validated list of risks is presented in Table V.In the proposed study fuzzy logic is used to address the issues in prioritization of these risks.In expert elicitation, the experts, from UAE public sector with extensive understanding of the AI procurement process, used fuzzy language phrases to define the risk variables O, S, and D. The O, S, D values for each risk are collected from the experts using an online survey.In total, 22 responses were recorded.The seven responses were discarded as data were missing or non-serious inputs are seen.Finally, we have taken the average value of O, S, and D for each risk elements and traditional RPN was calculated as shown in Table VI.RPN is a class based on the classes of O, S, and D assigned by the expert, rather than a product of multiplied values of O, S, and D. The different scales used to measure the two components O and S are shown in Tables I and II.The scale used to measure the parameter D is shown in Table III.These measures assess the likelihood of occurrence, severity, and likelihood of nondetection (see Table IV).The risks data came from a survey based on experts' opinion for AI procurement Process risk analysis.11 risk elements were identified and prioritized based on the appropriate RPN score (see Table VI).The complete fuzzification procedure was carried out in MATLAB using the fuzzy toolbox, with the triangular membership function representing the inputs O, S, and D as shown in Fig. 3(a), 3(b), and 3(c).The triangular membership function was also employed to depict the FRPN output membership functions (see Fig. 4).The input values of O, S, and D were provided by experts.As 15 experts' opinions were considered, an average value of O, S and D were taken into consideration.For the analysis, 1000 if then rules were developed as shown in Fig. 5(a) and (b).These criteria are intended to cover all conceivable O, S, and D combinations.The Mamdani min/max inference mechanism is utilised (input method: min; aggregate method: max), and the results are defuzzied using the centre of gravity approach (see Fig. 6).

IV. RESULTS AND DISCUSSION
A basic condition for fuzzy inference applications is that the fuzzy system's output be monotonic with regard to its inputs [40].The suggested model's rule base is nondecreasing as can be seen from the output surface plots as shown in Fig. 7  The values of the inputs (O, S, and D) from the data collecting process are supplied to the FIS system during implementation and result extracted to produce the Fuzzy RPN.The Fuzzy RPN values associated with each risk is shown in Table VII along with priority.It can be seen that the www.ijacsa.thesai.orgdistribution of output value of Fuzzy RPN is more uniform compared to traditional RPN.It can be also seen that Fuzzy RPN numbers obtained are more realistic to interpret.For example, using conventional process, RPN is calculated conventionally by multiplying O, S, and D (10*6*6) = 360.The standard RPN, which corresponds to class 7, has been improved by integrating the linguistic phrase "high medium" matching to this class.In order to compare and validate, the ranking of risk elements based on Fuzzy RPN, As demonstrated in Table VII, a focus group discussion was conducted.The experts from public sector of UAE government were invited and provided with the ranking obtained by this study.Majority of experts expressed their agreement with the ranking obtained by this method.The results obtained suggest that Privacy and Security risk is predominant in the case of AI Procurement Process.Various studies also suggested that the privacy and security risk is very important compared to others [41,42].The privacy and security risks involved with data privacy concerns, data security vulnerabilities, algorithm biasness, lack of transparencies, adversarial attacks on the system, dependency of third-party vendors and so on.Considering the stakes involved, this risk is considered as most important and having highest impact among all risks.
The next risk of major concern is the integration risk.The integration risk may be faced due to various issues such as data compatibility, system compatibility, customization requirements, skill gaps, user adoption, performance and scalability and vendor reliability.The integration risk is also found to be very important [43].Considering the importance of integration for example even a city-level law enforcement agency, for example, may not be aware of all the systems utilised across its many departments, how data is linked, and how the outputs shape their practises and policies.Sanchez-Graells, A. [3] highlighted that integration risk makes it difficult for the public and civil society to engage with the appropriate partners, gather information, and hold anybody accountable.The risk of system malfunction is also found to be a risk of high impact especially in areas where human life at stake such as healthcare sector [44].The AI system malfunction can be happening due to various factors such as system complexity and maturity, data quality and biasness, model transparency, improper testing and validation, lack in robustness of model, integration issues with existing system, security measures which prevent unauthorised use.During the procurement phase, due care to be taken so that system malfunction cannot happen.
Another risk that is categorized of moderately high impact is ethical risk which associated with raise ethical concerns and may result in harm to individuals, communities, or society as a whole.Kuziemski, M. et al. [45] also support that as AI technologies grow more prevalent, organisations must address the ethical implications during the purchase process.
Risk of miscommunication is also found to be of moderately high impact.
In AI procurement, miscommunication risks relate to the possibility of misunderstandings, misinterpretations, or imprecise communication between the parties engaged in the procurement process.As per Grewal and Sridhar [46], these risks might develop at any point during the procurement process, from early planning and needs collection to contract negotiations and implementation.Miscommunication can result in a variety of obstacles and issues, including unclear requirements, misaligned expectations, inaccurate scope and cost estimation, lack of understanding of technical specifications, data access and ownership, and vendor capabilities.
Other risk elements that are vendor related risks are found to be of moderate impact.In the AI procurement process, vendor-related risks refer to potential obstacles or concerns that result from the selection and engagement of an AI vendor or service provider.Choosing the correct vendor is critical since it has a direct influence on the success of the AI project and the organization's ability to meet its goals.Chopra A. [47] has identified several vendor-related concerns in artificial intelligence procurement such as vendor reputation and reliability, Vendor's financial stability, lack of expertise and experience, Vendor lock-in, intellectual property issues, cultural fit, etc..The legal risks involved in procurement of AI system or services are found to be moderate impact.This risk consists of issues arising due to intellectual property rights, data ownership and usage, data compliance, regulatory requirements, etc.Other risk elements are found to be of low to moderately low impacts are risk associated with time frame, finances, skill related and environmental risks.It should be noted that from the perspective of UAE public sectors, finance is not considered to be a major concern as well as skills.Further, timeframe and environmental related risks are also found to be of low impact on the AI procurement Process.www.ijacsa.thesai.org

V. CONCLUSION
The proposed Fuzzy FMECA method was applied to rank the risks involved in the procurement of AI Services/ Systems in UAE Public Sectors.The study was carried out by employing fuzzy linguistic variables for occurrence, severity, and non-detection, and then combining these variables using an if-then rule base to achieve Fuzzy RPN or FRPN.The outcomes of traditional FMECA and fuzzy FMECA approaches are compared.Fuzzy FMECA has been proven to be an excellent approach for prioritizing the risks associated with AI procurement Process.The Privacy and security risk are found to be most important, then, Integration risk and system malfunction risks.Other Moderately impact risks are ethical risk, miscommunication risk, vendor related risk and legal risks.The risk associated with finances, skills, environment and time frame were found be of moderate or low impact.The ranking of these risks is validated by a focus group study.This method can be extended to rank the risks involved in other complex systems or prioritizing the different alternatives for decision making.

Fig. 1 .
Fig. 1.AI procurement process common to the UAE Public Sectors.

Fig. 2
Fig. 2 depicts an overview of the fuzzy logic method.The study consists of three primary steps: (i) Fuzzification employing linguistic variables to turn the three risk factors S, O, and D into fuzzy membership functions.(ii) Rule assessment based on expert knowledge of the relationships between various risks and the effect, as represented by fuzzy ifthen rules.(iii) A de-fuzzification technique generates a crisp ranking from the fuzzy RPN to provide the failure mode prioritization level.

Fig. 2 .
Fig. 2. Schematic of fuzzy logic process for FMECA Using Matlab FIS editor.

Fig. 7 .
Fig. 7. (a): Surface plot of Occurrence and severity vs. FRPN in Matlab Surface viewer., (b): Surface plot of Occurrence and No-Detection vs. FRPN in Matlab Surface viewer.(c): Surface plot of Severity and No-Detection vs. FRPN in MATLAB Surface viewer

TABLE II .
SEVERITY SCALES USED IN FMECA SEVERITY 'S' Risk is hazardous and occurs without warning.It affects safe operation.A Risk is serious enough to cause injury, property damage, or system damage.Risk will occur without warning.
Risk can involve hazardous outcomes and/or noncompliance with government regulations or standards.Extreme www.ijacsa.thesai.orgRisk involves hazardous outcomes and/or noncompliance with government regulations or standards.Potential safety, health or environmental issue.Risk will occur with warning.

TABLE V
VI. RPN AND PRIORITISATION OF RISKS

TABLE VII .
FUZZY RPN AND PRIORITISATION OF RISKS