Review of Public Procurement Fraud Detection Techniques Powered by Emerging Technologies

Numerous studies and various methods have been used to detect and prevent corruption in public procurement. With the development of IT technology and thus the digitization of the Public Procurement Process (PPP), the amount of available data is increasing. Studies have shown progress in this area and have revealed many challenges and open issues geared to the various goals outlined in this paper. Different data mining and business intelligence techniques and methods are being used to develop models that will find any suspicious public procurement process, contracts, economic operators, or to classify observations as corrupt. In addition to using classification models, methods such as association rules and graph databases are used to find relationships between economic operators and contracting authorities, as well as to find daughter companies that participate in PPP collusion. Therefore, this paper addresses a comprehensive review of the emerging techniques and models used for the detection of suspicious or corrupted observations, their goals, open issues, challenges, methods and metrics used, tools, and relevant data sources. The findings show that models are mostly fitted on historical data and move in the direction of an early warning system. Moreover, the efficiency of fraud or anomaly detection depends on data set quality and detection of the most important red flags. The study is presenting a summary of identified fraud detection model objectives such as predicting fraud risk in contracts and contractors or finding split purchases, and detection of used data sources such as public procurement process or economic

www.ijacsa.thesai.org On average, corruption accounts for 5% of the total value of public procurement, which is around 14% of the European Union's (EU) GDP, or EUR 1.9 trillion within the EU, which is one of the main reasons why in former years many efforts have been invested in the field of corruption definition and detecting suspicious actions [5,7]. A 2020 study by the Association of Certified Fraud Examiners published the Report on Professional Fraud and Abuse. This report provided the results of an analysis of 2,504 cases of professional fraud that occurred in 125 countries worldwide [8,9].
Regarding studies and the fight against corruption in this segment, country authorities use various techniques mostly focused on regulating the public procurement process by using different questionnaires and establishing process control; however, the conclusions of the study clearly state that a correctly set public procurement law is insufficient, and there is a lack of control mechanisms for prevention [5]. Fraud committed by those you invited in (e.g., internal perpetrators, vendors/suppliers) represent nearly half of all fraud reported [10]. By Table III in the process of making public procurement fraud, the most responsible is the middle management-more than 37% of reported cases. There are several types of methods for corruption detection and measurement: surveys, administrative data from crime statistics, ombudsmen, pp offices, supreme audit institutions, pp governance risk assessments, and analyses of contracts [11]. A World Bank study presents a few major technology trends for public sector fraud and corruption such as big data, cloud computing platforms, artificial intelligence, and machine learning, biometrics (ID4D), FinTech digital money, distributed ledger technology or blockchain, and the Internet of Things (IoT) [8].
Nevertheless, it is difficult to create efficient corruption detection models if there isn't enough quality and diverse data. So, it is widely accepted that access to public information increases the level of transparency in the fight against corruption [11].
Each country has self-organized state-level preventative and anti-corruption agencies responsible to establish the mentioned procedures and monitor law enforcement. With the somewhat onward digitization of the public procurement process, there is an ever-increasing amount of data that is unconnected and largely unstructured; but, with some effort and specific techniques, scientists can use that data to analyze the public procurement process and find adequate corruption indicators. In this study, the approach of detection of public procurement corruption using advanced digital techniques and data models will be explored. With this modus, the study entered the Big Data area, where various advanced statistics and data mining techniques are used to elicit such knowledge. Thus, the fight against corruption in the public procurement segment is not a novelty.
Previous research related to the literature overview of using emerging techniques (e.g., Artificial intelligence) in public procurement fraud detection has four research questions: what are the characteristics of the organizations in which the investigations are carried out, the technological tools, and data mining methodologies and techniques [42]. The focus of the mentioned detection methods is based on data from public procurement contracts. Detection methods, as well as techniques, largely depend on the input data set so one goal of the study is to find and summarize the data sets and methods used for the detection of fraud in the public procurement process. Besides tools and methods, their metrics, challenges, and open issues, the relevant question is what indicators or red flags are used. Most emerging advanced technologies depend on data labeling, not only detecting corruption but also anomalies and suspicious tenders. The open question is how corruption is defined because models are estimating the probability of corruption, predicting the number of bidding tenders, predicting fraud risk in contracts and contractors, finding split purchases, etc. In this research, systems that use advanced technologies and tools for detecting anomalies, fraud, and suspicious public procurement procedures, although they represent modules and closed systems about which there is not enough public information will be detected. Overall, the study will try to obtain more robust results.
The paper is organized as follows: Section 2 presents an overview of the research done in the field of detecting fraud in public procurement by using data mining techniques and machine learning models. The section is divided into subsections where conclusions about the models, methods, metrics, data labeling approach, and corruption detection indicators used are presented. Section 3 represents a short description and list of tools that are used for analysis, monitoring, or fraud detection in the area of public procurement. In Section 4, open issues and further research opportunities are highlighted. Finally, conclusions in Section 5 are provided.

II. PUBLIC PROCUREMENT FRAUD DETECTION
By analyzing scientific databases (SCOPUS, ScienceDirect, Google Scholar, and Web of Science) in the period of last 5 years, after segmentation, a total of 23 scientific studies that are relevant to the study area have been detected and reviewed to gain this literature overview. The main inquiry is made from the combination of next keywords: public procurement; public procurement fraud, public procurement anomalies, public procurement indicators, public procurement red flags, public procurement application, public procurement system, procurement data mining, public procurement methods, public procurement artificial intelligence). Certain studies were focused on legal and organization frameworks, interviews, or statistical models so they are excluded from this overview. Within the scope of public procurement, different procedure types were found, such as open, restricted, and negotiated procedures, auctions, etc., for which different public procurement rules apply. Different rules result in different processes, and with a lot of dissimilar corruption indicators, this complicates fraud detection [4]. Accordingly, [12] dealt with collusion detection in auctions and provided a review of the methods and data set characteristics. The authors concluded that a large amount of different data is needed for the purpose of quality model results. Even though they described their research goals, the lack of described techniques, models, and the data mining process was noticed in this review. On the other hand, [13] of the total of six studies cited in the literature review, three studies related to the detection of procurement corruption. In contrast, others were related to segments such as the supply chain and the economic sector, and thus they are not relevant to this study. How approaches intersect and created a complex matrix that can be structured using technology and AI support were identified. Diversity in the approaches used in the selected cases leads to the main question: "What were the goals of the studies and which methods are used for fulfilling them?" Therefore goals of the studies and models with the used methods were extracted (Table IV). Also, for this inquiry, two more pieces of information are interesting. Therefore, dealing with the classification of observations or corruption prediction, it is interesting to find what kind of features or data are used to proclaim some observations as bad, suspicious, or corrupted and what metrics are used in order to compare results.

A. Corruption Detection Methods and Models
The use of various analytical and statistical methods was discussed by [5,14]. According to them, corruption detection was first done in the telephone, insurance, and banking industries, which takes a lot of time and domain knowledge from various areas including, legal, financial, commercial, and others. By data in Table IV, it is important to emphasize that the research in this segment is largely focused on the development and application of predictive models and the detection of relationships between economic operators and contracting authorities. In essence, this is a complex matter and is composed of statistical methods, various data mining methods, and machine learning. The literature review shows that researchers used two very familiar approaches, namely supervised and unsupervised learning. These methods differ in target variables, that is, in supervised learning, we have precisely defined target variables as the output of the model, while in unsupervised learning we do not have pre-set variables; so, the models are suitable for seeking anomalies. Still, depending on the model, improvement sometimes is needed to add classified observations [15]. It is important to emphasize that such models are used to detect anomalies, which may be the subject of analysis in some later steps [16]. In general, almost all studies show that the fraud detection model is divided into few steps showed in Fig. 1.
The most commonly used methods in the studies are linear and logistic regression, neural networks, and Naive Bayes algorithms since they are most used for classification and clustering. Namely, models are fitted on historical data and move in the direction of an early warning system that can provide pre-determined supervisory bodies with insights into the risks associated with concluding contracts with risky economic operators [1,18,21] or can identify potential cartels or collusion behavior using associative rules or graph databases algorithms to see the relationships between economic operators and eventually their daughter companies [12,25,26,29,32,43].
The observed studies and created models are used for several different purposes in the detection of corruption in public procurement and at various stages of the public procurement process. Following the observed studies by Table IV,

B. Corruption Indicators
One of the essential segments and research questions is certainly the input data. The studies are focused on the detection and analysis of high-quality corruption indicators, risk patterns, or red flags as representatives of corruptive or suspicious actions with the aim of developing models with the best predictive features [3,4,5,6,22,24,31,32]. Including different databases, pattern recognition, and elicitation knowledge is part of the Knowledge Discovery area. In short, studies have suggested that by applying the Big Data approach and data mining methods, better results will be achieved, and better indicators can be found [6,32].
For this very reason, sets of input data are being attempted to expand with different kinds of databases ( Fig. 2) to create a data lake or unified data set that can support patterns of suspicious or even corrupt behavior in the procurement process. Certainly, the quality of fraud or anomaly detection depends on the quality of the red flags.  Pattern is Implemented only once -Adopted According to [32].
More than 200 different indicators are known so far and are used as variables in algorithms, models, or techniques to perform some of the identified objectives in Table IV [31,32,33]. Due to space limitations, a few of them will be mentioned, as follows:  Unusually short deadline between the announcement of the tender and the deadline for submission of bids.
 The time between the announcement of the tender and the signing of the contract.
 A high percentage of administratively rejected bids in the procedure.
 An unusually small number of correct bids at the level of the procurement procedure.  High ratio of the value of contracts signed under special conditions in relation to the total value of all contracts of an individual client, etc.
In the same way, selecting non-open and less transparent tender procedures reduces the number of possible bids and opens space for awarding a contract to the same wellconnected company [23].
Fazekas and Toth used linear regression to find the most useful indicators. Nevertheless, there exist a lot of white papers or studies that present corruption indicators [4,31]. Still, the problem is always choosing and using the right indicators even if we have an indicator that doesn't mean that we have a right and useful red flag. The process of getting indicators seems to be manual by using the expert's domain knowledge, interviews, or surveys [3,4,28]. Authors have searched for different kinds of methods to automate and improve this process. They have implemented dimensional reduction to reduce and include indicators with the best performance using Correlation Analysis (CA), Principal Component Analysis (PCA), and Weighted Principal Component Analysis (WPCA) [27].
Data collected by government bodies or agencies are attempted to be merged, meaning data on the contracting authority or the economic operator, the people who run the company, political connections, etc. These data are actually the attributes needed for the model to make a conclusion or an output prediction, and if we are in a large area of input data, this data needs to be normalized. For this purpose, the Big Data approach is used to process the data in various ways and format it in a model-suitable format, e.g., text-mining techniques such as word tokenization, vectorization, and steeming are used in word processing [17,29]. It is also important to note the application of the above-mentioned method to documents that are a major part of the tender [28]. Keeping all this in mind, it is important to extract the knowledge from a set of data and find patterns and correlations between variables. From the results in Table IV, the used data sets with a few examples can be summarized as:  Public procurement process data (e.g., type of procedure, estimation price, data type attributes, number of bidders, call for tenders' modification, process duration, tender documentation).
 Economic operator data (e.g., board members, address, contact person, annual tender plan).
 Electronic invoices with products data (e.g., unit of measure, a specific product, product quantity, product price);  Databases of sanctioned contractors; blacklist contractors; court judgments (corrupt cases); political ties.
 Banking records containing specific details of each transaction.

C. Data Labeling
The next significant observed segment is the attributes, according to which certain models learn to recognize or detect certain prediction classes (mentioned target variables). It has been noted in the studies by [5,6] that only a small number of authors have a clearly specified data set that contains information on whether competition was corrupt, which would mean that there must be a verdict regarding a particular procurement process or a valid classification from that of a superior's institutions, which is not the case in all countries. For this purpose, the authors have taken different features to make some observations suspect, bad, or risky (not necessarily those names) and thus have created prediction classes and introduced certain metrics for that segment, e.g., "Suspicious": single bid tenders, extensions to contracts, sanctioned contractors, blacklisted contractors, and the same institutions on the same month and year that added up to more than 8,000€; e.g., "Risky": excluded contractors because of fraud, corruption, violation of anti-trust laws, temporary suspension of the bid, declaration of non-trustworthiness, impediment to bid and hire, suppliers' serious errors in the execution of any contract, and bidding company receives at least one severe penalty due to the serious irregularity.
As part of the current analysis, it is noted that the investigations aimed at establishing models of detection of corruption risks related to the execution of contracts or corruption by the Economic Operator are based on data contained in databases where irregularities in the execution of contracts have been reported due to fraud, corruption, or violation of anti-trust laws. On the other hand, calculation of corruption risk or classification of corrupt PPP is based on a variable such as the "number of bids," where the aim is to predict if the tender will end up with one bid [17,23,24,28,29,30,31]. The authors have proclaimed these kinds of observations as suspicious.

D. Metrics and Results
Objective testing evaluation requires appropriate methods for accurate measurement.  [20,34]. Another very used useful metric in the case of linear models is r2 (pronounced "R squared"), which ""measures the proportion of variation in the responses explained by the available predictor" [35].
Although there are no patterns used, the metrics themselves are different. Namely, accuracy ranges between 30% [16] and even 99% [24], which is pretty "bad", but also are results that are too good that even the authors commented on it. A similar situation is with recall and precision moving at similar intervals. What needs to be emphasized is that, in some studies by [4,26,34], other than accuracy, no other relevant metric is mentioned, which is not enough. AOC metric told us about the power of the model and was about 0.87 for the study that had abnormally high accuracy (more than 99%). [19] R2 was used by Fazekas, as its purpose is to show the power of the linear model. In some studies, r2 varies between 0.2 and 0.55. Detection of cartels and collusion behavior is based on associative rules or graph databases algorithms where the aim is to find relationships between economic operators and contracting authorities. The basic measure in this area is an indication of how often the rule has been found to be true named confidence. Process mining has also been used to analyze the differences between single-and multiple-bid tenders. Process mining has proved that procedures with more than one bid do last longer and that some single bid tenders lasted an extremely short period of time [31].

III. PUBLIC PROCUREMENT TOOLS
In previous chapters, some conclusions about the topic of the used methods, input data, labeling data, and metrics were made. All these components ultimately met conditions to create a system for monitoring or analyzing PPP. It is clear that the availability and reliability of the data are the basic premise for the model and can produce the best results. It is necessary to have quality and reliable communication between different state bodies and the connection of all relevant data that the model can use. Otherwise, the lack of the mentioned components can make the process of monitoring and the analysis of all these data quite complex [36].
Even though public procurement processes are defined by policy acts, states independently develop and digitize their systems. Croatia implemented a Public Procurement System (PPS) named EOJN (https://eojn.nn.hr/), which is fully electronic, but without any analytics or monitoring tools. India (https://eprocure.gov.in/) has the eProcurement System, which enables the Tenderers to download the Tender Schedule free of cost and then submit the bids online through this portal. The Irish government implemented an electronic tendering platform administered by the Office of Government Procurement (https://www.etenders.gov.ie/). "The site is designed to be a central facility for all public sector contracting authorities to advertise procurement opportunities and award notices". Some of the countries developed one central platform for PPP, but some have more than one managed by private sectors, for example, Germany or Austria. In the case of multiple PPS, there is a need to have one portal where economic operators can have an overall view of all tenders. For example, the USA have a private project named Tendersinfo (https://www.tendersinfo.com/) as an "online government Tender information provider company, helping business across the globe in finding business opportunities". On the EU level, there is TED (Tenders Electronic Daily https://ted.europa.eu) as an online version dedicated to the European public procurement overview with an amount of 746 thousand published procurement award notices yearly, including 235 thousand calls for tenders worth approximately €545 billion.
The basic components are certainly electronic forms for bid submission, but part of the public procurement system also contains tools for analysis and monitoring of the entire process, whose main purpose is to generate reports, monitor budget spending, and research anomalies. The European Commission and the member states protect their financial interests by using advanced technologies and by the digitalization of the public procurement process itself. Of course, as part of such processes, it is necessary to change and adjust the laws and policies that result from it [3,5,37]. As part of the research, several advanced systems in the field of public procurement were detected and will be described in the continuation of this paper.
Brazil's decision support system for fraud detection in public procurement is a robust tool implemented with the aim of systematic analysis and the identification of the main risk patterns, such as collusion between bidders, conflicts of interest, and risk companies using algorithms such as graph theory, clustering, and regression analysis with advanced data science methods [32]. A similar tool was developed in Africa, named Tendersure (https://www.tendersure.co.ke/), which is based on web technology but does not use advanced technologies and tools such as the system from Brazil. In Ukraine, as part of the national public procurement system, there is the DoZorro tool, which is based on artificial intelligence or supervised learning, and its purpose is to find suspicious tenders depending on risk indicators [33]. The Red Flag system in Hungary (www.redflags.eu) was created in a similar way. Its purpose is to detect risky public procurement procedures and thus present an early warning system. The system is still at an early stage of development. As can be seen from the details of the tools, not all tools are based on advanced algorithms or some form of artificial intelligence; some are also analytical and statistical tools. In Croatia, there is the Integrity Observer System (http://integrityobservers.eu), which is in the form of a dashboard based on data collected directly from the electronic public procurement system and data collected from interviews with the local community. The system is like ERAR (https://erar.si/), which is an online service made in Slovenia. That service provides information on the flow of public money and is linked to contracts between economic operators and the contracting authority. To give the public efficient and transparent public procurement procedure analytics, each country has its own electronically public procurement system that has at least some of the application modules adapted for such purpose (publicly published contract register, register of procurement plans, payment records, etc.). For international transparency, some of them are Macedonia, Georgia, Slovakia, Poland, etc. [38]. www.ijacsa.thesai.org It is important to emphasize that when the public procurement process is subject, tools designed to analyze the distribution and use of public money of European funds were included, whose contracting processes must also be carried out by public procurement laws. The European Commission has implemented a risk assessment tool, ARACHNE, to detect and prevent projects that are vulnerable to fraud, conflict of interest, and irregularities [5,6,20]. In addition to the mentioned tool, an analytical database with all the information about users and projects funded by the Directorate-General Communications Networks, Content, and Technology was created with the aim of detecting links about people and projects with all their data, such as phone numbers and addresses. This system is not a warning system, but it is used in cases of doubt if there are irregularities in the project [5]. DAISY is a tool for data mining developed by the Directorate-General for Research and Innovation with the purpose to identify links between users of funds in the scope of research projects. DAISY is used when there is a suspicious fraud of the specific user of the funds [5].
A lot of tools to mitigate fraud (Table IV and Section 3) or public procurement corruption detection was detected. It is difficult to find information on how they work and what methods of corruption detection uses. This is one of the open questions and action points for further research.

IV. FUTURE RESEARCH
There are several open issues related to the topic of estimation and detection of corruption in public procurement, which the authors have mentioned in their studies. Therfore, the red flags or corruption indicators are some of the most important points in the detection of fraud, since the segment is heavily dependent on the prediction itself. Fraud detection isn't a novelty; it is widely used in different areas, such as banking, insurance, company procurement, etc. [40]. The authors state that further work is needed to investigate the ranks of red flags and filter them in a certain way, as well as the interaction between institutions that monitor corruption in public procurement, all with the aim of a more precise corruption estimate [17,23].
To find the most important red flags, different methods are used. The main part of this activity is just a manual job, so to significantly improve the process and make it automated, the authors propose using entity recognition techniques [27]. The aim of the model is, in most cases, to get the best precision. In one study [19], the authors obtained almost ideal results, i.e., metrics around 0.99, and concluded that further research is needed to understand why the results are so good that an analysis can result in some discoveries in the relationship between variables. Also, further analysis is proposed, but in the segment of different types of fraud. Thus, the idea is to include new indicators that will cover the new cases previously ignored as well as the use of optimized algorithms in the parameterization of models [20].
Although the use of advanced data analysis techniques and knowledge elicitation was already identified in the literature review, the clustering technique is proposed to develop corruption risk profiles and to use the "item response theory to extrapolate from observed characteristics to latent corruption risks" [6]. Besides the classification of observations, certain studies have aimed at identifying anomalies [16]. Process mining has proved that procedures with more than one bid do last longer and that some single bid tenders lasted an extremely short period of time [31]. This segment raises the question of further analysis of the detected anomalies by the expert, all with the aim distinguishing whether the results are fraudulent. In addition to data-driven by companies, one of the future ideas is that, instead of analyzing Economic Operators, contracts need to be analyzed, which requires a lot of work in some countries because such contract databases are not related, or they don't exist. All processes at the end are governed by humans, and one of the studies showed that bureaucrats that are less reliant on political connections reduces the risks of corruption [39]. The final state is that the sources of data are rather scarce, which greatly affects the outcome of the classification itself [1,18], while on the other hand there is an opinion that there is a possibility of expanding models focusing precisely on economic operators, but with a risk management process approach to creating government services [21,22].
The digitization of the public procurement process certainly offers fewer opportunities for manipulating the process itself, but it is still necessary to increase the efficiency of the fight against corruption in public procurement by enforcing the law and making better use of government resources [6,41].

V. CONCLUSION
Detection of public procurement corruption in recent years has become one of the major issues around the world. The number of services and amount of money that goes through public procurement is quite large, and for this reason, it is necessary to detect and stop any form of corrupt behavior. Various authors, through various techniques and methods, have been trying to create models that will find any suspicious public procurement process, contract, or economic operator, or classify observations as corrupt or suspicious (Table IV). Of course, this is only one part of the goals that were identified in this paper.
Furthermore, the problem is that there is very little information on PPP that is defined as corrupt, which is a challenge in the techniques that learn from historical data. For this reason, researchers have introduced concepts such as suspicious, bad, or risky PPPs and thus marked the transactions. Data mining and machine learning methods, such as logistic and linear regression, neural networks, process, and text mining, etc. are used in this segment over a large amount of data collected from different data sets, such as contract registers, blacklist economic operators, business registers and so on [1,18,21]. In addition to classification techniques, with the aim of detecting connections between economic operators and contracting authorities, but also for finding daughter companies that participated in collusion of PPP, associations rules and graph databases algorithms were used.
The used metrics are related to the methods, so the mostused metrics in the area of classification or prediction are accuracy, recall, and precision, but unfortunately, this is not the case in all thematic studies, so it is difficult to make a true comparison only with the accuracy metric [4,16,24,26,34]. Moreover, the results obtained vary and depend on the quality www.ijacsa.thesai.org of the data. Much effort has been invested in detecting quality corruption indicators or attributes that have a particular connection to any form of suspect, bad, or risky transactions. Detection takes place at all stages of the public procurement process, from the pre-tender phase to the awarding and postaward phases, but the focus is on using the model as early as possible to prevent a loss of public money or the making of an early warning system. Unfortunately, it has been noted in works that such advanced systems have been integrated into only a small number of state agencies, such as the CGU (Brazilian Office of the Comptroller General) [13,16,21,25,26]. For this reason, the authors point out numerous open issues and suggest combining different methods to improve public procurement processes.
The most effective actions are identification, ranking, and addressing all risks among the ecosystem [5]. Policymakers should perform robust risk assessments, gathering internal input from participants across the ecosystem and across geographies to identify risks and assess mitigating factors. These assessments should also incorporate external factors.
There is a wealth of information available in the public domain, and ignoring it results in a big miss. Risks should be assessed at regular intervals (not through a "one and done" approach). Technology should be backed up with appropriate governance, expertise, and monitoring. One single tool won't address all fraud, and technology alone won't keep the process in place. Technology is often only as good as the expert resources, data management and visibility, robust controls, and regular monitoring dedicated to it. Finally, one of the most important actions is being able to react to fraud once identified. This is critical and is a foundational element of an effective fraud policy. The ability to quickly engage the right combination of people, processes, and technology can limit the potential damage. Disruptive fraud often disguises a strategic inflection point, triggering the opportunity for broader social transformation.