Comparative Analysis of Various Methods Treatment Expert Assessments

The paper deals with the problem of choosing the most effective methods of processing expert information if there are several results of expert evaluation on the problem. The problem of levelling expert assessments, which differ much from the other set of estimates, is considered. Ratios for the weighting factors of individual expert assessments, taking into account the extent of the deviation of each expert's evaluation of the resulting valuation to be obtained from them, are offered. For the problem of estimation of the degree of importance the different components of the computer to ensure the security of data processed in the personal computer, a list of five possible expert data processing methods is formed, and carried out an expert evaluation of the level of the components’ importance on the basis of linguistic variables. Expert estimations are processed by all presented methods. The results of evaluation allowed to identify the most effective methods of treatment; namely median variant of the maximum likelihood method, which is based on a stochastic model of peer review, and proposed in the paper method that takes into account deviations from the specific evaluations of the resulting values. Keywords──Treatment; Linguistic variables; Information processing; Evaluation procedures


INTRODUCTION
The problem of choosing the most effective methods of expert information processing among a large set of options [1] is one of the tasks that have to be solved in the process of conducting the peer evaluation procedures.Below is given an analysis of the solution of this problem for the case of the use of linguistic variables as estimates.As a result, the application object is considered the educational task of assessing the importance of various PC components from the security of data processing point of view.Note that this problem previously has not been considered.Among the closest papers to the work are [2,3].

II. A DESCRIPTION OF THE PROCESSING METHOD IN THE PRESENCE OF EMISSION ESTIMATES
Among the many tasks associated with improving the quality of the resulting estimates obtained by expert procedures, one of the most important is the levelling of the individual (rare) estimates which differ much from the other respectively comparable ones.Call them emission estimates, because these individual "outliers" can significantly affect the resulting estimates.When processing such data, in practice, often are used different approaches: throwing away (ignoring) estimates, having sharp deviation from the rest of assessments; discussion and re-conducting expert procedures with those of the experts, who put down these emission values, outliers other assessments; the use of various coefficients and factors, estimating the level of competence of individual experts.All of these approaches have their drawbacks, and generally, degrade the quality of the result.
Dropping emissions lost part of useful expert information.Also estimates which can be attributed to emissions, are often spread sufficiently and uniformly that does not allow convincingly enough to choose the threshold values below which the assessment is considered acceptable and above must be thrown.During the pre-additional consultations with experts who put emission estimates is often carried out a certain influence on the expert aimed at obtaining from him a reasonable estimate.Finally, the choice of the expert competence coefficients has a strong subjective component and depends on those who form these factors.Also, these factors relate to the expert in general and are not tied to a specific subject matter (object) under-assessment.Below is given a procedure that allows to "weigh" each of the estimates regarding its importance for the resulting estimate.
The proposed approach to solving this problem is the following.The basis of this approach, we rely on the assumption (hypothesis) that the closer evaluation of the expert to the final assessment, the more significant for investigation this estimate.It is supposed to assess the degree of importance of a specific assessment on its "distance‖ from the resulting assessment.For its, a function f() that describes the degree of closeness the evaluation of its expert and the resulting final evaluation is introduced.Then the resulting evaluation is a solution of the equation: Choice the most appropriate proximity function f() about the problem under consideration requires further analysis.In the paper is proposed the simpler version of this function: , and as the distance between u and the average assessment v increases, this coefficient decreases inversely proportionally to u-v.The constant b is chosen based www.ijacsa.thesai.org on the specific requirements of a particular situation; it determines the extent to which the expert opinion is taken into account when evaluating its mismatch with resultant: the low value of b, the more the evaluation of the expert is taken into account, including the emission estimates.Coefficient b also depends on specific features of the problem being solved, and in particular on selection unit of estimation.Its value is assumed to be defined either by an expert procedure or by fixing the degree of importance of evaluating a given value of the deviation from the true value of the test parameter numerically or by testing different versions of its values, comparing the estimates obtained for different values of b, and selecting the minimum value b, for which the degree of consistency expert opinion is acceptable.
Then (1) for the selected function f() can be rewritten as follows: Thus, equation ( 2) can significantly reduce the contribution of the total sum emissions amount when the difference x x i  assumes large values; in this case, the is very small.

III. POSSIBLE METHODS FOR PROCESSING EXPERT DATA
USING LINGUISTIC VARIABLES During the training sessions to the students was posed the following task: to evaluate the degree of vulnerability regarding information security of various personal computer (PC) components.Its solution was carried out on a base of expert procedure that used linguistic variables.In PC the following six basic components were identified: 1) the processor (PR); 2) random access memory (RAM); 3) readonly memory (ROM); 4) input/output devices (IOD); 5) network tools (NT); 6) motherboard (MB).The process of evaluation consisted of the following stages: Stage 1 (Data collection).Each of experts assesses the importance for information security using the scale of the five linguistic assessments.Linguistic evaluation obtained is converted into numeric form.Emission values cannot be obtained by using standard methods of processing based on their conversion scales.
Stage 2 (Getting the expert assessments): Students were divided into five groups -five experts.These linguistic scores were converted to interval ones using Harrington scale [3].These interval assessments are converted to numeric.Namely, the numeric assessments were taken at the middle points of the corresponding interval.As a result, the following Table 1 of numerical estimates was obtained: Stage 3 (Analysis of results): Further processing of data can be done basing on the most common algorithms for constructing the resulting estimates, and also by (2) to conduct a comparative analysis of the results.

The first method:
The average values for all the experts (i.e. the average values for each row) are taken as the resulting estimates for each component.As a result were obtained the following resulting estimates, which are arranged in descending order of assessment of their vulnerability (next to the assessment recorded in brackets resulting assess their vulnerability): IOD (0.755); NT (0.605); RAM (0.555); ROM (0.42); MB (0.305); Pr (0.125).
However, the expert procedure is incomplete because it does not assess the degree of consistency of expert opinions.As the degree of expert opinions consistency assessment measures will choose the most simple method of consistency` assessment, based on the value of variation coefficients , since the amount of data (5 cases) is not sufficient for using methods of mathematical statistics.Here m х is the average value of this indicator expert assessments,  is the value of the sample variance of this estimate.If the calculated value of the coefficient of variation is not more than 0.3, the degree of consensus of experts considered acceptable examination results are accepted as a measure of the vulnerability of the component, and expert assessment procedure of this component is stopped.If the value of the coefficient in the range of (0.3, 0.7), the degree of consensus is the average, and the decision on the admissibility or inadmissibility of the results should be taken by the organisers of the expert procedure.If the value is greater than 0.7 the degree of consensus is low, and the results of the expert procedure cannot be accepted as the assessment of investigated characteristics.Calculating the values of the coefficients of variation for estimates of each component based on the last resulting coefficients of variation values it can be concluded: expert opinion on estimation vulnerabilities of permanent memory, I / O devices and the motherboard are much differing and, the consistency degree is low.Therefore, on these parameters, the expert procedure should be continued.The results of the expert procedure for assessing the vulnerability of the processor, RAM, and network resources are accepted [5].For the rest of the components of an expert, the procedure was continued after collective discussion and justification of their assessments by each of the experts.As a result, the degree of consistency of expert opinion was acceptable, and we arrive at the following final result.All PC components can be arranged in the following series in descending order of assessment of their vulnerability: IOD (0.85); NT (0.605); MB (0.285); RAM (0.555); ROM (0.515); Pr (0.125).
The second method: Each of the five components will be evaluated by the relation (2) for each PC component.Equation ( 2) is solved by using one of the most effective methods for solving algebraic equations -the method of secants.First, consider the problem of estimating the degree of vulnerability of the processor.Let , where x i is the i- . The value of t 0 is fined on base on the following relation (n = 1): As u 3 -u 2 = 0.12643 -0.12643 = 0 < ε, the search procedure is stopped, and as the resulting assessment the value 12643 .0 The third method: It is building by the available set of probabilistic laws that describes the spread of the different expert evaluations.In practice, as such a distribution laws often beta-distribution with density ) ( , x f b a , depending on two parameters a > 0 and b> 0, is used where, is the Euler beta function.The desired estimate is based on the method of maximum likelihood (MML-assessment), or on the basis of the method of least squares (MLS-assessment).For finding MML-assessment for given component the likelihood function is formed for the component: . Using Stirling's formula for the gamma function, we find that L() as   t has the order of t .Therefore, to find the maximum value of the function it is necessary to impose additional restrictions on change range of a and b.It is easy to verify that We obtain the following MML-assessments for vector ( MLS-assessments of the parameters a and b are the solutions of the following system of equations: . In this case, the average of the assessments coincides with the values obtained on the basis of the first method, i.e.The procedure of processing described above can be used for a solution of any problem connected with using expert assessments that are obtained by using linguistic variables.

IV. CONCLUSIONS
1) Estimates derived from the different expert data processing techniques, numerically different, but broadly in line with the basic results of processing, based on the first method.It is because in the case linguistic variables there cannot be emission assessments.The PC components can be placed in the following descending order of assessment of their vulnerability: the I / O device, networking tools, motherboard, random access memory, constant memory (ROM), the processor (Pr).Therefore, the choice of expert estimates the processing method for the majority of cases is not important when linguistic variables are used.
2) Median assessment compared to the average estimates often underestimates low assessments.www.ijacsa.thesai.org 3) In the case of having a computer data processing opportunities it is preferable to use methods for estimating the average, taking into account competencies, and median MMLassessment, which have a better theoretical characteristic (stableness, the rate of convergence).
The results of this work can be used to build a secure PCbased system, taking into account the vulnerability of competencies.In the future, the authors will more thoroughly analyse the processing methods and carry out various expert assessments [6,7].
This work was partially supported by motivational payments system faculty MIREA [8].
evaluation expert estimation for Pr, the result; u k is the auxiliary point on the k-th search step.find f(t 0 ) and f(t 1 ): 00655 the variables a and b is unbounded, what can prove by examining the order of the function () L at infinity along the direction one, as an additional restriction we can require that the variation of the function dispersion of not more than two orders of magnitude; it is sufficient to impose the condition c b a   , c = 100.For the end result this restriction is not important, since in   с both average and median estimates of vulnerability tend to some limit.Under this additional constraint we will calculate the maximum value of the function () L .www.ijacsa.thesai.orgLet a 0 and b 0 be those values of a and b, at which the maximum value of the function () L is achieved.Then, as the resulting assessment is taken the value of the average

table ,
On the basis of the www.ijacsa.thesai.org

TABLE II .
TABLE OF RESULTS