Dissemination and Implementation of THK-ANEKA and SAW-Based Stake Model Evaluation Website

The purpose of this study was to provide information about the dissemination and implementation of the THK-ANEKA and SAW-based Stake model evaluation website at Vocational Schools of IT in Bali. THK is an acronym for Tri Hita Karana. ANEKA is an acronym for Akuntabilitas, Nasionalisme, Etika publik, Komitmen mutu, dan Anti korupsi (in Indonesian) or Accountability, Nationalism, Public ethics, Quality commitment, and Anti-corruption (in English). SAW is an acronym for Simple Additive Weighting. This study used a development approach with the Borg and Gall model which consists of 10 development stages. Research in 2020 was focused on the dissemination and implementation stages. The research location was at several Vocational Schools of IT in Bali Province. The subjects involved in assessing website implementation were 110 respondents. The tool used to assess was a questionnaire. The analysis technique was carried out by interpreting the effectiveness level of dissemination and implementation. It was a reference to the eleven scale effectiveness standard. The research results showed that the dissemination and implementation of the THK-ANEKA and SAW-based Stake model evaluation website at Vocational Schools of IT in Bali had gone well. It was able to be seen from the documentary evidence of the dissemination activities implementation. The percentage results of the website implementing effectiveness were 88.973% and the simulation results of implementing the SAW method which was already accurate. It showed the evaluation aspects that support the realization of positive morals and students’ learning quality. Keywords—Evaluation website; stake model; THK; ANEKA; SAW


I. INTRODUCTION
Evaluation activities are very important to do to determine the effectiveness of computer learning implementation at the Vocational Schools of IT. Several evaluation models that can be used to evaluate the computer learning implementation include: CIPP [1,2]; CSE-UCLA [3]; Formative-Summatif [4]; Discrepancy [5]; and Countenance [6]. However, not all of these models can produce accurate recommendations. The expected accurate recommendations are related to aspects that support positive moral improvement and the students" learning quality in the computer learning process. One effort that can be made to obtain these accurate recommendations is to present a web-based evaluation application. This web-based evaluation application can integrate the Stake evaluation model with the THK concept, the ANEKA concept, and the SAW method.
The Stake evaluation model [7][8][9][10][11] is one of the evaluation models used to provide recommendations based on a description and judgment matrix. The THK (Tri Hita Karana) concept is one of the Balinese local wisdom that teaches people to recognize the three causes of happiness. The three causes of happiness [12][13][14], included: Parahyangan (good relationship with God), Pawongan (good relationship with fellow human beings), and Palemahan (good relationship with nature and the environment). ANEKA is a concept that teaches internalizing the values of a positive attitude and selfquality that must be possessed by a civil servant in Indonesia. It is as a foundation for carrying out his/her professionalism as a good servant of the country. ANEKA consist of several components [15,16], included: accountability, nationalism, public ethics, quality commitment, and anti-corruption. SAW (Simple Additive Weighting) is one of the methods in the Multi Criteria Decision Making (MCDM) [17][18][19][20], which is how it works to determine the assessment score based on the multiplication results of each alternative with the decisionmaker weight.
Aspects of the Stake model were used as the basic criteria for measurement in evaluating the computer learning process at Vocational Schools of IT. ANEKA components were internalized into the description matrix which contained in the Stake model. The aim was to ensure the positive attitude and students" learning quality in the computer learning process had been in accordance with the context, process, and impact variables in the description matrix. THK components were internalized into a judgment matrix in the Stake model with the aim of being used as a main basic in determining recommendations. The SAW method was used to determine the dominant aspects that need to be encouraged to realize students" learning quality and positive moral improvement.
The THK-ANEKA and SAW-based Stake model evaluation website can be said to run optimally if it has been disseminated and implemented. Therefore, it is necessary to conduct the dissemination and implementation of this website on a wider scale. Based on these, then the right question for this research was "What are the dissemination and implementation results of the THK-ANEKA and SAW-based Stake model evaluation website at Vocational Schools of IT (case study in Bali Province)?" Several previous studies had provided a stimulus and effect for the realization of this research. It was like the research conducted in 2018 by Ihsan and Furnham [21], which showed the existence of several technologies that can be used as a source for assessing personality. Some of the technologies referred to included: social media, wearable technology, mobile phone, gamification, video resume, and automated personality testing. The limitation of Ihsan and Furnham"s research was that it only introduced some of the technologies used for personality assessment, but it had not yet explained in detail how the technology works. Besides, Ihsan and Furnham"s research only focuses on personality assessments based only on the affective domain and it had not based on cognitive and psychomotor domains. Research was conducted in 2017 by Boitshwarelo, Reedy and Billany [22] demonstrated the use of online tests to measure 21 st century learning outcomes. The limitation of Boitshwarelo, Reedy and Billany"s research was that it had not been discussed in detail about measuring learning outcomes in the affective and psychomotor domains. Their research only focuses on the cognitive domain as measured by using an online test. Research was conducted in 2018 by Kyllonen and Kell [23] showed a test measuring cognitive ability and personality measurement. Measurement of cognitive abilities was measured using cognitive tests, such as multiple choice and essays. Personality measurement used attitude scale questionnaires. The limitation of Kyllonen and Kell"s research was that it had not shown any measurement in the psychomotor domain. Research in 2015 by Mariš [24] showed that there were character measurements based on the individual character dimension scores. The limitation of Mariš"s research was that it had not been shown the measurement of cognitive and psychomotor abilities in individuals. Research in 2018 by Elmahdi, Al-Hattami, and Fawzi [25] showed a formative assessment of the student learning process used Plickers technology. The limitation of the research of Elmahdi, Al-Hattami, and Fawzi was that it had not specifically shown any assessment in the affective and psychomotor domains, because they focus on cognitive assessments. Research in 2018 was conducted by Daniawan [26] showed the use of the SAW method in evaluating lecturer performance in teaching. The similarity between Daniawan"s research and this research was that both of them apply the SAW method in making decisions. Daniawan"s research limitation was that it had not shown specific criteria for measuring the cognitive domain. Daniawan only focused on showing ten criteria in the teaching process which focused more on the affective and psychomotor domains.
Based on the research question and previous research that had provided a stimulus, then the authors were interested in conducting more in-depth research. It was related to dissemination activities and the implementation of the THK-ANEKA and SAW-based Stake evaluation website at several Vocational Schools of IT in Bali Province.

II. METHOD
This research was development research that had carried out from 2018 to 2020. The model used in this development research was Borg and Gall [27][28][29] which consists of 10 stages of development. Five stages which were carried out in 2018, included: 1) research and field data collection, 2) research planning, 3) design development, 4) preliminary field test, 5) the main product revision. Two stages which had carried out in 2019, included: 1) main field test and 2) operational product revision. Three stages which had carried out in 2020, included: 1) operational field testing, 2) final product revision, 3) dissemination and implementation of the final product.
Based on the research questions previously disclosed, so the discussion in this paper focused on the dissemination and implementation stages of the final product. There were 110 respondents involved in the dissemination and implementation stage of the THK-ANEKA and SAW-based Stake model evaluation website. The 110 respondents consist of 80 students and 30 teachers from Vocational Schools of IT in Bali Province.
The tool used to obtain quantitative data in dissemination and evaluation website implementation was the questionnaires. The research location was carried out in several Vocational Schools of IT in 6 regencies on Bali Province, included: Tabanan, Buleleng, Klungkung, Gianyar, Denpasar, and Badung. The analysis technique used in this research was descriptive quantitative by interpreting the results of the effectiveness level from dissemination and implementation. It was based on the effectiveness standard which refers to the eleven"s scale. The formula used to determine the effectiveness level of dissemination and implementation can be seen in equation (1) [30,31], while the standard of effectiveness which refers to the eleven"s scale [32] can be seen in Table I.  Before showing the implementation results of the THK-ANEKA and SAW-based Stake model evaluation website, it was necessary to carry out dissemination activities to users. Dissemination activities were carried out by holding online workshops through zoom media and direct assistance to schools. The workshop and mentoring activities can be seen in Fig. 1. Details of the material provided in the dissemination activities can be seen in Table II.  The successful implementation of the THK-ANEKA and SAW-based Stake model evaluation website at several Vocational Schools of IT in Bali was able to be obtained from the assessment results of 110 respondents (30 teachers and 80 students). The assessment results of all respondents can be seen in Table III. The assessment activities documentation of evaluation website implementation can be seen in Fig. 2.
The successful implementation evidence of the evaluation website also was obtained from the results of SAW method calculation accuracy in addition to the assessment results from the 110 respondents. The SAW calculation process can be carried out if simulation data are provided (can be seen in Table IV) and the weight of decision-makers (can be seen in Table V). Based on the simulation data shown in Table IV and determining that all ANEKA components are included in the benefit attribute, the normalization calculation process can be carried out. The formula used for normalization calculations [17] refers to equation (2).  Based on the normalization results, then the conversion was carried out into matrix-R. The display of matrix-R can be seen in Fig. 3 Based on the matrix-R and the weight from decisionmakers shown in Table V, the ranking calculations can be performed. The formula used to calculate ranking [18]  Based on the ranking results, it can be determined the most dominant aspect recommendations in supporting the realization of positive moral improvement and student learning quality. The aspect referred to is C-10, namely the aspect of "it is always obey the school rules". This aspect was chosen because it had the highest compared to other aspects. The C-10 aspect is an aspect of the Palemahan component.
The dissemination activities that had been shown previously in Fig. 1 were carried out through two activities. The first activity was an online workshop on 11 materials related to the operation and management of THK-ANEKA and the SAW-based Stake model evaluation website. The second activity was assistance related to matters that were not clearly understood in the online workshop. It was discussed in-depth and directly through face to face at school.
Implementation of the THK-ANEKA and SAW-based Stake model evaluation website had been carried out well generally. The Evaluation website categorization had been classified as good and effective to determine appropriate and accurate recommendations. This recommendation was related to the www.ijacsa.thesai.org supporting aspects of increasing positive morale and student learning quality in computer learning at Vocational Schools of IT in Bali. It was reinforced from the effectiveness percentage results in the evaluation website implementing was 88.973%. When it is viewed from the effectiveness standard of the eleven"s scale, it is classified in the good category because the percentage is in the range of 85% -94%.
The effectiveness percentage results were obtained from the respondent"s assessment data on the website implementation by using a questionnaire containing 20 questions. Item-1 was about ease of website installation. Item-2 was about the website appearance. Item-3 was about the consistency of each layout form. Item-4 was about the suitability and accuracy of the login design. Item-5 was about the suitability and completeness of the features available on the main menu.
Item-6 was about the suitability and completeness from the features available on the input form of indicator and weight. Item-7 was about the suitability and completeness of the features available on the input form of evaluation aspect assessment data provided by the respondents. Item-8 was about the suitability and completeness of the features available on the evaluator data input form. Item-9 was about the suitability and completeness of the features available on the antecedent form located in the description matrix. Item-10 was about the suitability and completeness of the features available in the transaction form which was located in the description matrix. Item-11 was about the suitability and completeness of the features available in the form outcomes which were located in the description matrix.
Item-12 was about the suitability and completeness of the features available in the judgment matrix form had referred to the Tri Hita Karana and ANEKA aspects. Item-13 was about the suitability and completeness of the features available in the recommendation and decision form. Item-14 was about the suitability of evaluation aspects in the accountability section in the description matrix form. Item-15 was about the evaluation aspects suitability of the nationalism section in the description matrix form.
Item-16 was about the evaluation aspects suitability of the public ethics section in the description matrix form. Item-17 was about the evaluation aspects suitability of the quality commitment section in the description matrix form. Item-18 was about the evaluation aspects suitability of the anticorruption section in the description matrix form. Item-19 was about features that make it easy to store data, edit, update, and delete. Item-20 was about the website accuracy in calculating the SAW method and showed the right recommendations.
This research had succeeded in being a solution to the limitations of Ihsan and Furnham"s research [21]; Boitshwarelo, Reedy, and Billany"s research [22]; Kyllonen and Kell"s research [23]; Mariš"s research [24]; and Elmahdi, Al-Hattami, and Fawzi"s research [25]. The solution was the Stake model evaluation website implementation at Vocational Schools of IT in Bali. It was able to show an assessment of the affective domain through internalizing the Tri Hita Karana concept, cognitive and psychomotor assessments through internalizing the ANEKA concept. It was reinforced by the research results of Divayana, Sudirtha, and Gading [33]. They showed that there was a Countenance evaluation model application design that was integrated with the Tri Hita Karana and ANEKA concept. It is used to measure the character aspects so the cognitive and psychomotor aspects of students in computer learning.
Another research result [34] that strengthens the position of this study is the research of Assielou et al. It showed that emotion (affective domain) can affect student performance (cognitive and psychomotor domains) in the learning process using Intelligent Tutoring Systems. The research conducted by Sokkhey and Okazaki [35] also strengthens the position of this study by showing the existence of a website-based decision support system. It was used to predict poor student performance in the learning process. The principle was the same with this research which also developed a website to evaluate student performance as a whole both from the moral side (affective domain) and from the learning quality side (cognitive and psychomotor aspects).
Although this research had succeeded in being a solution to the limitations found in the five previous studies, this research also has several limitations. The limitations of this research are: 1) The THK-ANEKA and SAW based-Stake model evaluation website has not been implemented at Vocational Schools of IT in all Indonesia regions; 2) This evaluation website has not been combined with robot technology so that the input activity indicators and evaluation weights are still done manually by evaluators or decisionmakers.

IV. CONCLUSION
Generally, dissemination and implementation results of the THK-ANEKA and SAW based-Stake model evaluation website had been carried out well at Vocational Schools of IT in Bali Province. It was evident from the results of documentation in dissemination and implementation. The effectiveness percentage result of 88.973%, which is in the good category at the eleven"s scale effectiveness standards indicated the success of evaluation website implementation. Likewise, the application simulation results of the SAW method in determining the dominant aspects of realizing positive moral improvement and student learning quality. Those had also proven the success of this evaluation website implementation. This research obstacle can be answered by doing the right work in the future. Some future work that can be done, included: 1) Dissemination and further implementation of evaluation website to several Vocational Schools of IT in western and eastern parts of Indonesia; 2) Development of evaluation website in the future is embedded in robotic technology so that the website will be more reliable in processing decision-making.