Components and Indicators of Problem-solving Skills in Robot Programming Activities

The objective of this research was to study the components and indicators of problem-solving skills in robot programming activities for high school students. This is done by analyzing the second order of confirmatory factor analysis (CFA) based on data from the behavioral assessment with regard to the robot programming activities of 320 students from specialized science schools. The results of the research revealed that the problem-solving skills in robot programming activities had five components and 15 indicators. All the components were tested for consistency using CFA statistics with the support of R-Studio program. The model analysis results were found to be consistent with empirical data with Chi-Square = 98.273, df = 80.000, pvalue = 0.081, GFI = 0.961, NFI (TLI) = 0.924, CFI = 0.985, RMSEA = 0.027, RMR = 0.007. This indicates that all the identified components and indicators are involved in problemsolving skills in the robot programming activities of high school students. Keywords—Components; indicators; problem solving; robot programming


I. INTRODUCTION
The Organization for Economic Cooperation and Development (OECD) [1] has published the OECD Future of Education and Skills 2030 report, indicating that robotics engineering is the number one task that the world will need. Consequently, inspiring students and encouraging learning with regard to robotics engineering should go along with the critical thinking and problem-solving skills [2] that are components of learning and innovation within the 21st Century Skills [3]. These are widely known in education, and are in line with the views of the World Economic Forum (2016) [4] which has defined critical thinking and problem-solving skills as the core competencies for students in 21st Century. In addition, both of these higher-order thinking skills are essential aspects of fostering skills across all learning and innovation skills groups [5].
In this research we focus on problem-solving skills. By this we mean the capability to use thinking methods based on knowledge and experience, in order to achieve the expected goals. This is done in a step-by-step fashion, by gathering and linking factors and facts [6 -8]. These skills are considered to be the most important and fundamental skills for learners in the 21st Century [3,4]. Therefore, at present, the education profession is making great efforts to devise a learning management approach that can be used to improve problem-solving skills. Since 2018, the Ministry of Education of Thailand [9] has set the standard for developing thinking skills, calculation skills, analytical thinking skills and systematic problem-solving skills at the high school level. Moreover, the Ministry of Science of Thailand [10] has encouraged the organization of robot programming activities for high school students in order to support high school educational standards throughout the country.
Robot programming is an activity used as part of the learning process for the development of many important 21st Century skills [11 -14], especially problem-solving, which are proven to respond well to programming [15 -18]. At the present time, Thailand has no clear standards for assessing problem-solving skills in terms of robotics programming activities for high school students [19]. In addition, the researchers found that the search term "problem solving skill component and indicator on robot programming" does not appear in any Google Scholar databases from 2010 to 2020, nor are there any concrete elements and indicators. Therefore, this research studied the components and indicators of problem-solving skills in programming activities for high school students in Thailand to model prototype guidelines that can be used for student activity design and skill measurement in the future.

1)
To study the components and indicators of problemsolving skills in robot programming activities for high school students.
2) To design the model hypothesis of components and indicators of problem-solving skills in robot programming activities for high school students.
3) To evaluate the validity of the model hypothesis of components and indicators of problem-solving skills in robot programming activities for high school students by analyzing the second order of confirmatory factor analysis.

III. THEORETICAL FRAMEWORK
In conducting this research, the researcher reviewed the literature to synthesize the components and indicators of problem-solving skills in robot programming activities for high school students. This aspect is divided into two main parts: Part I, problem solving skills; Part II, robot programming procedures. The details are as follows: (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 12, No. 9, 2021 133 | P a g e www.ijacsa.thesai.org

A. Part I: Problem-solving Skills
Problem-solving skills refer to a capability to use thinking methods based on knowledge and experience to achieve an expected goal. This is done step-by-step, by gathering and linking factors and facts [6 -8]. Consequently, many scholars have detailed the components of problem-solving skills including Bransford and Stein [20] who defined the problemsolving skill component as "IDEAL" which consists of 1) identify 2) define 3) explore 4) act and 5) look back. This conforms with the work of Foshay and Kirkley [21] who defining the solution components in terms of principles for teaching problem solving to be consistent with 1) identifying the problem 2) defining the problem through thinking about it and sorting out the relevant information 3) exploring solutions through looking at alternatives, brainstorming, and checking out different points of view 4) acting on the strategies 5) looking back and evaluating the effects of the activity. In addition, there are Polya's problem solving techniques [23] that is a generally-accepted problem-solving process in mathematics which consists of four steps to solve a problem. These are as follows: 1) understand the problem 2) devise a plan (translate) 3) carry out the plan (solve) 4) look back (check and interpret). The details with regard to the components of problem-solving skills from many other sources can be synthesized as shown in Table I.
From Table I, which is the result of the synthesis of the components of problem-solving skills gleaned from theories and academic articles, the researcher can conclude that the components of problem-solving skills consist of five elements: 1) Identifying the problem. This refers to explaining the details and the boundaries of the problem, and determining what needs to be solved.
2) Goal setting. This refers to sorting out related information which leads to things which need to be done for the problem to be resolved.
3) Creating a solution. This refers to looking for alternative ways to resolve a particular problem through brainstorming and reviewing to create a problem solution.
4) Acting on the solution. This refers to the implementation of the created solution.
5) Returning to check the results. This refers to the assessment of the results, and evaluating the success in order to ensure the problem has been solved.

B. Part II: Robot Programming Procedures
Robot programming refers to the control of a robot by writing computer programs which can be used to create and instruct the mechanical device using electronic systems to perform a desired task [26 -29]. Nowadays, many scholars have detailed the procedures of programming. For example, Sharma [30] described the Software Development Life Cycle (SDLC) as a process that programmers use to create productivity outcomes. These consist of 1) requirement analysis 2) planning 3) software design 4) software development 5) testing and 6) deployment. These conform to the five-step programming process outlined in Wikibooks [32]: 1) clarifying / defining the problem 2) designing the program, 3) coding the program, 4) testing the program, and 5) documenting and maintaining. In addition, the School of Computer Science at the University of Birmingham [36] provides a simple four step programming procedure: 1) identify the problem 2) design a solution 3) write the program and 4) check the solution. The detail of the programming procedures from many other sources can be synthesized as shown in Table II. Jonassen [8] Great Schools Partnership Aspects [22] POLYA Aspects [23] Astuti, Suranto & Masykuris [24] Franestian, Suyanta &Wiyono [25] Identifying the problem  Table II, the researcher can conclude that synthesizing the robot programming procedures consists of five steps: 1) Identifying the problem. This refers to analyzing the problem and determining the Input and Output processes that need to be incorporated in order to solve the problem.
2) Designing a Solution. This refers to arranging the order of the algorithms used by drawing flowcharts or writing pseudo code.
3) Coding the program. This refers to converting the instructions and sequence of methods from the flowchart to computer language. 4) Testing the program. This refers to the validation of the grammar of the computer language and the interpretation of the results for computer operation purposes. In addition, it involves testing for compatibility with hardware, including the Input and Output Processes. 5) Implementing the program. This refers to the results based on the program used. This should be followed up with further improvements.
From studying the problem-solving skill components and the robot programming procedures as shown above, the researcher can create a conceptual framework with regard to problem-solving skill components, and the indicators in robot programming activities. This framework is as shown in Fig. 1.

A. Population
In this research, the population consisted of students in a group of the Princess Chulabhorn's Science High Schools (one of the Thai governments specialized scientific schools) who had completed a course in "robot programming" in the academic year 2020. This involved three schools in the northern province, three schools in the central province, three schools in the northeastern province, and three schools in the southern province; a total are 12 schools: each of which had 40 students in grade 10, and 40 students in grade 11, a total are 80 students per school. Consequently, the population was 960.

B. Samples
The researcher performed multi-stage sampling, starting with a sampling unit of three schools which was randomized to two schools per province, for a total of eight schools and 640 students. Then, stratified random sampling was used to select 30 students in grade 10 and 30 students in grade 11. As a result, the sample group was 60 students per school, totaling 480 students. The final step was taking a random sample of 40 students per school, and a total net sample of 320 students. This number conformed to the sampling size suggested by Yamane [37] and related to the appropriate sample size for structural equation model (SEM) statistics [38].

C. Research Instrument
In this research, the researcher used behavioral frequency self-assessment in terms of robot programming activities which was validated by seven experts. The assessment design was based on the Likert 5-point Scale to divide the behavior frequency (Level 5 = Always, 4 = Usually, 3 = About half the time, 2 = Seldom and 1 = Never).

D. Research Experts
To validate the research instrument in the form of a behavioral frequency self-assessment instrument in terms of robot programming activities, the researcher worked with seven experts from various fields, whose qualifications were as follows: 1) Two Lecturers in educational evaluation. The research was conducted in five steps as follows: 1) Study of the theories and research related to "Problemsolving skills" and "Robot programming" to identify the components and indicators of problem-solving skills with regard to robotics programming activities to obtain model hypotheses.
2) Creation of behavioral assessment in order to identify problem-solving skills in terms of the robot programming activities. This was based on the components and indicators of problem-solving skills in robot programming activities that followed the researcher's model hypotheses.
3) Evaluation of the validity of the behavior assessment instrument by using the Index of Item-Objective Congruence (IOC) involving seven experts.
4) Collection of behavior assessment data (Sum of the sample made up of 320 students in eight schools.).

5)
Analyzing the data using confirmatory factor analysis (CFA) with the support of R-Studio software to examine the construct validity of the problem-solving skill components and indicators with regard to robot programming activities.

1)
To create and evaluate the validity of the behavioral frequency assessment instrument, seven experts determined the validity of the questionnaire in robot programming activities for high school students using the Index of Item-Objective Congruence (IOC). The results had been shown in the Table III conclude that the statements in the model hypothesis are acceptable. [39].
2) From Table IV, it can be seen that the 5 components have a standardized factor loading (β) greater than 0.4, and consequently passed the standard statistically criteria [40]. In addition, by examining the goodness of it statistical indicator, the Chi-Square value was 98.27 at 80.00 degrees of freedom (df), with a probability (p-value) of 0.08. In addition, Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Normed Fit Index (NFI) and Comparative Fit Index (CFI) were 0.96, 0.94, 0.92 and 0.99, respectively. The Standardized Root Mean Square Residual (SRMR) was 0.04 and the Root Mean Square Error of Approximation (RMSEA) was 0.03. It can be concluded that all 15 components and 5 indicators with regard to problem-solving skills in the robot programming activities fit the empirical data, and correspond to the researcher's hypothesis [41].
3) From Table V and Fig. 2, it can be seen that 15 indicators have a standardized factor loading (β) greater than 0.4. Consequently, they achieved the standard statistically criteria [40]. As a result, these 15 indicators could be appropriate for use as element of all five components.  From the results, the researcher can summarize that the components and indicators with regard to problem-solving skills in robot programming activities for high school students, are composed of five components and fifteen indicators. The details of the supporting information are as follows: 1) Component P3 (Creating a solution) had the highest factor loading at 0.94, followed by component P1 (Identifying the problem) which had a loading factor of 0.88. The third component was P2 (Goals setting) which had a loading factor of 0.86. In addition, the factor loading of the 3 components was more than 0.7, implying to these 3 components have sufficiency variance effected on problem-solving skill [42]. Moreover, the conclusion confirms to the meaning of the phrase "Problem-solving-approach" as defined by the APA Dictionary of Psychology [43]. That is, it is "The process whereby difficulties, obstacles, or stressful events are addressed through the use of coping strategies." Accordance to Jonassen [8], programming activities could be classified as one solution for design problem-solving that focuses on analysis and planning. This also corresponds to Chandrasekaran [44], who states that the key to problemsolving is the critical thinking step, in order for the student to understand the problem, and determine the structure and sequence of work to fit the problem.
2) The factor loading of component P4 (Acting on the solution) and component P5 (Returning to check results) were 0.56 and 0.61, respectively. This means that both of these components are important when it comes to problem-solving skills and is statistically acceptable [40]. Similarly, McFadden [45] demonstrated that programming must follow the plan strictly in order to achieve its goals. In addition, the continuous review and development of the results can increase problem-solving fluency. In line with Taheri, Sasaki and Ngetha [46], we can conclude that problem-solving skills can be built from the programming process, while repeating trials are also a part of the development of problem-solving ability. Moreover, Glenda and Westhuizen [47], also suggest that programming can demonstrate logical reasoning, numerical reasoning and language reasoning, all of which are essential components of problem-solving skills.
3) Indicator P1.1 (Study the problems deeply and in detail), P1.2 (Compare the current problem with other problems that they might have encountered), and P1.3 (Find the key issues relating to the problem) have factor loadings of 0.67, 0.63 and 0.67, respectively. These three indicators are statistically acceptable, which means that all the indicators under component P1 (Identifying the problem) are elementary problem-solving skills [40]. According to Kember [48], identifying the problem means the perception of the problem, and the development of a clear problem framework from digesting information. This leads to effective and accurate problem identification. This in line with Staniewicz [49] who said that problem identification is the first step in coming up with an engineering solution by gathering previously used similar concepts to those of the present problem, then identifying what should be improved. 4) Indicator P2.1 (Group the problem details individually), P2.2 (Specifying the input process and the output of the problem-solving program), and P2.3 (Set goals from the details of the problem) have factor loadings of 0.65, 0.67 and 0.49, respectively. These three indictors are statistically acceptable, which means that all the indicators under component P2 (Goals setting) are elementary problemsolving skills [40]. This conforms with Wharton Executive Education [50], which indicates that a good solution should not stick by a lot of problem information, but should be manage with factor-and-effect relationships to set a new definitive scope for the problem. Moreover, Markman [51] explained that problem resolution targeting involves classifying a large number of existing problems and looking for a correlation in terms of the information to achieve inventive and precise solutions.  [40]. According to Crews & Ziegler [52], writing flowcharts with regard to algorithmic sequencing and pseudo-coding was a process of great importance with regard to engineering solutions. Additionally, Bryant [53] encouraged the use of the Critique, Explore, Compare, and Adapt framework in the process of designing, in that it can give results that are both comprehensive and well-informed. 6) Indicator P4.1 (Write the program conforming to the flowchart by using the algorithm sequence), P4.2 (Use variable names to write programs and program commands that correspond to the pseudo code in the flowchart.), and P4.3 (Execute the program command corresponding to the input and output parts that have been defined) have factor loadings of 0.71, 0.61 and 0.90, respectively. That these three indictors are statistically acceptable means that all the indicators under component P4 (Acting on a solution) are elementary of problem-solving skills [40]. According to Whipp, Tenkanen and Heikinheimo [54], computer programming is a step-bystep aspect of the problem-solving process. Naming each variable and choosing the correct command will support the solution of the problem more easily. It also conforms with Bilotta and Pantano [55], who explained that programming must consider the number and type of sensors which are part of the input, as well as the number and size of motors that are part of the output, to achieve the robot mission satisfactorily. 7) Indicator P5.1 (Check off the program command and structure with the designed flowchart), P5.2 (Run the program by testing in a real environment), and P5.3 (Modify the www.ijacsa.thesai.org flowchart and program commands to achieve the goal completely) have factor loadings of 0.54, 0.73 and 0.64, respectively. That these three indictors are statistically acceptable means that all the indicators under component P5 (Returning to check results) are elementary of problemsolving skills [40]. This conforms with Young, Sharlin and Igarashi [56], who explained that robot control programming needs validation results in terms of the robot's performance in a real environment. The program developer will systematically collect data with regard to the robot's performance in order to improve the robot operation in order to complete the mission. In addition, when solving a robot foot walking problem, Maicon, Aramizo, Houman and Mohammad [57] found that using a robot errors recovery programming technique involving equation generation could reduce the errors in the next operation, and solve the problem more quickly.

A. Integration of the Emerging Model with the Literature
The components and indicators of problem-solving skills were evaluated by confirmatory factor analysis (CFA). It was related to the result of synthesization of problem-solving skills by literature review [20][21][22][23][24][25]. Based on theoretical studies and research articles related to problem-solving skills and robot programming, the researchers were able to summarize the five components of problem-solving skills: 1) identifying the problem, 2) setting goals, 3) creating a solution, 4) acting on the solution and 5) checking the results. In addition, the researchers concluded that the robot programming procedure consists of five steps: 1) identifying the problem, 2) designing a solution, 3) coding the program, 4) testing the program, and 5) implementing the program.

B. Reflection on Methodology and Limitations
Because problem solving skills are explicitly a latent variable, the researchers had to transform latent variables into observable variables by using the behavioral statement in the problem-solving process. In addition, the researchers apply the statement with samples only found in science/research institutions. This point is because data for accurate CFA processing in this research is required only from those expected to have the required skills in robot programming.

C. Suggestions for Future Research
1) This research uses the confirmatory factor analysis (CFA) technique to study the structural validity of the components and indicators. In the next step, creating a standardized skill measurement instrument means that information can be tested for content validity and the accuracy of the questions checked before applying them to each context.
2) In the case of constructivism learning theory, which is popular in education systems dealing with such aspects as problem-based learning (PBL), project-based learning (PjBL) or challenge-based learning (CBL) [58], all the focus is on developing higher order thinking skills. Therefore, the researcher suggests that the five components and fifteen indicators identified in this study could be used in the design of learning and evaluation processes in modern education.