Selection of Learning Apps to Promote Critical Thinking in Programming Students using Fuzzy TOPSIS

The aim of this research was to use intelligent decision support systems to obtain student-centred preferences for learning applications to promote critical thinking in first year programming students. This study focuses on the visual programming environment and critical thinking as the gateway skill for student success in understanding programming. Twentyfive critical thinking criteria were synthesized from the literature. As a quantitative study, 217 randomly selected students from an approximate target population of 500 programming students to rate four learning Apps, namely, Scratch, Alice, Blockly and MIT App Inventor, against critical thinking criteria to establish the App that best promotes critical thinking among first year programming students. There were 175 responses received from the 217 randomly chosen programming students who willingly contributed to the study. Consequently, the distinctiveness of this paper lies in its use of the Fuzzy TOPSIS (Technique for Order Preference by Similarity to Ideal Situation) multi-criteria decision-making algorithm to rank criteria for critical thinking, calculate their weights on the basis of informed opinion and hence scientifically deduce the best rated App among the available alternatives that promote critical thinking among first year programming students. The results showed that Scratch promoted critical thinking skills the best in first year programming students whilst Blockly promoted critical thinking skills the least. As a contribution to the study, policy-makers and academic staff can be potentially supported to make informed decisions about the types of learning Apps to consider for students when confronted with multiple selection criteria. Keywords—Critical thinking; visual programming environment; multi-criteria decision-making; fuzzy TOPSIS


I. INTRODUCTION
The problem associated with teaching programming to novice learners is exacerbated by the complex and abstract nature of the field and the heavy reliance on 21st century skills such as critical and computational thinking. As a result, a kaleidoscope of research into programming self-efficacy, the complexity of the field, teaching methods and a variety of teaching tools, have emerged over the recent past. Furthermore, learning to program can be perceived as both tedious and difficult to the novice programmer for reasons such as the abstract nature of computational thinking and the burdensome syntax and semantics associated with many programming languages [1][2][3]. The literature has also repeatedly revealed that visual block-based programming environments have a positively powerful impact on the performances of novices learning to program [4]. In a case study performed by Pinto-Llorente et al. in [5] on the impact of the visual programming environment (VPE) among primary school learners, it was determined to have promoted their critical thinking skills and problem solving abilities. Various other studies have revealed that visual programming environments support the constructivist approach to learning and contribute to enhancing one's skills of independent learning, creative thinking, problem-solving, reflection and collaboration [6][7][8][9][10][11][12]. This has resulted in an upsurge of educational programming environments and tools aimed at stimulating the learners' interest, making programming accessible to people of all interests and ages and minimizing the challenges of understanding programming [13].
The significant growth in diverse learning Apps in the last decade and the varying commercial aspirations these Apps may be designed to serve, have impacted on the need for university academics to make rigorous and well-considered decisions around their choice of learning App. Further, an online learning App becomes especially appropriate post Covid-19, where higher education institutions are strategically searching for alternate teaching methodologies to the traditional teaching approach. Interestingly, a number of learning Apps originated over the years with a variety of different characteristics, each serving a slightly different need [4]. Multi-criteria group decision-making (MCDM) entails weighing each of several alternatives against multiple, conflicting, quantitative decision criteria executed by many decision-makers to derive the optimal alternative [14][15][16]. The various criteria will be synthesized from an in-depth analysis of the literature in the relevant field.
According to Başaran and Haruna in [17], manually selecting these Apps can be biased, tedious, inconsistent and time-consuming, ultimately resulting in a premature selection. Therefore, to address ambiguity and uncertainty associated with human judgements, this quantitative study will use the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), a scientific and mathematical technique in multi-criteria decision-making (MCDM) that preserves integrity and objectivity of process [16,18]. The TOPSIS method, developed in 1981 by Yoon and Hwang, was grounded on the fundamental principle that the best option from a range of alternatives will be positioned closest to the www.ijacsa.thesai.org positive ideal solution and farthest from the negative ideal solution [19].
In a study conducted by Junior, Osiro and Carpinetti in [39], a comparison of the Fuzzy AHP and Fuzzy TOPSIS methods was undertaken to choose from a number of suppliers. The criteria central to the comparison were: "adequacy to changes of alternatives or criteria; agility in the decision process; computational complexity; adequacy to supporting group decision-making; the number of alternatives and criteria, and modelling of uncertainty". The study revealed that Fuzzy TOPSIS provided more rigor in the decision process, lower time complexity when there was a large number of decisionmakers, offered no restrictions on the number of criteria and alternatives and became the preferred choice when there were changes to alternatives or criteria as rank reversal posed a potential problem with Fuzzy AHP.
Fuzzy set theory, which was introduced by Zadeh in 1965, can be used to manage the uncertainties and subjectivity of the evaluation process [15]. More specifically, this study will implement an expansion of the TOPSIS method to accommodate fuzziness in the decision problem setting. Fuzzy TOPSIS is widely and extensively applied among researchers in diverse fields of economics, business management, engineering, ICT and education, to solve various MCDM problems associated with site selection in mining and engineering, equipment selection, risk analysis for complex infrastructure projects, portfolio selection, selection of e-Learning approaches, supplier selection and software selection [16][17][18][19][20][21][22][23]. The advantages of Fuzzy TOPSIS include its ease of implementation, capacity to compare the strongest and weakest alternatives quantitatively, that it enables linguistic expressions to be represented as fuzzy numbers, and its practicality and ability to handle incomplete or partial quantitative data [24]. A questionnaire disseminated to 217 first year programming students is used as a tool to gather the ratings of alternatives against multiple selection criteria, and the different weightings per criteria are obtained from academic experts in the field. These sets of evaluations use logical interval judgements represented by linguistic terms which can be parameterized by triangular fuzzy numbers [24].
Choosing an optimal learning application (App) is therefore a complex problem requiring an intelligent decision support system that will use multiple decision-makers to simultaneously ruminate multiple criteria such as the prior skills of the learner, the technical infrastructure of the learning environment and the 21st century skills required for success in the higher order thinking of computer programming [25]. There are no studies in the extant literature that use an intelligent decision support system that is able to find the optimal choice of learning App to suit the needs of the instructors and learners in acquiring improved student success not only to encourage their interest and excitement for programming, but also to acquire a proficiency to create their own applications. The aim of this paper is to use the fuzzy TOPSIS multi-criteria decision-making technique to rank learning Apps according to its potential to promote critical thinking among programming students. The paper comprises five sections, namely, section 1 introduces the topic, section two surveys the extant literature, section three presents the fuzzy TOPSIS method, section four explains the results and section five culminates with the findings of the study.

II. LITERATURE REVIEW
The definition and promotion of critical thinking for learning has become increasingly important in the 21st century information and digital age, where organizations are competing for applicants who have the ability to investigate, apply, and transform data, to collaborate and innovate, to reflect, analyze and think critically, and to arrive at optimal decisions [26]. The literature defines the higher order thinking processes associated with programming as closely aligned with the concept of computational thinking (CT), where problem-solving is implemented through the mental process of abstraction, logic, analysis, synthesis and constructive thinking [5,[27][28]. It has been justified through various studies that CT skills should not just be accessible to everyone at a much younger age, but also have the potential to be applied to various disciplines and hence to be incorporated into the basic curriculum [2,5].
There are a variety of learning resources and learning Apps to develop CT [29,30] and each of these include aspects of robotics [2,4,[31][32], gaming [33][34], augmented reality simulation [35] or multimedia such as graphics, sound and animation, with the latter referred to as visual programing environments [4,9,28]. This study is significant in its application as it aims to extend multi-criteria decision-making using Fuzzy TOPSIS with multiple decision-makers choosing the most appropriate learning App based on well-synthesized critical thinking criteria.
Multi-criteria decision analysis (MCDA) has successfully been applied to solve a wide range of complex real-life decision problems in a variety of fields [36], despite researchers encountering many challenges associated with partial ignorance and unquantifiable, incomplete, unobtainable, uncertain, ambiguous and vague information [15]. The choice of MCDA technique rests predominantly on the nature of the decision problem and its ability to address many of the aforementioned challenges. The fuzzy approach is best suited to scenarios that are prone to ambiguity, subjectivity and uncertainty when the decision-maker is expected to make an evaluation [18,37]. Furthermore, the TOPSIS method suggested by Hwang and Yoon in 1981, is a popular method that can rank the best alternatives quickly, address conflicting situations, is easy to use and can be integrated with other decision-making methods [18]. The fuzzy set theory combined with TOPSIS has the added advantage of addressing any uncertainty encompassing the assessment process [15].
Several refined MCDA methods have been introduced over the years. Some of the popular methods include the analytic hierarchy process (AHP) requiring many comparisons based on human preferences, with the possibility of compromising the consistency of the decisions when compared to the best-worst method developed to reduce the number of comparisons in traditional AHP; the TOPSIS method and simple additive weighting (SAW), which depend on mathematical operations, and the outranking models like ELECTRE and PROMETHEE, which are premised on the assumption that the decision is a process whereby decision-makers can alter their preferences after thorough reflection. The hierarchical model of the AHP www.ijacsa.thesai.org method requires the decision-maker to pairwise-compare multiple criteria and uses ratio and semantic scales to arrive at the decision-maker's preference. Memari et al. in [21] compared the TOPSIS method with two outranking methods namely, PROMETHEE and ELECTRE. According to the authors, TOPSIS is easy to learn and apply compared to outranking methods which proved to be more complex and less transparent to decision-makers. Sahin et al. in [38] included TOPSIS in their study due to its unique approach to the problem and it's intuitively appealing and easy to understand qualities. Palczewski and Sałabun in [36] further highlighted simplicity, computational efficiency and comprehensive mathematical concept as well as its support for group decisionmaking as key contributing factors to the popularity of the Fuzzy TOPSIS technique.
In the learning App selection problem, the multi-criteria decision-making approach to be applied needs to be carefully considered. The number of decision-makers, the various types of measures to promote critical thinking, the method used to weight the various criteria and the degree of uncertainty and vagueness encompassing the problem scenario are some of the qualifying factors. This study involves a sample of 217 first year programming students and questions on the performance of each alternative against multiple critical thinking criteria, some more easily measurable than others. Nursal, Omar and Nawi in [14], identify the TOPSIS method with assigning crisp or exact numerical values when rating alternatives against criteria and when rating criteria importance, which are also inadequate to simulate human judgements in real-life scenarios. Therefore, to accommodate the imprecise or vague nature of the group decision-making assessment pertaining to critical thinking criteria, this study will use a more authentic approach by enabling the use of linguistic variables to capture the decision-maker's rating more accurately and consistently [40]. Hence the Fuzzy TOPSIS approach is being more suitably applied to model the linguistic ratings as a fuzzy triplet (a, b, c) because the most promising value b will better capture the rating compared to parameter a representing the smallest possible value and parameter c representing the largest possible value in a fuzzy event [40,41]. This becomes especially important in the group decision-making process.
The research has demonstrated various MCDM problems being solved through the application of fuzzy logic. These include facility location selection, machine tool selection problem, plant layout design problem and robot selection [42] to equipment selection [16] and supplier selection [21,36,40]. Although this study uses the quantitative approach, it also identifies some subjective criteria that are difficult to measure, hence the need for fuzzy sets which are capable of representing vague data [41]. According to Kahraman et al. in [42], the strength of fuzzy logic lies in its ability to imitate human reasoning capabilities during the cognitive process of decisionmaking, to be captured mathematically and more precisely, resulting in a better expected performance in this case scenario.
In everyday pragmatic circumstances which are frequently unpredictable, it is often inadequate to describe phenomena in crisp and precise terms. Therefore, fuzziness occurs in many areas of human judgement, reasoning, evaluations and decision-making. In this study, for example, the evaluation of learning Apps against various subjective critical thinking criteria are more easily articulated in natural language terms such as very good, poor or satisfactory, which may affect vagueness and ambiguity. However, fuzzy set theory, introduced by Zadeh (1965), offers a rigorous mathematical framework to efficiently resolve the indistinctness associated with the subjectivity of human judgements, and in which vagueness and uncertainty can be precisely and rigorously studied [43].
Rashidi and Cullinane in [40] highlighted fuzzy sets and fuzzy logic (FL) as "powerful mathematical tools for modelling uncertain systems in industry", while de Barros et al. [44] define fuzzy logic as a "multivalued logic, that uses intermediary values between conventional evaluations such as true/false, yes/no or high/low". According to de Barros et al. [44], the language of linguistics is helpful in describing uncertain and ill-defined qualitative data, which otherwise cannot be subjected to quantitative analysis. These linguistic variables can then be represented as computational-efficient fuzzy-triples for appropriately quantifying vague information [37].
According to Samaie et al. in [37], elements in a fuzzy set have different degrees of membership, designated by membership functions, which allocates a grade of membership ranging between 0 and 1. In applications that use fuzzy techniques the decision problem can have complex mathematical membership functions such as triangular or trapezoidal. According to Rajak and Shaw in [18], modelling the decision problem using triangular fuzzy numbers yields a better result. The graphical representation of a membership function F(x) of a triangular fuzzy number Ã, is illustrated in Fig. 1 where the x axis represents the universe of discourse and the y axis represents the degrees of membership in the [0,1] interval [37,40]. A triangular fuzzy number is defined in Fig. 1 and expressed as a membership function in Fig. 2.  High (H) Good (G) (5,7,9) Very high (VH) Very good (VG) (7,9,9) In fuzzy set theory, linguistic terms are converted into fuzzy triples using conversion scales. Usually, a scale of 1 to 9 is applied for weighting the criteria and ranking the alternatives, as illustrated in Table I. Uniform intervals are used to represent the fuzzy triangular numbers for each of the five linguistic ratings [23].

III. METHODOLOGY
In this research paper fuzzy set theory is introduced to model ambiguity and uncertainty in a MCDM problem and integrated with TOPSIS to appreciate the benefits of its practicality and ease of use, enabling evaluations to be expressed in a linguistic language and then converted to triangular fuzzy numbers and implementing the algorithm using a software tool.
Assume the decision problem has k decision-makers (D 1 , D 2 D k ), with m possible alternatives (A 1 , A 2 A m ), which is evaluated against n criteria ( 1 , 2 … n ). The rating of criteria and weight with respect to each criterion can be accurately represented in the form of matrices for each decision-maker.
Step 2: Aggregate the evaluation of the criteria and alternatives. fuzzy ratings and is described as triangular fuzzy numbers ( where K = 1,2,…, k, then the aggregated importance can be evaluated as: Step 3: Normalize triangular fuzzy numbers. The raw data is normalized using linear scale transformation to bring the various criteria scales into a comparable scale. If represents benefit criteria, then: If represents cost criteria, then: Step 4: Compute weighted normalized fuzzy values. becomes after normalization, is new after aggregation. Let the weighted normalized value be . (5) where N = 1,2 … n; K = 1,2 … k; M = 1,2 … m; The corresponding triangular fuzzy number of is Step 5: Calculate fuzzy positive ideal solutions (FPIS) and fuzzy negative ideal solutions (FNIS); Step 6: Calculate the distance of each alternative from ( and ( which is calculated, respectively, as follows: Step 7: Calculate the closeness coefficient ( The CC M value is then used to determine the ranking order of all alternatives for the purpose of selecting the best one from among a set of feasible alternatives.

IV. RESULT
In this section, the results produced from the Fuzzy TOPSIS software application implemented on MatLab R2020a is presented with explanations in steps according to input and outputs of the application. This application utilized complex mathematical equations on the backend which corresponds with the Fuzzy TOPSIS method presented in the methodology www.ijacsa.thesai.org section. The results are based on preferences shown by decision-makers (first year programming students) on choice of visual programming learning applications based on set critical thinking criteria. Table II shows the fuzzy rating scale where linguistic terms, namely, Not Important (N), Slightly Important (SI), Moderately Important (MI), Important (I) and Very Important (VI) are expressed as a fuzzy triple using integers in the range 1 to 9. These linguistic terms are used to rate the criteria.   (1,3,5) Moderately Important (MI) (3,5,7) Important (I) (5,7,9) Very Important (VI) (7,9,9) I  V  I   M  I   V  I   M  I   M  I  I  I  V  I  VI  VI  I  MI VI  VI  VI  MI SI  SI  MI I  I  VI  I  I   D2  I  V  I  I  M  I  I  SI I  M  I   V  I  VI  VI  I  MI I  I  VI  I  MI MI I  MI MI MI I  I   D3  V  I   V  I  I  V  I  I  I  M  I  I  I  VI  VI  MI MI I  I  I  MI SI  SI  SI  SI  SI  I  I  I   D4  I  I  I  I  I  I  M  I   V  I   V  I  VI  VI  I  MI VI  VI  VI  MI MI MI MI MI MI I  I  VI   D5  V  I   V  I  I  M  I   V  I   V  I  I  V  I   V  I  VI  I  I  MI I  I  VI  I  MI I  VI  I  VI  VI  VI Table IV shows the linguistic rating of the 25 criteria by 10 Academic Experts teaching programming at university level.
Their input values are represented by the symbols D1 to D10 in the table below.
The Best Non-fuzzy Performance value (BNP) for a criterion weighting j, can be calculated using the following equation [13]: The BNP values give an indication of the relative importance of the criteria. Table V shows the aggregated score for each criterion.
Table V also shows the calculated BNP value for each criteria using Equation 12. Linguistic Weights were assigned based on the BNP values according to descriptions given in Table II. Table VI shows the ranking of the criteria based on the BNP values indicated in Table V. Table VI shows that C10 (Logic and reasoning: The App quickly alerts me when the sequencing of steps in my solution is logically incorrect) was the most highly ranked criteria by the expert decision-makers. The second most highly recommended criterion is C9 (Logic: The App helped me to improve my logic skills), followed by C16 (Application: The App allows me to apply the concepts of sequence, selection and iteration) and C14 (Alternate solutions: The App helps me to think about solving the problem in different ways). The least important criterion to decision-makers is criterion C18 (Multimedia: The interface is visually rich in multimedia and includes sound, color, graphics and animation).   Table VII shows the fuzzy rating scale for weighing critical thinking criteria. Linguistic terms such as Very Poor (VP), Poor (P), Satisfactory (S), Good (G) and Very Good (VG) are expressed as a fuzzy triple with integers in the range 1 to 9. The Not Sure (N) option expressed as (0, 0, 0) is included in the application to cater for students who are not sure what rating to assign to criteria.   Good (G) (5,7,9) Very Good (VG) (7,9,9)      The CCi value is a ratio scale rating showing the ranking of learning App by 175 decision-makers from highest to lowest, CCi (Scratch) > CCi (Alice)> CCi (MIT Inventor) > CCi (Blockly).
The results show that Scratch is the most preferred application for learning programming as it had the highest CCi value. This is congruent with various studies that compared visual programming environments using different methods under different contexts and found Scratch to be the most preferred learning tool [4,11,45]. The App with the lowest CCi value is Blockly, which therefore is the least preferred application. www.ijacsa.thesai.org

V. CONCLUSION
The study used a population of 500 and a sample population of 217 decision-makers to rate four learning Apps against each of 25 critical thinking criteria. Applying the Fuzzy TOPSIS MCDM method to the decision problem would have made handling the matrices both inconvenient and cumbersome. The researcher created code using MatLab R2020a to automate the process. The implementation of the coding for the Fuzzy TOPSIS MCDM method resulted in the ranking of four learning Apps, from the App that promoted critical thinking the best to the one that promoted it least among 1st year programming learners; these Apps were Scratch, Alice, MIT App Inventor and Blockly. The study is significant for lecturers teaching introductory programming modules to novice learners across educational institutions, especially during the global Covid-19 pandemic, when online learning has become commonplace and its teaching tools have become a requirement. Although the research suggests the positive impact of the visual programming environment on one's critical thinking skills, further research is required to investigate its subsequent impact in a text-based programming environment.