Critical Review of Technology-Enhanced Learning using Automatic Content Analysis Case Study of TEL Maturity Assessment Formulation

—Technology-enhanced learning (TEL) continues to grow gradually while considering a multitude of factors, which underpins the need to develop a TEL maturity assessment as a guideline for this gradual improvement. This study investigates the potential application of TEL’s expert knowledge presented in various research articles as qualitative data for developing assessment questionnaires. A mixed-method approach is applied to analyze the qualitative data using systematic literature review (SLR) with automated content analysis (ACA) as quantitative data processing to strengthen the trustworthiness of the findings and reduce researcher bias. This process is carried out six steps: conducting SLR, data processing with ACA using Leximancer, organizing resulting concepts with facet analysis, contextualizing each TEL facet, constructing the assessment questionnaire for each context, and establishing TEL maturity dimensions. This study generates 64 questionnaire statements grouped according to the target respondents, namely students, teachers, or institutions. This set of questions is also grouped into dimensions representing aligned context: student performance, learning process, applied technology, contents, accessibility, teachers and teachings, strategy and regulation. Further research is required to distribute this questionnaire for pilot respondents to design the improvement roadmap and check data patterns to formulate maturity appraisals and scoring methods.


I. INTRODUCTION
Technology-enhanced learning (TEL) exploits technological advancement for continuous learning improvement. However, these advances cannot always be applied simultaneously in every region or educational institution. Several factors influence TEL application success, including accessible technology, supporting infrastructure, the conditions of learners, teachers, and the institution where learning takes place. These factors imply that TEL must be applied at different rates but gradually improved. Therefore, guidance for the application of TEL is required, as conceptualized in the TEL maturity model with its assessment instruments.
This research generates an assessment questionnaire grouped in related dimensions to build this instrument, which constructs a TEL maturity model. In related works, questionnaire formulations are derived from similar questionnaires in existing research, such as the capability maturity model (CMM), as seen in the maturity model for mobile learning [1]. Another example is a study about digital game maturity, which does not use CMM but still formulates the maturity's instrument based on the defined game development process [2] [3] . However, the maturity referred to in this study, namely the conditions for TEL application and how to gradually improve it, is inconsistent with maturity assessments emphasizing the maturity process. The other techniques in related work utilize qualitative analysis to exploit experts' knowledge or implement literature reviews from previous articles with a similar topic [4]. Combining these two techniques opens the prospect of developing TEL assessment instruments consistent with the previously determined TEL maturity context. Regardless, issues with previous studies concern researcher bias affecting the process and the reliability of the terms used during the coding process in the qualitative analysis. However, in a deeper view, TEL practitioners and experts have made their knowledge explicit in various research journals. Thus, this study employs a literature review approach investigating experts' knowledge captured within related research articles.
The most common method for conducting a literature review is by searching for relevant articles. For example, this research has searched for various studies containing the keyword "TEL" and specifically discussing affecting factors, TEL assessment, or certain technology maturity. However, this process is considered insufficient because the obtained articles' scope of discussion does not meet the previously defined TEL maturity context. Furthermore, the existing literature review approach has shortcomings, such as the method's reliability in finding relevant articles, the possibility for researcher bias, and cognitive limitations in extracting knowledge from the vast amount of available research articles. Moreover, the requirement to assess TEL maturity requires understanding the entire scope of the TEL discussion. Thus, a systematic literature review (SLR) approach may reduce researcher bias and meet the requirement for a TEL maturity assessment.
In SLR-related works, statistical data processing has used only specific attributes from the selected articles. However, there is potential to uncover the underlying experts' and 385 | P a g e www.ijacsa.thesai.org researcher's knowledge in those articles. Additional approaches include research that seeks to automate the process of qualitative analysis [5]. Some of these approaches underlie why this study exploits automated content analysis (ACA) as a text analysis method using a tool called Leximancer. ACA produces a set of concepts (key terms) and themes (clustered concepts based on relevance). Previously, interpretation of ACA results, particularly those using Leximancer, have been limited to describing a topic's state of the art or research trends, as shown in [6], [7]. Nevertheless, the potential is enormous. Depending on how to organize the resulting knowledge, this collection of concepts can be interpreted from various perspectives.
Facet analysis is one method for organizing knowledge [8], including data that form a collection of concepts. As a crossdisciplinary approach, facet analysis is primarily used as a library classification method. Facet analysis can help represent the content of a broad discussion covering many documents. This study analyzes TEL facets, which can then be interpreted in various ways depending on the research objective, which in this work is to construct assessment instruments and dimensions for TEL maturity. Thus, this research examines the research question of how SLR and ACA can be used to develop a TEL assessment instrument.
The following five sections are structured to address the research question. Section 2 describes the rationale for this research's importance. Section 3 explains underlying theories and concepts to understand the research context. Then, Section 4 presents the methodology of conducting SLR using ACA, including structuring and interpreting the result. The following section discusses the interpretation of the previous step's results, which later become assessment questionnaires. The last section concludes the study and suggests further work to refine the instrument into a complete working framework for TEL assessment.

A. Requirement for a TEL Maturity Assessment
Technology continues to advance, including learning technology. Various initiatives in implementing technological advances are also increasing, focusing on digitization, process acceleration, and learning improvement. The continuous application of technology advancement is the underpinning principle of technology-enhanced learning. According to Kirkwood [9], TEL represents the use of information and communication technology (ICT) for learning and teaching. Programs for implementing technology to improve learning typically use a top-down mechanism, which refers to how policymakers implement programs at the operational level. However, these programs do not always consider preexisting conditions such as the availability of supporting infrastructure and differences in students' abilities to access and use technology.
As a result, we require guidelines for implementing TEL that consider a variety of factors and gradual improvement. This gradual mechanism is encapsulated in the concept of a maturity model and its assessment instruments. However, Nicoll et al. [10] stated that research articles exploring TEL evaluation remain limited. This study attempts to build this maturity model, which is intended for TEL evaluation and a gradual improvement guide. The TEL maturity model has essential components including model domains, attributes, appraisal and scoring methods, and improvement roadmaps [11]. Two components are constructed: (1) an assessment questionnaire as model attributes and (2) dimensions as model domains. The concerning issue is establishing these attributes and domains considering that TEL covers an enormous scope of discussion. The subsequent issue is what point of view objectifies the TEL maturity concept. Thus, this study implements a qualitative analysis that explores the underlying knowledge from the entire scope of TEL discussion.
Previous research has attempted to formulate a conceptual framework for TEL, incorporating discussions about technology and learning [12]. However, there are two drawbacks: the object discussed is e-learning, where the term is not quite suitable for the research context. The second is that the result is insufficient to be further analyzed as a working framework. Thus, this study tries to answer these shortcomings.
Moreover, TEL is a broad field of study as it covers both technology and learning discussions. Therefore, it is difficult to determine the scope of discussion of TEL, particularly if the purpose of the research is to find the dimensions of TEL. This challenge suggests a systematic literature review, as explained in the next section.

B. SLR with ACA to Gain the Whole Scope of TEL Discussion
The research context is the TEL domain, which includes discussions on technology's progression and application to learning advancement. The context is building a model that evaluates TEL maturity to recommend a strategy to improve the impact of technology use on learning. Then, the challenge is to find the factors representing TEL as a subject matter, which become the objects to be measured. Extracting insightful knowledge through literature reviews is an attempt to overcome this challenge.
Determining the factors to be measured is typically done by collecting knowledge from experts using the focus group discussion method or in-depth interviews. Meanwhile, every scientific publication article in renowned conferences and journals is also a form of knowledge externalization from experts or knowledgeable people in the TEL domain. This knowledge base justifies why the literature review may be used to acquire knowledge, followed by knowledge organization, with the result used to develop TEL maturity instruments.
According to Kitchenham [13], a systematic literature review is essential because evidence-based rather than expert opinion is also required. The article states that evidence can be in the form of a synthesis of best-quality scientific studies on a particular topic. In contrast to an expert review using ad hoc literature selection, an SLR is a methodologically rigorous review of research results. The objective of SLR is not merely to aggregate all existing evidence on a research question, it is also intended to support the development of evidence-based guidelines for practitioners. 386 | P a g e www.ijacsa.thesai.org The SLR uses specific criteria to select the papers to be reviewed. The review process is followed by statistical analysis and interpretation, resulting in understanding the subject under discussion. However, the literature review should draw on substantial knowledge suggested in the articles. Also, there is a requirement to extract knowledge in model development research that can be justified, reflecting the more thorough subject-matter discussion.
This research expands the technique of implementing quantitative data analysis on SLR by processing the entire content of the text from research articles rather than only particular attributes. An ACA approach is used to target this goal, employing the Leximancer tool. The concept of ACA is derived from text analysis, where all the terms in the various data source articles are parsed, cleaned, assigned weights, and sorted. Then, the relevance of each term to each other is calculated. Leximancer helps data processing visualize conceptual maps by generating main concepts contained within the text and determining how they are related. ACA with Leximancer is usually utilized for sentiment and data marekting analysis, as seen in [14] and [15]. Though, the resulting concepts and themes may deliver many useful insights with further analysis. It is then become one of the background idea in this research.
Regarding ACA, similar activities have been done in previous works using simpler text analysis techniques, as shown in Fig. 1. The first research utilized text analysis with the help of mini program built using Python, with a resulting list of key terms sorted by frequency of occurrence. Regarding interpretation of results, the previous study used Luhn's theory, which describes the relation of curves to Bradford-Zipf distributions to determine significant words [16]. These sorted words are located in between the upper and lower cut in the Zipf law distribution. In more recent research, the determination of the upper cut and lower cut was still not clearly stipulated, though a pattern was seen [17]. This provision of Luhn's theory was again adopted in this study. In the Methodology section, we discuss how this determination is established. The result of this study is a collection of TEL characteristics. However, this result cannot be directly converted into TEL assessment instruments.
Another work attempted to adopt the term frequencyinverse document frequency method [18] rather than a simple frequency count as a basis to sort the resulting list of key terms. However, this study also has a limitation in that it processed only 100 articles; therefore, its suitability for this research objective is dubious. Another study has already employed ACA [19], however, a drawback in the work is a lack of clarity on interpreting the results. In that study, the dimensions were directly derived from themes generated from Leximancer without further analysis, yet the underlying knowledge lies in the concepts generated. The resulting list of concepts requires different processing methods, which is what underlies the use of facet analysis.

C. Facet Analysis for Knowledge Interpretation and Organization
The output of ACA is a collection of key terms that are considered the most relevant and represent the scope of the TEL discussion, called the concept. However, these results require further analysis to establish dimensional candidates and assessment questionnaires. Quantitative data processing, automated with Leximancer, needs to be counteracted with interpreting the result, which is the significant proposal of this study.
Facet analysis departs from the classification theory originated by Raghnathan [20]. The technique recognizes several aspects of a topic of discussion and summarizes these aspects to appropriately describe the concept. A "facet" is a collection of terms that have the same relationship with the global subject, reflecting the application of a fundamental principle of division. Certain subjects are delivered in various perspectives so that the representation of knowledge can be in its entirety, not as a subordinate of something else.
According to Usman et al., facet analysis is one of the most frequently used classification structures in education and computer science [21]. In addition, the discussion of TEL is multidimensional and multiperspective (facets). Facet analysis is also widely studied in social science (21%), and computer science (11%), both of which are also domains that frequently examine TEL. Thus, facet analysis is not new in computer science and may be suitable to be applied in this study.

III. METHODOLOGY
This study addresses the research question by following the steps depicted in Fig. 2. The diagram shows how each step has inputs and outputs and employs particular techniques. In the following subsections, each step is explained.

A. SLR for Data Gathering
After various data selection processes applied to 1,030 articles filtered by inclusion, exclusion, and quality-assessment criteria, 792 journal articles were obtained. Inclusion and exclusion criteria include Scopus-indexed journals, in English, published between 2010 and 2021, available for full-text download, and mentioning "technology-enhanced learning" in the titles or abstracts. The process continued by performing text analysis on the entire content of the selected articles using the ACA method. The aim of the analysis was to explore underlying knowledge representing the overall scope of the TEL discussion rather than only specific attributes. 387 | P a g e www.ijacsa.thesai.org

B. ACA with Leximancer
Leximancer assists the ACA process by importing data, generating concept seeds, generating a thesaurus, and visualizing the results. The process begins with importing previously selected articles. Then, the tool parses the terms from all articles, weighting them based on both the frequency and likelihood of occurrence. This weighting score becomes a basis for sorting terms, eliminating low-scoring terms, and assigning high-scoring terms as resulting concepts. The data learning process is continuous, starting with selecting a concept seed from the list of terms. A concept is a highly relevant term that represents the topic in a collection of research articles as data.
The concept seed is the starting point of the definition of such a concept. The process then adds any highly related terms into the concept's definition. Thus, if there are more relevant terms than the seed, a new concept seed can be generated. This process continues until all terms have been processed. The term with the highest weighting score becomes a concept that represents the various meanings of the terms. The concepts that are highly related to each other will be clustered into a higher level of data representation, called a theme.
Leximancer will then visualize the formation themes and the sequence of concepts based on the count of hits, as shown in Fig. 3. The illustration depicts the resulting concepts (relevant and meaningful terms) and themes (a group of interrelated concepts). The themes are frequently analyzed as dimension candidates in related works, though they can change according to the theme size setting. However, there is also no exact formula for the optimal size. In addition, some of the generated themes are a collection of outliers, namely concepts that are not closely related to the central concept. Such outliers are inappropriate for this study, which looks for concepts and dimensions capable of representing TEL. In the previous work [19], the selection of candidate dimensions was done only by the highest score concepts.
Selection of candidate dimensions in this research adopts Luhn's theory, using concepts' hits of occurrence as the basis for analysis. The related work also used this theory [18] for choosing the key terms (called concept in this research). This study chooses the main themes for candidate dimensions by determining an upper and lower cut. The size setting for the theme is determined with prudence at 25%, considering the condition where the bar chart could present the hits of occurring numbers with a clear distinction between the upper and lower cut, as shown in the Analysis Synopsis tab in Fig. 3. This setting represents a condition where several themes located between the upper and lower cut can be assumed to be the most relevant, themes above the upper cut are considered too general, and those below the lower cut are less relevant. The resulting themes cover "Students," "Used," "Study," "Different," "Educational," and "Teacher." These selected themes will be further examined during facet analysis. The next step is to break down each selected theme into a list of concepts as the output. The Luhn theory [16] is also used in this process. The concepts are examined based on the terms' occurrence counts and the likelihood of occurrence related to the theme as the main concept. Fig. 4 and Fig. 5 show graph illustrations that present sorted concepts based on the occurrence likelihood and count. The detailed theme in the figures is Students.   The likelihood is the possible percentage of the occurrence between two concepts. In the likelihood graph (Fig. 4), the concepts on the left side lower cut are chosen, which is the condition before the curve is flattened. The count is the frequency of the related concept and the topic of discussion. The chosen concepts have counts located between the upper and lower cut. The lower cut is when the curve starts to flatten, while the upper cut is when the curve starts to decrease, but not too sharply. Next, the resulting list of concepts from Fig. 4 and Fig. 5 are cross-compared to the concepts that occur in both, thus generating a list of concepts for a theme. Each chosen theme is further explored following this approach.

C. Facet Analysis for Organizing ACA Results
The output of ACA is a collection of the most relevant concepts representing the scope of the TEL discussion. These findings, however, cannot be directly converted into assessment questionnaires. Accordingly, the contribution proposed in this research is how to interpret this list of concepts and translate it into expected results according to the research objective.
The generated concepts serve as the TEL key terms. The next step is to apply facet analysis by examining each concept using the specific criteria. The first criterion, which is listed in the concepts in the topic guide, is one of Leximancer's outputs, representing a subject index for an extensive document collection. The second is listed concepts that are not topicspecific or highly related to research and writing terminology, such as conjunctions, verbs, or adjectives. This resulting concept collection is then organized into a logical classification, written as hierarchical structures. It is the performed process to construct TEL facets. Every structure represents a distinct aspect of a story related to the topic under discussion. Several alternative methods exist to develop a facet: drawing from Leximancer's topic guide as it is (clear description); structuring several concepts into a make-sense facet (need analysis), and digging deeper into sub-concepts to attain the meaning (need deeper analysis). The three alternatives may become a recommendation if weighting is required in the TEL maturity assessment.
The results of the facet analysis for the Students theme can be seen in Fig. 6. The first result, referred to as a "clear description," is the "problem students" facet, which has a subfacet covering problems, skills, thinking, and understanding. The "need analysis" is a student facet due to the logically structured various concepts using the meaning of terms. The school facet covers "teachers," "classroom," "class," and "digital" as sub-facets. The last step is incorporating another concept (term) to the chosen concepts to deliver helpful insight, which is part of the "needs deeper analysis." For example, in the student inquiry facet, the concept of "questions" complements the meaning. This process is repeated for all the themes, resulting in the TEL facets. The next concern is how to obtain useful knowledge from this collection of the TEL facets.

D. Quotation Analysis to Contextualize TEL Facets
The next step is to investigate the underlying insight from TEL facets by putting them into context. Facet analysis allows us to comprehend the topic of discussion using various aspects. A quotation is used to learn the context of an aspect, which is a phrase or sentence in which the facet occurs. The process searches for quotations that contain the facet, covering a combination of concepts. Table I shows an example of examining the context of a TEL facet by finding several relevant quotations from the Students theme. Each facet may contain more than one aspect and insightful knowledge.
For example, the facet on student problems delivers the context as learning objectives. This context is determined using quotations about learning objectives, student activities in class, and technology to improve learning. The quotation search also employs Leximancer by defining searching query using a combination of concepts with the help of an "AND" operator. 389 | P a g e www.ijacsa.thesai.org • "Using these scaffolds, and WISE as a meta-context, students constructed and solved problems." • Help students find or generate their own goals in the problem solving • Students' learning processes in technology-enhanced …… and students exercised …. skills and developed understanding …. design thinking, problem-solving, critical thinking, ….. .
Problem-solving, exercise skills, thinking, and understanding in class lessons 2 School (Teachers, Classroom, Class) • Every classroom was fitted with an interactive whiteboard; there was a class set of wireless laptops as well as a small ICT suite; and each teacher had their own laptop. This process repeats to analyze each aspect of each chosen theme. The use of concept combination queries, such as using two or three combinations, is also expected. For example: "problem + students" or "student problem" as a compound word, or "problem students + skills," and so on. This alternative query combination is used to improve the quality of the obtained quotations. After many iterations searching for a facet quotation, the process starts to deliver quotations with the same meaning. The process is reasonable considering that each facet together represents one main topic. As a result, the redundancy elimination process must begin at this point. These collections of concluded contexts became the basis for formulating the TEL questionnaire.

E. Interpreting TEL Facets to Formulate Assessment Questionnaires and Dimensions
This advanced step explains how to formulate the TEL assessment questionnaire by understanding the TEL facet's context, with the illustration shown in Table II. A row corresponds to a facet. Each context is associated with one or more statements. A questionnaire is then designed as a series of statements accompanied by a Likert scale asking how much the respondent agrees with the statement. Each context and the questionnaire item are also scrutinized for its type of respondent target. This process is repeated until all possible contexts for each candidate dimension have been discussed. Further analysis is performed to compare each question representing each candidate dimension to reduce the possibility of redundant statements. The result is a distributed array of statements for each candidate dimension. This process repeats until all possible contexts for each theme have been analyzed. Further analysis is performed by comparing each theme's questionnaire statements to reduce the possibility of redundant statements. As a result, we can finally obtain the complete questionnaire formulation. The resulting statements are naturally clustered into groups based on the dimensions candidates generated in the previous step (subsection B). However, this study conducts more clustering processes to determine additional representable groups with aligned and explicit ideas, called dimensions. These findings are consistent with the grouping of questionnaire statements that have minimized redundancy.

IV. RESULT AND DISCUSSION
The discussion includes insight about assessment weighting recommendations from the quotation analysis, constructed questionnaire examples, and the dimensions covering the questionnaire.

A. Quotation for Context
In line with the facet determination process, three criteria related to the difficulty of the analysis process also determine the quotation analysis. The process begins by identifying quotations that clearly describe the context, those that require simple analysis, and quotations requiring further analysis. The ease of performing the analysis is related to the context's relevance to TEL.
The relevance is high if the meaning is clear and explicit, and vice versa. Establishing the quotation may also become a basis for defining the TEL assessment weighting. Thus, every facet has different measure in depicting the data story as follows: clear description: High; needs simple analysis: Medium; requires deeper analysis: Low. Table II shows an example of the statement formulation for the Students dimension, with the sample questionnaire for each dimension as delivered in the Appendix. There are several concerning issues, however, in constructing the questionnaire statements. The first is translating the English context into Indonesian Bahasa statements. This study was conducted in Indonesia; therefore, the prospective respondents are Indonesians who are not English natives.

B. TEL Maturity Assessment Questionnaires
The second issue is a requirement to identify respondents' demography. Therefore, preliminary questions are needed to help understand the characteristics of the respondents, that is, respondents' assessment questionnaires. The questions are not formulated from the previously described process but are customized to the needs of the research. As a result, the customized preliminary questions reduce potential confusion when applied to pilot respondents. The information is then used to improve the questionnaire before it is widely distributed.
Concerns are also evident regarding what role is appropriate for a respondent for a specific question. Based on the analysis, it was discovered that the students could not answer all of the questions. As a result, the respondents' selections became broader; namely, students, teachers, both students and teachers, and institutions. The expanded scope raises the question of how to process the data and regulate the proportion of each respondent's role in the TEL maturity assessment. 390 | P a g e www.ijacsa.thesai. org  TABLE II.  ILLUSTRATION OF HOW TO DEVELOP A TEL MATURITY ASSESSMENT QUESTIONNAIRE   No  Contexts  Target  Respondent  Questionnaire Statements   1 Problem-solving, exercise skills, thinking, and understanding inclass lessons

Students, teachers
Existing technology can assist students in comprehending the solutions to problem or questions encountered during a learning activity.

2
• Technology to improve academic achievement • Technology helps student assessments • Technology enables more detailed feedback • Technology has both good and adverse effects on student performance

Students, teachers
• [student/teacher] Technology can help students improve their academic performance (improvement of grades). • [teacher] Existing technology can aid in the process of evaluating student learning outcomes. • [student/teacher] Technology has been used to reach all students for them to receive feedback (comments, improvements) on the outcomes of their work on their assignments. • [student/teacher] Technology positively impacts student learning outcomes (e.g., quiz/exam scores). • [student/teacher] Technology harms student learning outcomes (interference) (example: due to cellphone addiction for less-productive matters) Another concerning issue arises when establishing the context of the collection of quotations. It may be possible that researcher subjectivity is present. However, this study includes efforts to reduce researcher bias and subjectivity while increasing the trustworthiness of the research by conducting quantitative data processing using ACA before the context analysis. This subjectivity may be assumed as part of the researcher's reasoning when analyzing and interpreting the data.

C. TEL Maturity Assessment Dimensions
The following seven dimensions were determined based on the initial dimension candidates, context adjustment, and regrouping the generated questions with an aligned idea. Each dimension represents influencing factors and being influenced by technological usage. Table III describes the dimensions and the corresponding assessment questionnaire statements. In previous work [19], the result only consists of three general dimensions: technology advancement, improved learning design, student's achievement. Related to the formulation of learning strategies using technology and conformity to the rules Technology Governance 9 Regarding technology, governance to support learning The number of questionnaire statements for each dimension is determined by general statistical provisions stating that the number of related questions should not be excessively different. The method assumes that each dimension is equally important in achieving a level of TEL maturity. The following activity tests the questionnaire on pilot respondents. Thus, the weight of each dimension influencing TEL maturity can be recalculated in future research. Moreover, the formulation of these dimensions can be helpful in the construction of a TEL working framework. Recommendations for improving the TEL maturity assessment results will also be made for each dimension.

V. CONCLUSION AND LIMITATIONS
This study investigates how to use SLR in conjunction with ACA to create TEL maturity assessment instruments constituting questionnaires and dimensions. This research attempts to combine automatization in qualitative data analysis using ACA and qualitative data interpretation using facet analysis. The methodology include data gathering using SLR, data processing using ACA, organizing resulting concepts using facet analysis, searching quotations matching the meaning of TEL facets, inferring contexts from TEL facets, and determining dimensions of TEL maturity.
This research is part of a more extensive study to determine the TEL maturity assessment instrument, which attempts to take a novel approach. The approach used in this study can become a recommendation for how SLR can help novice researchers formulate assessment questionnaires through a literature review and discover how qualitative analysis can be initiated with a quantitative approach to reduce bias and subjectivity of the researchers.
In this study, 64 questionnaire statements were assembled and categorized based on the respondents' target, either the institution, the student, the teacher, or both. This questionnaire statement set is also grouped based on the TEL maturity constructs, which are referred to a dimension. The seven dimensions are students' performance, learning process, applied technology, contents, accessibility, teachers and teaching, and strategy and regulation.
This study has some limitations, including a critical point in implementing certain stages. The first obstacle is when the context is reduced to questions in different languages (from English to Indonesian) so that pilot respondents are still required to accept the predetermined questionnaire formulation. The second is the point of view used in developing the questionnaire: a higher education institution in this study. As a result, the findings may not apply to other 391 | P a g e www.ijacsa.thesai.org educational levels. The third issue is that the focus on which instrument was built in the use of technology primarily from the learner's perspective to improve learning, so the pedagogical aspect is not discussed in depth. The fourth issue is how to justify the influencing portion of each type of respondent on the measurement results.
Future research is required to test the questionnaire on pilot respondents to see if they can understand, conform with the assessment's objective, and examine the correlation between questionnaire statements and dimensions as a mutually exclusive entity. These findings would then be used to develop a maturity appraisal and scoring method, including the weighting to establish the TEL maturity working framework. Furthermore, the questionnaire's development results are used as a reference for developing TEL improvement roadmap. 1 The existing technology can help students understand problem solutions/answer the questions in a learning activity.