UX Testing for Mobile Learning Applications of Deaf Children

Many studies are focusing on deaf children mobile learning. However, they are not concentrating on user experience (UX) testing. Current UX testing is based on existing UX evaluation models that are hard to apply due to the comprehensive measurements and lack of description on how to conduct evaluation for a more specific mobile learning process. Moreover, the existing UX evaluation models are not highlighted to be applied in testing UX for deaf children’s mobile learning. Hence, this paper proposed questions for UX testing for deaf children’s mobile learning to explore UX issues in offering an enjoyable learning application. Smileyometer is used to capture the data from deaf children after using a selected mobile learning application, KoTBaM and Learning Fakih. This study involves deaf children aged 7 – 12 years old familiar with the mobile application. The survey is divided into two sections: i) demographic information and ii) 24 questions that the respondent must answer using a smileyometer. The survey included 38 deaf children from Malaysian Deaf School. The participating deaf children completed the questionnaires with the assistance of their teachers after using the mobile learning application in the classroom. Yet, various issues needed to be addressed in order to improve the deaf children's user experience. Special exercises should be developed for deaf children connected to their school syllabus to consolidate their knowledge and self-learn everywhere. Furthermore, games elements should be adapted so the deaf children are able to learn while playing. Keywords—User experience; UX testing; UX dimension; UX metrics; mobile learning application; deaf children; smileyometer


I. INTRODUCTION
UX testing is a method to measure how easy an application is. It is also performed by actual users for a selected application [1]. In determining the experience that occurs when a user interacts with a system [2], the result from this test can be employed in a system's design [3]. As Moran (2019) indicated, UX testing can determine if there are any problems in the design of an application, any parts that do not cover users' meet and learning about user behaviour [1]. Through the UX testing's result, the mobile learning application can be constructed appropriately focusing on enduser [2]. In addition, most mobile learning applications developed specifically for deaf children are less used than common applications for general users [4]. Deaf children require less effort to complete tasks and spend less time understanding the flow of mobile learning, which directly contributes to determining their UX.
Furthermore, there is a lack of UX testing for deaf children's mobile applications [5]. Thus, in order to perform UX testing that focuses on deaf children's mobile learning, appropriate questions that focus on the subject are essential [6]. It enhances UX questionnaires such as UEQ [7] and meCUE [8], which are developed for general UX testing. It is one of the efforts to provide equal rights and opportunities to this community. Thus, disabled children should not be left behind in adapting self-learning to ensure equal opportunities like ordinary children.
In Malaysia, disabilities are classified into few categories, such as mute, deaf, blind, mental disability, and physical disability [9]. Likewise, this study focuses on deaf children since the number of deaf people is increasing yearly [10] and is expected to grow to 700 million in 2050 [11]. On the other hand, [12] reported that in 2025, the number of smartphone users will increase to 30.74 million in Malaysia. The deaf community may contribute to growing these statistics. Thus, UX of the deaf should be a highlight in developing an excellent mobile learning application.
Mobile learning application has proven to increase learning interest and motivate deaf children [10]. Moreover, in the Covid-19 pandemic, deaf children have to self-learn to ensure they are not left behind in their education even though the pandemic situation. Hence, mobile learning applications become a vital aid for them to learn and do revision. Consequently, UX testing has to be done to ensure the deaf mobile learning applications are entirely designed and give a good experience for them. Indirectly, it will overcome the less-used issues of existing deaf children mobile learning [4].
The UX testing can be conducted efficiently if the assessment is focused on mobile learning applications for deaf children [6]. Therefore, some questions are adopted from UEQ [7] and meCUE [8], where both of these instruments contain general UX questionnaires. [13] in their work have proposed usability questionnaires for deaf mobile application interfaces. All the questions are modified to make them related to deaf children's mobile learning. These questions are applied during the UX testing where children will answer them using Smileyometer after they have used the selected mobile learning applications. Smileyometer is a familiar method used for study among children [14] [15]. The deaf children are assisted by their teachers to complete the smileyometer since the researchers are not allowed to communicate directly with the deaf children in the classroom. The deaf children are more Fundamental Research Grant Scheme (FRGS/1/2020/ICT10/UUM/02/1) www.ijacsa.thesai.org comfortable with their teachers than outsiders [16] [17]. Thus, the questionnaires are distributed through their teachers, who briefed them about the UX testing process.
There are many previous UX studies among children but very few of them concentrated on disabled children, especially deaf children. Thus, this study is needed to measure the UX among deaf children using mobile applications designed specifically for them. However, this study is limited to testing on children who are deaf only since there are many types of deafness in Malaysia. According to the Social Welfare Department of Malaysia (2018), deafness can be classified into three, which are deaf only, deaf and dumb, and deaf with other disabilities [9]. Thus, the participants for this study will be selected among the deaf-only children by their teacher. This paper is aim to test the UX of mobile learning application for deaf children. Hence, it was organized as follows: introduction, background, material and method, results and discussions, and conclusion.

II. BACKGROUND
This section covers past studies regarding UX of mobile learning for children since UX among disabled children is very few [5] and selection questions related to UX dimensions suggested in previous studies. The background of smileyometer as an instrument in collecting data is also explained in this section. It is also supported by some previous studies that applied this fun toolkit [18].
There are many studies involved in UX testing among children, such as [15][17] [19]. Those studies are used selected mobile learning applications during UX testing sessions. [15] were studied about ascertaining the UX of Word Mania among kids at UUM International School, Sintok Malaysia. Nevertheless, this study has not applied any questionnaires even though they use a smileyometer to capture the data. Based on their finding, the kids need more time to answer the questions in Word Mania. Thus, the simplest task should be involved in children's mobile learning applications.
While [17] had test UX among 64 children aged 7-12 years old using Fantasy Land. The children are required to answer UEQ questionnaires. However, some questions are difficult to understand since advanced terms are involved, so [17] have to explain the terms using simple sentences that suit the children's age. Thus, the UEQ questionnaires should be improvised to suit the capabilities of the children. For the same purpose of study, [19] also studied UX among children using mobile technology. They are motivated to learn it since there is a lack of UX studies regarding educational applications such as science and mathematics. So, this study is to explore the hedonic factors such as enjoyable, exciting, upsetting, and confusing experiences. [19] found that gender, culture, and religion are important matters that affect the children's experience.
On the other hand, there are minimal UX testing among deaf children. [20] studied UX testing among attention deficit hyperactivity disorder (ADHD) children. The UX testing was involved ADHD children from Mexico. Due to the covid-19 pandemic, the children had very limited face-to-face activities in school. Thus, [20] design a learning application using virtual reality (VR) technology. The children feel like in a face-to-face class when they are using this application. [20] obtained satisfactory experience among ADHD children who are using the VR educational application. While [21] were studied UX testing among autistic children in comparing three prototypes of learning applications. Smileyometer, fun sorter, and again-again table are used to collect data in this study. Smileyometer has been applied before and after the study. The children are given around 5-10 minutes to play with the application before completing the instrument. Based on this study, [21] concludes that very few different experiences among the three prototypes and few usability issues to the specific prototype.
Many research cover UX of mobile learning among children, which recommended some dimensions such as emotion [ [26]. Likewise, deaf accessibility is also considered as a dimension in developing the questions for this study since it is highlighted on the deaf itself [13]. As mentioned earlier, studies regarding the UX testing of mobile learning among disabled children are minimal [2]. Thus, the mentioned studies were taken based on targeting children.
Most of the studies related to children use smileyometer in capturing the data [26]. Smileyometer is being chosen, and it was believed in capturing data from children [27]. Smileyometer is one of the fun toolkits besides fun sorter and again-again table. It was introduced by Janet C. Read [18] in 2006.
There are five emojis, as shown in Several studies applied smileyometer, such as evaluating the user experience of playful interactive learning interfaces with children [14], ascertaining the UX of the word mania mobile app for children using fun toolkit v3 [15], and understanding the fidelity effect when evaluating games with children [21]. Moreover, smileyometer is used in highlighting the children's momentum feeling [25]. Hence, smileyometer is chosen for this study in evaluating the UX of mobile learning for deaf children. Based on previous research that using smileyometer proved that this is the interactive scale in obtaining data from children [25]. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 12, No. 11, 2021 296 | P a g e www.ijacsa.thesai.org Two mobile learning applications for deaf children, which are KoTBaM and Learning Fakih, were chosen. KoTBaM is a learning application developed based on Bahasa Isyarat Komunikasi for deaf children in Malaysian Deaf School [28]. The participants might feel easy to understand content of KoTBaM since they have learned about the content in school. Besides, some videos on constructing sentences using sign languages were provided too, giving participants more attention to learning [29]. While Learning Fakih is a learning game application for deaf children developed based on the Fakih method [30]. The Fakih method is a technique applied by the teachers in Malaysian Deaf School in teaching Hijahiyyah letters using sign languages. It has quizzes elements to make the lesson fun as claimed by González et al. that enjoyment should be provided to avoid boredom and improve effective state among the children [31]. The quizzes are divided into three levels which are easy, medium, and hard. These two applications are chosen since the content of the application are related to lesson in school. Besides that, there are also recommended by the teachers of deaf children.
Besides that, as stated earlier, the UX dimensions can be a reference for this study in developing UX metrics for UX testing of deaf children's mobile learning. Since smileyometer was very familiar in previous UX studies among children, it also measured the UX among deaf children. Hence, a set of questionnaires have been developed based on the dimensions applied in previous studies. It was discussed thoroughly in the next section.

III. MATERIAL AND METHOD
This study aims to evaluate the UX of deaf children's mobile learning by the actual user, and it was briefly explained in this section. The process of identifying the selected questions is out of the scope of this paper. This study was performed to measure UX for two learning applications which are KoTBaM and Learning Fakih using smileyometer and adheres to [24] UX testing approach on how to conduct UX test in a classroom environment which is used by many past researchers [14][17] [25] and finds an approachable method. There are three steps involved in evaluating UX of deaf children's mobile learning applications for this study.

A. Identifying the Participants
A total of 38 deaf children aged between 7 to 12 years old has been recruited from Malaysian Deaf School. The pupils are screened based on their experience of working on mobile and mobile applications. According to Lazar et al., 10 participants are substantial for this study since they are deaf and considered disabled people [32]. Purposive sampling is used in this study which common approach in identifying a user in UX [33]. Purposive sampling means that participants are selected based on the needs of the study. This approach was applied in a previous study that involved a deaf sample, such as [7][17] [18]. Teachers select the deaf participants based on their familiarity in using mobile learning applications.

B. Instrument's Development
A set of questionnaires are adapted from UEQ [7], meCUE [8], and questionnaires from [13] that are supported to five dimensions, as shown in Table I. The arrangement of words and understandability of the questionnaires for the participants are checked by an expert who is an author of textbook Bahasa Isyarat [28] in Malaysian Deaf School. The questionnaires use smileyometer scale to tick by the participants after interacting with the KoTBaM and Learning Fakih.
Twenty-four survey questions are designed for this study. The questions are based on existing UX questionnaires, as mentioned earlier in the previous section. The metrics depend on five dimensions: emotion, satisfaction, efficiency, effectiveness, and deaf accessibility. The distribution number of questions is stated in Table I. According to Table I, five questions regarding emotion dimension, four questions about satisfaction, deaf accessibility, and effectiveness dimensions, while seven focus on the efficiency dimension. All the questions are going for a reliability test to know how reliable the questions are. The test uses SPSS to get the Cronbach alpha value. According to Sekaran, a reliability coefficient of 0.70 or higher is considered acceptable, while 0.90 to 1.00 consider having excellent coefficient reliability among the items in the questionnaire. Reliable coefficient survey questions for this study is 0.967 as shown in Table II. Thus, it was considered acceptable to use in this study [34].

C. Test Administration
Participants have been given a brief description of the UX testing conducted. Since the participants are deaf, Malaysian Deaf School teachers become translators throughout the evaluation process. All the participating documents have been passed to the teachers during the briefing session. It is because the researchers are not permitted to meet the participants directly due to the covid-19 pandemic. The Malaysia Ministry of Education limits physical activities to curb the transmission of the virus. Therefore, translators are given space in translating the instructions into sign language for the participants. Two mobile learning applications, KoTBaM and Learning Fakih are involved during the session. The participants are given 5 to 10 minutes to use every learning application. They are allowed to end the session early if they are feeling bored while using the application. Once participants understand and agree, testing is conducted. The participants are required to answer the survey and assist by their teacher.

D. Data Collection and Data Analysis Method
Every dimension has been tested through the questionnaires during the UX testing session. The feedback has been analyzed using the mean value for each dimension (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 12, No. 11, 2021 297 | P a g e www.ijacsa.thesai.org involved, as suggested by [8] to compare the two learning applications. The mean value can be determined which applications are giving good UX based on the dimensions analyzed. Thus, the analyzed result can be used as guidance to improve the design of the learning application in the future.

IV. RESULT AND DISCUSSION
This section reports the result of UX testing. They are classified into two (2) sub-sections, which are 1) demographic information and 2) data results. The data was examined through SPSS to report on the results. The data analysis yields a comparison of which learning application gives better UX for deaf children.

A. Demographic Information
This part aims to assess the demographic analysis of the users involved in the UX testing in terms of gender and age. For the UX testing, 38 aged ranged between 7 and 12 years old participants were chosen due to the availability of the selected schools. Most participants are age 11 years old, stating 37% followed by 30% for participants aged 12 years old. At the same time, participants aged ten and nine years old have shown 20 % and 13% of the participants involved (Fig. 2).  Fig. 3, 50% were boys while the rest were girls. Among them, MDS Perlis participants were the highest, consisting of 43%, MDS Johor Bahru were 40% of a total participant, and finally, MDS Temenggong, Segamat comprised 17% of overall participants (refer Fig. 4).

B. Data Analysis Result
The data was collected through survey questions (refer to Table II) that were rated by the participants using smileyometer. The data have been analyzed by each dimension involved in this model through their mean value. It is also presented in a graphical bar chart to show the comparison between criteria in the dimension. Meanwhile, the results are also presented in comparison between two learning applications: Learning Fakih and KoTBAM. Learning Fakih is a prototype for a mobile learning application, while KoTBAM is a learning application that has been published and can be downloaded from the play store.
As presented in Fig. 5, two mobile learning applications have been evaluated for UX testing among deaf children. The mean value is shown based on the dimension involved. The effectiveness dimension for KoTBAM has stated the highest mean value, which is 4.4, compared to the highest mean value for Learning Fakih, which is 3.75 that stated in the emotion dimension. The satisfaction dimension stated the lowest mean value for both applications, 3.69 for KoTBaM and 3.4 for Learning Fakih. However, all mean values for KoTBaM are higher than Learning Fakih, as reported in Fig. 5. In this case, it can be summarized that majority of the deaf children will choose KoTBAM compared to Learning Fakih since all the mean values on all dimensions for KoTBAM are higher. Based on personal communication between the teachers who conduct the UX testing session, the deaf children are quite slow to answer quizzes at the hard level provided in Learning Fakih. It may be difficult for them to remember [35] the sequence of Hijahiyyah letters and baris orderly. It may make the questions quite difficult for them to answer since they cannot remember new things quickly. Likewise, they were pretty enjoyed the learning since they are able to play it while learning. Hence, repetition is significant to remember and memorize the thing that they had learned.
Furthermore, KoTBaM is more accessible for deaf children compared to Learning Fakih. Deaf children need guidance, and it was provided in KoTBaM but not in Learning Fakih. KoTBaM provides a user manual to guide deaf children on how to use the learning application. On the other hand, the user manual also can be a reference for the normal user such as teachers or parents of deaf children in using the deaf learning application. These features may make deaf children feel more confident to use it. It may contribute to the mean value of deaf accessibility for KoTBaM being higher than Learning Fakih.
Moreover, the contents in KoTBaM are based on the syllabus that deaf children have learned in their schools. It looks like the deaf children do revision for the subject through KoTBaM. Every word in the contents of KoTBaM is provided with videos of sign language compared to Learning Fakih, which only provides images of the hand gestures of the sign language. The contents do not focus only on the topic, but other lessons included like alphabets, numbers, and name of animals. The sign languages are provided only for hijahiyyah letters but not for others. The deaf children might be confused about the content since it involves many subjects. Therefore, the Learning Fakih is not practical for deaf children, affecting the mean value of the effectiveness dimension rated by the participants.
However, there are no quizzes or exercises provided by KoTBaM, but there were some quizzes in Learning Fakih. Quizzes and exercises may need to test the understandability of the deaf children about their learning. The deaf children do not perform the quizzes in Learning Fakih since they feel that it is so hard for them to choose the correct answer. As claimed by Marschark & Spencer, the deaf children are slower four times than normal children [35]. Thus, the exercise provided should be easier and simpler for them.

V. CONCLUSION
In conclusion, the survey questions of this study can be a guideline to measure for a more pleasant and practical deaf children's mobile learning besides can be improved the basis of learning for this special education children. Besides that, the result also can be a reference for UX practitioners and mobile application developers in designing a practical and enjoyable mobile learning application for deaf children. However, the results are evaluated among deaf children only and to be applied on another two variants of the deaf, which are deaf and dumb, and deaf with other disabilities.
In addition, these survey questions can also be applied for UX testing for other mobile learning applications for deaf children. The proposed dimensions in this work are customized and highlighted on deaf accessibility. Hence, other disabilities are not recommended to utilize this survey. The result may not be accurate if these dimension are tested for other disabilities since different disabilities may need different accessibility features to support learning using mobile applications. In the future, the authors may extend this study to investigate how these dimensions are associated to one another in measuring a UX for mobile learning application for deaf children.