Usability of Mobile Assisted Language Learning App

— The aim of this study is to evaluate the usability of Mobile Assisted Language Learning i.e. Literacy and Numeracy Drive (LND) which is smartphone application to learn language and mathematics in public sector primary schools of Punjab, the biggest province of Pakistan. In this study, usability tests were conducted which included surveys of questionnaires from teachers and students. The user experience, reliability, and performance of mobile application assessed, along with user satisfaction. The LND mobile application has not been found to be successful, with a poor user interface and requires improvement. The "Using Experience," "Ease of Use" and "Usefulness" variables have been the lowest scorers in terms of user experience. Mobile device specifications were not simple and confusing; the services provided by the LND were not appealing and effective for students or teachers. This research suggested several improvements in the usability and functionality of this LND application based on assessed user experience. Many schools have chosen to use mobile apps for the teaching and evaluation of language at school. The use of mobile-assisted learning at public sector schools in Punjab, invites us to gauge the usability and effectiveness of this approach at such a huge scale which will make it more effective.


I. INTRODUCTION
Ubiquity and Accessibility of Mobile Phone and Internet: The growing popularity of the digital technology of the newer generation, the growth of mobile data and the fall of smartphone prices have created a strong basis for the use of smartphones on the web. This has led to a growing global number of mobile phone users. [25] announced 1375.5 million smartphone units were exported by the sellers worldwide. However, the sellers have delivered for the very first time over 1 billion units in 2013, 38.4% above last year's numbers.
In Chinese smartphone sales, the same pattern was observed. In 2013, 423 million smartphones were exported [37]; this estimate is 64.1% greater than last year, with the number of Chinese mobile users rising steadily. A total of 527 million units of Chinese mobile telephones were developed by the [9] in July 2014 and the 34th China Internet Network Development Statistical Reports were released and in 2014-15, the number of mobile subscribers in Pakistan rose to 114.7 million in 2019, about 72 million wireless subscribers and in September of 2019 more than 3G/4 G subscribers and more than 74 million subscribers with broadband services [42]. In China [8], 86 percent of university students had smartphones, and only 02% had regular mobile phones with no internet browsing capabilities.
Mobile-based learning: M-learning has various aspects, offers cost-effective learning and expands learner space [59]. This means that computing is a valuable tool in education through the development and growth of mobile technology in the various mobile telecommunications industries, which are actually the principal technology in use [14]. The key advantage of mobile learning is that students use these smart devices for learning purposes and connectivity quickly, easily and with the rapid progress of smartphone technology [45].
LND an initiative of Government of the Punjab, Pakistan: Keeping in view the benefits of M-learning, The Government of Punjab initiated a mobile-based learning application named Literacy and Numeracy Drive (LND) for the learning and assessment of public sector school students of grade 03. This project was launched in 2015, in whole province of Punjab having 36 Districts, in which 52,394 public sector schools, 403,172 teachers and 12,268,981 students enrolled [41] to eliminate the old-fashioned procedures of large scale pupil assessment such as Punjab Examination Commission (PEC) are expensive, occasional, and complex [58].
Thus, a low-cost, tablet-PC pupil evaluation application ( Fig. 1) was incorporated by school monitoring personnel during their monthly visits to education by the School Education and Punjab Information and Technology Board. The evaluation app is connected to a comprehensive question bank and the related student learning outcomes are identified with each question [32]. Main student learning outcomes (KLOs) are currently evaluated for grade 3 learners for English (Fig. 2), Urdu (Fig. 3) and Maths (Fig. 4). The key areas measured were comprehension (Fig. 5), completing sentences, addition, and subtraction of two and three-digits, multiplication (Fig. 6), and division. The students are evaluated by a monitoring and evaluation team personnel (MEA) who also uses an android-app. A tablet is used for evaluation whereby questions for each class, subject, and main SLO are randomly rendered from a central question-bank. The student to be tested are presented with these multiple-choice questions (Fig. 5). Through exam in over 47,000 public schools in Punjab brings together a total of seven questions. It takes less than five minutes per pupil to complete the process. The monthly assessment takes place at nearly 329,000 participants. Up to now, the MEAs carried out almost 6.7 million evaluations (Fig. 7). Information is exchanged via an Internet dashboard and SMS alerts [32] with training administrators. *Corresponding Author www.ijacsa.thesai.org

Motivation of this Research
Due to the continuous popularization and increase in the usage of smartphones, patrons prefer to use the enhanced features of mobile applications for their studies [26] [2]. Therefore, this study emphases on the LND app (Shown in Fig. 1) adopted in public sector schools of Punjab province. Since its commencement, this app has been downloaded & installed in 52,394 schools and in use for learning and assessment but till now, no study has been made for its usability either this app encounters a problem or it needs improvement. Researchers intend to assess whether or not the LND app can be stably operated on various terminals in the present study. Current usability studies have been recognized as important in the evaluation of goods and systems [21] but for the LND app, no test of usability has been developed since its enforcement. Therefore, these tests need to be conducted among educators to collect feedback and counterpoints and to recognize how students and teachers feel while visiting the LND mobile app at school. In order to measure performance, efficiency and user satisfaction, utility analysis is performed on the LND App, which is implemented by schools of Punjab public sector [60].
The rest of the article has been structured as follows. Section 2 presents the relevant literature. While the methodology used to evaluate the usability and effectiveness of LND has been presented in Section 3. The analysis and results of the study have been presented in Section 4. Discussion on the results and recommendations for improvements have been presented in Section 5. Lastly, Section 6 concludes the article.

A. Usability
Usability was initiated at the end of the eighties [7] and is commonly used in measuring product and services quality and acceptance [54], [17]. While usability study results might not be the most efficient strategy for site assessment, they provide dependable quantitative projections for user satisfaction and (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 1, 2020 356 | P a g e www.ijacsa.thesai.org performance [1]. In fact, existing mobile usability studies [57], [22] have identified nine generic usability attributes: User satisfaction, Simplicity, learnability, effectiveness, errors, efficiency, memorability, comprehensibility, and performance of learning. These usability characteristics help to evaluate the quality of certain products and user-friendliness. The current usability research is among these qualities, aimed at measuring user experience, efficiency and quality with LND apps used in public sector schools of the province of Punjab.
Such three attributes are frequently taken to check the user experience of the mobile devices used for learning purpose [5], [12], [13], [15], [31]. Effectiveness means integrity and consistency in the achievement of certain objectives by specified users. Efficiency corresponds to the fast and effective manner in which users can perform the task or the tools they need to accomplish their objectives with respect to reliability and completeness [51], [53], [47]. Satisfaction with users reflects users ' position on the use of mobile applications.

B. Usability Testing
Small tests are usually used in usability research. Five participants were found to have a total of 85% of usability issues, especially when these users are [39], [20]. Nonetheless, in recent studies, large numbers were employed to study the functionality of mobile apps in light of the diversity of mobile devices and platforms. Twelve participants, for example, were requested to check the functionality of their website [40], [53], [18], 12 more were willingly hired to evaluate the usability of another related website [55], [36] and to a re-designed mobile portal of the library of Health Sciences was reclassified to 10 participants [44], [35].
In order to measure the functionality of mobile applications two main methodologies [57], namely field studies and laboratory experiments, are implemented. Field experiments are favored to laboratory tests because the actual and changing world in which operations are usually carried out is preserved. The user experience acquired in a real-world situation is, therefore, more accurate and practical than in a laboratory environment. Nonetheless, three main difficulties in performing field studies have been identified in the literature [16]. Real studies, far from trivial to use established assessment techniques, were difficult to establish and data collection was not easy. Laboratory studies often benefit from several benefits relative to field studies, including complete control over tasks, simple performance assessments and the ability to capture usage patterns using video [57].
The significance or utility of experimental and mobile research variations was not defined by agreement in previous literature [4], [43], [56]. Nonetheless [56] claimed that field research is better suited for the usability testing of mobile applications. [34] concluded that repetitive and expensive conventional laboratory testing is often incapable of representing practical uses. [53], [54] stated that lab experiments may be helpful but cannot test people's behaviors in conditions in real life. Actual mobile apps should, therefore, be used as far as possible [57]. It is important to determine the actual experience and behavior of consumers with a real environment because such interactions help to improve the mobile LND application.

C. Usability of the Mobile Educational Applications
We reside in the mobile technology era and numerous people have smartphones which are often used for internet surfing. Worldwide, five billion people use mobile phones. The number is impressive and a new mobile segment is increasing rapidly including tablets and phones. Kids use items like the iPad and tablet [24]. Their use is widespread among kids. Digital education is the way we learn, everywhere and wherever. Mobile devices support it, and it includes student and content mobility in the sense that it is available from everywhere or at any time [10]. Most software apps are commercially available for small children, and the growing popularity of digital teaching for childhood has brought about a fresh wave of mobile education. In the United States, 88 percent of public schools have an appropriate student usage plan for transferring telephones, according to a study [10], [50].
The usability of mobile apps is an important part of it and it can also be prevented through properly designed software [29]. Usability testing for mobile apps is an evolving research field, which is difficult because of its unique features, such as the smaller screen width, restricted input space, and shifting user contexts [22], [29].
A detailed review of the functionality of the mobile learning software was made in response to the disappointment voiced by students at the National University of Fiji. The study was carried out by 30 students who tested the system to make it simple to use, efficient and satisfactory. The findings showed minor problems in usability and suggestions for further development [27], [28]. The researchers analyzed students ' understanding of the effectiveness of Google Apps for Education (GAFE), how it was challenging or simple for students to communicate with GAFE and how students perceived software as a specific benefit in terms of accessibility and efficiency [6], [28].
A Primo discovery tool usability test was performed which is a medium-sized library research tool to detect patterns in user search behaviors. The researchers explored key elements of the design and features of Primo on the basis of specific research questions. In order to understand the usability of Primo for users, a diagnostics usability evaluation was conducted. Display behavior, graphical gestures and verbal commentaries were analyzed in investigators, which helped to test software usability. The study concluded many problems faced by users (respondents who misunderstood themselves in searching primo features, query boundary labels) and technologically (contradictory results compared to search indexing methods, documents obtained from digital library inhouse were not shown properly) [38], [27].
CAMEG is created to make learning more appealing to the students in an interactive game-based mobile application. The researchers measured the utility of student interpretation of the play. The researchers have taken Management Information System (MIS) courses in a game developed a digital science park in 03 labs and questioned approximately 23 groups of between three and four pupils to fill out the CAMPRG usability survey. The researchers have collected user reviews and proposed that software be best used [33], [28]. www.ijacsa.thesai.org MOSAD was developed using a template material from the Universiti Teknologi PETRONAS (UTP) course System Analysis, Development and Science (SDS). In order to improve efficiency and usability, a heuristic analysis with five experts in the field of human-computer interaction (HCI) was carried out, accompanied by the Post Analysis Quasiexperimental design performed by 116 UTP participants of second years testing the efficacy and usability of MOSAD in the field of SAD. Findings above 3.5 for each compound, based on the rating 1-5, were also given by the usability test which measured the uniqueness, the acquirability, the durability, the minimal behavior and the minimum memory load. The results thus show both that MOSAD is beneficial and reliable for tertiary education pupils as an instrument for the revision of education [23], [27].
As far as our study is concerned, usability is an important aspect to make the game-based learning especially gamebased mobile learning effective or efficient. Hence, usability is a key quality attribute for learning. The success of game-based mobile learning requires positive student attitudes to mobilelearning, together with suitable usability and good user experience in the systems [49]. With the use of better technology in smarts phones, a number of mobile applications have developed but in most of the applications, usability is not primarily focused, which is the main reason for the failure of such applications [48].
Literacy and Numeracy Drive (LND), a well-known mobile-based application adopted by public sector schools in the Punjab province for grade 03 students (male and female). Given its widespread use in over 52,000 schools [41], though, no research was conducted on the usability of this mobile app. Usability work leads to enhancing the user experience of mobile LND apps and the approach and assessment findings offer useful guides to enhance participant and technical user experience to achieve good outcomes.

A. Setting and Sample
This study took place in public sector primary schools of Pakistan which is consisted of four provinces, Punjab, Khyber Pakhtunkhwa, Balochistan, and Sindh. The Punjab province is the most populated area of Pakistan selected for this study which is further divided into 36 districts. It has 52,394 schools (male and female), 12,268,981 students (male and female) and 403,172 teachers (male and female). Furthermore, District Sheikhupura is selected for the survey and interviews of the teachers, which is sub-divided into 05 regions (Ferozwala, Muridke, Safdarabad, Sharaqpur, and Sheikhupura) and students for survey only from the same region. It has 1,247 schools (male and female) out of which 21 schools were randomly selected to visit in whole regions. This study is to measure the usability of the LND app, from the teachers and students in the school who were using LND app in the class for teaching and students who were using LND app in class and/or at home for learning. The sample was drawn from teachers of 21 randomly selected schools out of 1,247 from District Sheikhupura and 57 teachers who were teaching English, Math and Urdu subjects, also using LND application in the class were invited to participate in the study by accomplishing questionnaire and interview in class timing. A strength of 300 students who were using the LND application at home and at school was invited to complete the questionnaire. The rate of response was 100% because all teachers and students participated happily.

B. Participants
For the study of teachers, 57 respondents participated from which male respondents were 12 (21.1%) and the majority of respondents were female which was 45 (78.9%). The age group of 18 (31.6%) participants were 26-30 which is the majority whereas 09 (15.8%) respondents having the age group of 31-35 whilst 13 (22.8%) and 17 (29.8%) respondents have the age group of 36-40 and above 40 respectively.
For the study of students, 300 respondents participated from which male respondents were 138 (46.0%) and the majority of respondents were female which was 162 (54.0%). The age group of 06 (2.0%) participants were 05-07, the overwhelming majority of 217 (72.3%) participants belonged to 08-10 age group whereas 76 (25.3%) participants having the age group of 11-13 whilst only 1 (0.3%) participant was above 13 of age.

C. Instrument
The questionnaire includes a total of 40 questions was developed and reliability was measured by Cronbach's alpha which was (.856) that indicates a high level of internal consistency. The questionnaire includes five factors: demographic information, User Experience, Ease of Use, and Usefulness of application was measured. In the students' questionnaire having 30 questions that include: Demographic information, Availability of electronic devices, User Experience, Usefulness, and Ease of use for LND application was measured. The reliability of the questionnaire was measured by Cronbach's alpha which was (.805) thus indicating that the questionnaire survey was vastly reliable and stable.
For the teachers section of demographic information: age, gender, education, and location of school was asked from the participants whereas in the section of Usability and User Experience: icon of application is easy to find, icons and buttons are eye-catching and identifiable, Color scheme of button is attractive, Interface is easy to use, font size is easy to read, easy to use touch screen, easy to use input, step by step assistance, difficulty, performance and improvement, navigation keys, voice instructions, animations, videos, recommendations of questions, question bank and advertisements; in the section of Usefulness: enhance of vocabulary, saves my time, pictorial presentation, improvement of knowledge, skillful in learning and confident in speaking was asked where all items based on 5-point Likert scale, ranging from 1-Strongly disagree to 5-Strongly agree.
For the students' section of demographic information: age, gender, residential area, and location of the school was asked from the participants where the language was the first question then the availability of devices like mobile phones, smartphone, tablets, laptop, and computers at home or at school was asked. In the next section, the Availability of electronic devices, User Experience, Usefulness, and Ease of www.ijacsa.thesai.org use of the LND app was asked. The questions having yes/no options were asked from the students because they were unable to make the decision on a Likert scale due to their educational capability and age group.

D. Data Collection
The data was gathered during the winter session in 2018 and the questionnaire was administered to all English, Math, and Urdu subject teachers and from the student side, only grade 03 students were focused from the public sector schools in which the participation of respondents was voluntary and anonymous. For the analysis of questionnaire data, Statistical Package for Social Science (SPSS 25.0) was used in which frequencies were generated for demographic information of the participants whereas for the sections of LND (Usability, User Experience, Usefulness, and Ease of use), Mean and Standard Deviation were calculated accordingly.

A. Teachers Result
After distributing the questionnaire to the teachers in public sector schools of Punjab province, following are the results compiled by the SPSS (version 25.0) and presented in the form of a table and interpreted as well: According to the demographic information in Table I. and  Table II Education of the respondents is the important factor for thinking / suggesting if the respondent has higher level of education will provide appropriate answer for the question asked and the   Note: Scale ranging from 1-Strongly Disagree to 5-Strongly Agree Usability of LND: The majority of the respondents in Table V show the lowest mean score for all items except item 07. The role of usability in mobile applications considered vital because better the usability leads to the smooth and effective use of an application. From the item 1 to 10 in the table depicts the problems faced by the respondents while using LND application in which item 09, and 10 has the lowest mean score for non-availability of selfrecommendations questions and steps to improve the application for ease to use by the respondents. Item 03 depicts www.ijacsa.thesai.org the problem of input by (90.3%) majority where ( = 1.56 = .50) and no assistance availability by (93.5%) majority where ( = 1.45 = .50) for the difficulty faced by respondents in item 05. Advertisements in an application have a bad impression and it affects the usage of the application for the meaningful purpose which is described in item 7 (highest mean score), where (91.2%) respondents ( = 4.91 = .29) strongly agrees that the LND application displays so many advertisements that distract the process of teaching and learning of the respondents.
Ease of Use: From items 11 to 20 of Table VI shows that respondents have the problem for using the LND application easily and efficiently. Font size is a major element for consideration while reading text on the mobile-based application and if the font is not appropriate for reading then it is difficult to use such application. For the purpose, item 12 in the table, explains that the majority of respondents (80.7%) where ( = 1.94 = .44) are not agreed with the font size used in the LND application because the respondents felt difficulty while reading the text from the screen of the application. The performance of application also has an effect on the smooth and motivational use of it because if the application performance is slow while performing the task then the user will not learn effectively in due time. Item 17, in which (98.4%) majority where ( = 4.75 = .43) strongly agreed that LND application performance is slow while using it into the class.
For items 18 to 20, the mean score is the lowest for the voice instruction, animations and videos for learning because these elements motivate the respondents for better and ease of learning with better results. In these items the overwhelming majority (100%) where ( = 1.00 = .00) strongly disagreed that the LND application provides voice instructions, animations and videos for learning. Usefulness of LND: The purpose of the mobile-based application is to enhance the knowledge for meaningful purpose with specific content. But according to Table VII for LND usefulness, all the items have low mean score and item 01 describes that (96.5%) majority respondents where ( = 1.54 = .75) disagreed that the LND application enhances the vocabulary of the respondents. The pictorial representation of concept makes the topic easy and more useful that motivates the respondent towards its use but in this table, the lowest mean score is with item 03 in which the majority (93.0%) respondents where ( = 1.07 = .25) disagreed that the LND application provides any pictorial presentation of concepts.
Equality is necessary for using technology between students in the class, so every student can get proper time with equal access to the application to learn. Item 07 and 08 describes that (96.5%) majority where ( = 1.61 = .82) disagree that each student gets equal access & time for the practice of the LND application in the class because every school is allowed to have only 01 tablet in the huge strength of the class. The weakness is an element that helps a respondent to improve it after getting some evaluation using some technology automatically. If the application provides such feature then it will be much effective towards interactive and efficient usage but in LND, the overwhelming majority of the respondents (98.5%) where ( = 1.25 = .43) disagreed that the LND application provides weakness of the students on the basis of results.

B. Students Result
After distributing the questionnaire to the students in public sector schools of Punjab province, in which some teacher helped students to solve the questionnaire according to their answers, following are the results compiled by the SPSS (version 25.0) and presented in the form of the table and interpreted as well.
According to Table VIII. Age of only 06 (02%) respondents was in the group of 05-07 whereas the majority of the respondents 217 (72.3%) were in the age group of 08-10. In the age group of 11-13, 76 respondents which are (25.3%) and lastly only 01 respondent (0.3%) belongs to the above 13 age group. According to Table IX. Out of 300 respondents, male students were 138 (46%) and the majority of respondents were female which were 162 that is 54% for a total of 100%.
According to the Table X. only 18 (6%) respondents have the tablet at their home whereas majority of the respondents 227 (75.9%) have simple mobile phone at their homes to use but 186 (62.2%) majority of respondents have smart phones at their homes to use for playing games or educational purpose. Only 16 respondents (5.4%) and 100 (33.4%) respondents have laptop and computers respectively at their homes to use it for playing games, watching cartoons or educational activities.
According to Table XI, in the schools, 100% of respondents have tablets and computers to use in educational activities because Government of Punjab provides 01 tablet and 16 computers to every public school in the whole province whereas the access of computer is only for Grade 9 th and 10 th www.ijacsa.thesai.org students for their learning activities according to the instructions of teachers.
According to Table XII. In which only 72 respondents have computers or access to computers at their homes further responded by 67 (93.1%) respondents who use a computer for playing games at home and 58 (80.6%) respondents use a computer for watching cartoons but only 8 (11.1%) respondents use computer for their homework at homes.   According to Table XIV, the item 15 describes that only 62 (20.7%) respondents think LND software is suitable for learning English but majority of the respondents 238 (79.3%) do not think LND is suitable for learning English because overwhelming majority 283 (94.3%) of respondents think the LND application is not interesting. The interactive screen with suitable color scheme attracts the users to use the mobile application for learning but respondents said that LND application does not have an interactive screen, colors scheme, navigations and animations whilst 280 (93.3%) of respondents responded that they cannot learn English easily with the help of LND application. The curriculum of mobile-based application must be suitable and appropriate for learning topics effectively but it is reported by 283 (94.3%) respondents that the content used in the LND application is not suitable.
The suitable content/curriculum defined by Government department/officials is necessary for the learning of topics in the class but in the absence of suitable content, there will be a bad effect on learning as well as the assessment of pupils. The item 20 and 21 describes that majority 289 (96.3%) of respondents cannot learn comprehension easily with the help of LND application because this application is not providing suitable content automatically which is also proved by 100.0% of majority. This is also the reason that 99.0% of the majority did not take interest to learn comprehension using this LND application.
Assessment is the necessary important element in the class to see the progress of the student and with an appropriate assessment method, the teacher may know that to what extent students learned the topic effectively. The majority of the respondents 200 (66.7%) are not agreed with the assessment method used by LND because the assessment is not taking www.ijacsa.thesai.org part to improve comprehension for the English language which is responded by 283 (94.3%) respondents. The application also does not provide the content to learn comprehension on the basis of result which is justified by 100.0% of majority and 99.7% of respondents say that the LND application does not provide the weakness to learn comprehension for the pupil. According to Table XV, the majority 283 (94.3%) respondents said that the LND application does not have interactivity in screen or buttons to attract the pupils to use this application with the passion to learn English. It is also mentioned by the majority of respondents 218 (72.7%) that content and exercises are not properly available for the practice of learning English. Animated videos and voice pronunciations attract the pupils to learn effectively and also can produce good results but according to 298 (99.3%) and 291 (97.0%) respondents, these features are missing in LND.

V. DISCUSSIONS AND RECOMMENDATIONS
After conducting a survey from teachers and students to see the usability of LND in public sector schools for teaching and learning for students of grade 03, several problems were identified and the teacher also recommended some improvement for mobile-based application LND to overcome all the issues.
 Game-based feature should be introduced to enhance the user experience of students and teachers in current LND app which is also described by [19] and using game-based features, challenges also be included for the pupils to enhance the usability and to keep focus while on good and interactive design for learning application which is also mentioned in [46].
 The design of the interface should be accurate that have a good impact on the success of the learning application and to develop the reading and to encourage the reading by playing video games. This is also described by [30]. The presentation of learning material through games and animations inspire the pupils to improve comprehension and also to gain cognitive skills. This has been described by [11] in their study.
 Ease of use, usefulness, and clarity of goal in the gamebased application that inspired the learners for using games. The designers must keep in mind these factors while designing and teachers also need to integrate these factors to get the results better for encouraging behavior from learners by participating in such learning activities. This also mentioned by [52] in their study. www.ijacsa.thesai.org  The feature of music in game-based learning applications should be incorporated, which helps the user to develop their motivation towards better learning. It is also necessary to add a feedback feature to improve the learning capability through a tactile observation of substances. These points are also described by [3] in their study.

VI. CONCLUSION
In the coming years, the development of a smartphone LND app is projected to increase exponentially. Smartphones are becoming more and more popular, and both school and house learners practice using this software by downloading them from smartphones of their parents. Usability assessments evaluate not only the performance, efficacy and customer engagement of mobile application but also provide multiple enhancement recommendations based on user experience. These results will surely make a significant contribution to designing and improving such systems. In fact, improving the functionality of the mobile application tested will also encourage people to use the LND mobile app and significantly enhance the quality of this product. There is a strong need to improve such games to increase their support in teaching and learning. Furthermore, they can lead to developing applications for personalized learning support at a massive level.