Validating the Usability Evaluation Model for Hearing Impaired Mobile Application

—Usability is an important element that enables the identification of the efficiency for application or product. However, many applications have been developed for general users’ needs and are unable to provide adequate applications usage for disabled people. This study focuses on the development of usability evaluation model and the validation process on the proposed model through experts. The developed model later evaluated by group of experts through focus group method. Focus group method enables to identify the 13 variables derived to develop the model are appropriately placed and useful in the evaluation process. The results shows that the selected variables are appropriate to identify usability of mobile application for the hearing impairment through three variables tested namely, gain satisfaction with the model, satisfaction with the model presentation, and support for tasks. Conclusively, the developed model can identify usability of mobile applications for hearing impairment and enable in identifying useful criteria to be included during application development process in real life process. As future study, the model can be tested among the hearing impairment people and practitioner to establish the results obtained which contributes to usability practitioners and application developers for the disabled.


INTRODUCTION
Usability is an important element in any system or application to analyse any incurring usability issues. Usability commonly referred as a guideline for measuring usability of the system using models. These models provide insight into measurements to be used for usability analysis data collection [1]. Models such as International Standard Organization (ISO) 9241-11 [2] and Nielsen"s (2005) [3] model are among the common usability evaluation models that will be used in user evaluation to identify the issues [1]. However, when a specific targeted user application is developed, the user's requirement must be incorporated into the application. If the requirement is missing, the application will fail to satisfy the user and becomes difficult to use. Different disabilities having different levels of cognitive and mental strength [4][5] [6]. Thus, applications developed for these people should consider these issues to ensure the usefulness of such application. Mobile devices are proliferating at an incredible rate. Statista (2022) [7] releases that currently, smartphone users have increased up to seven billion worldwide whereas almost 80% of the population in the world is owning a smartphone. These rates are extraordinary, given to relative introduction of many applications in mobile that eases people"s daily life.
As such, this mobile application industry is growing remarkably, and the penetration of mobile apps can be seen growing in foreseeable future. Mobile phones once have only been used for answering calls now have grown up for muti use among the users. Many have gained benefits from various use of the mobile phone [1]. As this is the case, mobile phone does not bound only for the normal people but benefits the disabled people around the world as well. The compatibility of these mobile phones and its applications are being studied continuously as an objective to enhance the usability of all type of people as said in [1] to be more useable.
Usability has been a pivotal part of the discussion in many domains [8] [9]. Usability evaluation is being conducted to measure an application satisfaction achieved by the users and commonly measured subjectively which is a collection of satisfaction rating of application among users [10]. However, this method is less defined [10]. Few renowned usability models are used as references to measure the applications such as ISO 9241-11 (1998) and Nielsen"s (2005) [3] model although many other usability models have been constantly studied thereafter.
The focused issue is about the application developed for special people which is rather challenging compare with applications for normal people. Measurements used for general purpose application are unable to measure important features of the needs of these disabled users [1] [11].
According to a survey conducted in the United States, the number of hearing-impaired users using smartphone applications is in the second highest percentage (i.e. 31%) after physically disabled people [12]. This shows that the hearing-impaired are one of the major disabled users of the smartphone and its applications. The hearing-impaired are those whose hearing is impaired to some degree at the time of birth [13]. These people are unable to produce coherent speech due to the lack of auditory input and thus the inability to monitor their voices [1] [14]. They rely on sign languages or any methods with gestures for communication purposes [15] [1].
As such, a study was conducted and a usability assessment model for the hearing -impaired, called Model for Mobile www.ijacsa.thesai.org Application Evaluation for Hearing Impaired (MAEHI), has been proposed to overcome these deficiencies [1]. This model proposed consists of six dimensions, 15 criteria and 47 metrics. This study will conduct the expert review methods for the developed MAEHI model for verification process to ensure the reliability of the proposed MAEHI model that has been developed.
The verification of the model performed through the method of expert review. The expert review was conducted to verify the proposed MAEHI for usability assessment of mobile applications with hearing impairment. One of the important ways to detect and remove defects is by expert review [16]. This study therefore adopted this technique to verify the proposed MAEHI. All the components developed, and the appropriate organization and presentation can be confirmed by verification [17].
This paper consists of the verification process of the model that was conducted to ensure that the model needs to be met for the deaf user. As such, the next section consists of discussing the focus group method for the assessment of the model followed by section three discussing the results of the focus group and section four discussing the conclusion.

II. METHODS
The main aspect to be verified in the proposed MAEHI is the use of the appropriate dimension, criteria and metrics and the overall applicability, originality, and comprehensibility of the proposed dimension in MAEHI model [1] as shown in the Fig. 1, which is categorized into five components: consistency, ease of use, comprehensible, verifiable, and overall impression. Among the 15 dimensions that were shortlisted, only six were chosen for final model development of MAEHI [1]. These criteria with the dimensions are designed in the model form as full model version and presented in [1].
Potential usability experts were therefore identified for the validation process of the model in Fig. 1, particularly researchers, academics, application developers or practitioners, as well as disability experts, particularly on the hearing impaired. As suggested by Hallowell, and John Gambatese (2009) [18] and Rogers, Margaret, and Emilia Lopez (2002) [19], the expert was selected. Once the model was verified, the validation was carried out to ensure that the model developed can provide adequate results in evaluating the deaf application's usability.
Domain users and experts from Malaysia National Research and Development, User Experience Lab, Malaysian Institute of Microelectronic Systems (MIMOS) participated in the MAEHI evaluation. This is a well-known software development organisation with numerous technology-focused areas and the only usability laboratory certified in Malaysia with MS ISO / IEC17025:2005 [20]. As part of this study, the model evaluation with experts and usability testing was conducted in cooperation with MIMOS. Experts validated the MAEHI in terms of its ability to be used in real testing environment through the focus group discussion.
The next section is implementing the focus group activity which includes planning, conducting, and analysing the outcome [21] [22]. The objective of the focus group is to evaluate and validate the MAEHI in terms of its ability in producing reliable results to be implemented in real-world environments. As such, conduct planning is made up of a few steps as it is important to ensure that the focus group is properly conducted.
Planning was carried out thoroughly to ensure a good implementation of the focus group. There are four activities for the focus group implementation, adapted from Mazza, Riccardo, and Berre [23] to define focus group objectives, identify participants, prepare materials, and schedule meetings. Each activity is further explained in the section below.

A. Defining Objective of Focus Group
In particular, the focus group's objective is to evaluate and validate the MAEHI in terms of its ability in producing reliable results to be implemented in real-world environments.

B. Identification of Participants Defining Objective of Focus Group
At the MIMOS National Research and Development Center, Kuala Lumpur, participants for the focus group were selected from a group of application developers and usability practitioners. This study, however, only considers practitioners with more than three years of experience in this field [24]. Through emails and phone calls, 11 software developers and usability practitioners were approached. However, only seven were willing to participate as the others were involved in the organization's projects. The seven participants are within the recommended range for a focus group [25] (i.e., between six to ten members). The focus group was organised for the convenience of the expert group at MIMOS User Experience Lab in Kuala Lumpur.

C. Meeting Scheduling for Focus Group
The meeting of the focus group was scheduled at the Kuala Lumpur MIMOS User Experience Lab. The meeting place was chosen within the organization itself to accommodate its busy schedule with organizational projects. Since the venue was within the organization itself, it is convenient for the participants [26]. www.ijacsa.thesai.org

D. Material Preparation for Focus Group
Based on the activity to be carried out, the material used for the focus group was prepared. Materials such as participant documentation for evaluation have been prepared earlier to ease the focus group process.
The focus group was conducted with the developer"s group and usability practitioners as scheduled. Upon arrival, all the participants were warmly and informally welcomed, as some arrived earlier than the scheduled time. This creates a space for knowing much better about the field in which they work and makes them feel relaxed before the formal discussion begins. Next, the materials needed for the session of the expert focus group were given. A brief introduction to the focus group objective was mentioned and the focus group purpose was reminded to experts. They were briefed on the model for validation purposes to be evaluated. The explanation given was quite easy for them as they were from the usability domain. It took nearly two hours to discuss the entire model assessment process, which is a common and acceptable duration [23] [24].
All participants tested the developed MAEHI in the mobile application they downloaded from the app store on their smartphones during the evaluation process. Experts filled out the evaluation form to indicate their satisfaction with the ability of the developed model to be implemented in usability measurements in the real-world environment.

E. Instrument Development
The experts validated the developed model through an evaluation with predefined criteria adapted from past studies [27]. These criteria include gain satisfaction with the model, satisfaction with the Model presentation, and support for tasks. In the first criterion that is gain satisfaction with the model variables were "Relevance for the intended application," "Perceived usefulness," and "Clarity". This criterion is to reveal a model or framework's accuracy. The second criterion that is satisfaction with the Model presentation is represented by two variables that are "Ease of Use" and "Organization". This allows the experts to determine whether the model developed is practical and easy to evaluate and well organized for usability [28].
In the third criterion, four variables are used to measure task support satisfaction, where they were tested on "Practicality," "Completeness," "Incomprehensibility" and "Ability to produce expected results." These variables allow the model's understandable relevance to imply an evaluation and completeness of the application [29]. Each criterion will be assessed based on its own variables with two scales in, "Agree" and "Disagree."

F. Data Analysis
Once the evaluation is completed by the experts, the forms are collected, and its data is analysed by obtaining the ratio value for each item and an overall proportion of the measures to identify the experts" level of agreement on the model. The model evaluation findings will be further discussed in the next section.

III. RESULTS AND DISCUSSION
After testing the developed model and discussing its ability to be used in the real-world environment, The results of these measures were the endorsement in the real-world environment of the validity of the proposed model. Consequently, each criterion and the results will be discussed as below.
Through this, the model has been validated by the experts in executing for usability assessment use on its relevance. Table I shows the experts" feedback.   TABLE I. EXPERT AGREEMENT ON GAIN SATISFACTION WITH THE MODEL

Variables % Results
Relevancy to the intended application 86% Experts agreed that the model developed is relevant since it focuses on targeted users for accessibility. They also found that the developed model appropriately highlighted accessibility.

Perceived usefulness 86%
Experts agreed that the usability model developed is suitable for the evaluation of hearing-impaired mobile apps. Furthermore, all experts agreed that this usability assessment made it possible to identify problems for hearing impaired users as the existing usability models are not able to meet their needs.

Clarity 100%
All the experts agreed unanimously that the phases provided for the developed model were clear in all the evaluations and tasks. The method provided for both data calculation and collection were agreed for use as clear.
The results of the expert evaluation validation were shown in Table I for their satisfaction gain with the model. Based on the experts" view, the developed model is identified as being useful for evaluating the usability of the mobile application with hearing impairment [30]. In addition, expert feedback also revealed that the model developed is clear for use with real users and can identify usability -related issues especially for hearing impaired users.
The expert feedback based on the, Satisfaction with the Model presentation is outlined in Table II.   TABLE II. EXPERT AGREEMENT ON SATISFACTION WITH THE MODEL

Variables % Results
Perceived Ease of use 86% Experts agreed on the model's ease of use for the intended application. However, one expert was not convinced that this evaluation could be carried out on the hearing impaired as they would face communication problems during testing as they would need a translator if they communicated with normal hearing people.
Organization 86% Experts agreed that the model developed was well organized and satisfied with the way all the procedures to be evaluated were organized in terms of their structure sequence and understandability arrangement. www.ijacsa.thesai.org According to Table II, recorded by the experts" feedback, they found that the usability model was easy to implement in the real environment, apart from well-organized measurements. The experts were pleased to find the model suitable for the evaluation of hearing-impaired mobile apps. There was, however, a slight reluctance to collect data with the hearing impaired as communication barrier is an important issue for usability practitioners to understand [31]. Table III shows that the model is practical for the hearingimpaired usability assessment to be conducted. The model is also comprehensive and easy to understand and implement. In addition, the results show agreeable views that the model developed can produce adequate results.

Practicality 100%
Experts showed some reluctance to practice with disabled people on the real environment. They agreed, however, if the evaluation was carried out using a controlled method, in which it would be possible. The other experts mentioned that the metrics and criteria were suitable and that expected results were easy to obtain.

Completeness 86%
Experts found the model to be appropriate in the use of usability assessment for hearing impaired mobile applications.

100%
The proposed model was viewed in an understandable way as all the experts agreed that it was a well -organized evaluation and that it was easy to understand, especially for those with usability experience.
All the experts agreed on the proposed model that it could produce expected results. They expressed satisfaction with the conduct of the evaluation and the dimensions covering the requirements of users with hearing impairment. In addition, experts also agreed that the model could produce expected results and identified usability issues in the evaluation of the application.
A summary of all the percentages obtained and calculated on average identifies the overall score for each criterion. For all variables, the overall percentage of agreement for each criterion is above 86%, indicating a high percentage of agreement on the model and acceptance of the model's applicability in the real-world environment. Thus, these results show that the proposed model is practical for the usability assessment of hearing-impaired mobile applications to be implemented in the real-world environment.

IV. CONCLUSION
Experts delightfully wanted to participate with the actual hearing-impaired users in the usability test to further evaluate the model. This also demonstrates the experts' eagerness to directly test the model developed in the real environment. They also shared the intention to collaborate with the hearingimpaired with this developed model in future testing and usability evaluation as they found that the model is particularly useful when the user is disabled. Emphasis has been given to the fact that usability assessment model is less important for disabled user applications, which many developers tend to ignore. The developed model will be used for usability testing for future study to demonstrate the practicality of use in the real environment and to identify the model's ability to collect useful analytical data.